We analyze the convergence of the harmonic balance method for computing isolated periodic solutions of a large class of continuously differentiable Hilbert space valued differential-algebraic equations (DAEs). We establish asymptotic convergence estimates for (i) the approximate periodic solution in terms of the number of approximated harmonics and (ii) the inexact Newton method used to compute the approximate Fourier coefficients. The convergence estimates are deter-mined by the rate of convergence of the Fourier series of the exact solution and the structure of the DAE. Both the case that the period is known and unknown are analyzed, where in the latter case we require enforcing an appropriately defined phase condition. The theoretical results are illustrated with several numerical experiments from circuit modeling and structural dynamics.
We propose new approximate alternating projection methods, based on randomized sketching, for the low-rank nonnegative matrix approximation problem: find a low-rank approximation of a nonnegative matrix that is nonnegative, but whose factors can be arbitrary. We calculate the computational complexities of the proposed methods and evaluate their performance in numerical experiments. The comparison with the known deterministic alternating projection methods shows that the randomized approaches are faster and exhibit similar convergence properties.
In this work, we are interested in building the fully discrete scheme for stochastic fractional diffusion equation driven by fractional Brownian sheet which is temporally and spatially fractional with Hurst parameters $H_{1}, H_{2} \in(0,\frac{1}{2}]$. We first provide the regularity of the solution. Then we employ the Wong-Zakai approximation to regularize the rough noise and discuss the convergence of the approximation. Next, the finite element and backward Euler convolution quadrature methods are used to discretize spatial and temporal operators for the obtained regularized equation, and the detailed error analyses are developed. Finally, some numerical examples are presented to confirm the theory.
A homogenization approach is one of effective strategies to solve multiscale elliptic problems approximately. The finite element heterogeneous multiscale method (FEHMM) which is based on the finite element makes possible to simulate such process numerically. In this paper we introduce a FEHMM scheme for multiscale elliptic problems based on nonconforming spaces. In particular we use the noconforming element with the periodic boundary condition introduced in the companion paper. Theoretical analysis derives a priori error estimates in the standard Sobolev norms. Several numerical results which confirm our analysis are provided.
We study the implicit upwind finite volume scheme for numerically approximating the advection-diffusion equation with a vector field in the low regularity DiPerna-Lions setting. That is, we are concerned with advecting velocity fields that are spatially Sobolev regular and data that are merely integrable. We study the implicit upwind finite volume scheme for numerically approximating the advection-diffusion equation with a vector field in the low regularity DiPerna-Lions setting. We prove that on unstructured regular meshes the rate of convergence of approximate solutions generated by the upwind scheme towards the unique solution of the continuous model is at least one. The numerical error is estimated in terms of logarithmic Kantorovich-Rubinstein distances and provides thus a bound on the rate of weak convergence.
The asymptotic stable region and long-time decay rate of solutions to linear homogeneous Caputo time fractional ordinary differential equations (F-ODEs) are known to be completely determined by the eigenvalues of the coefficient matrix. Very different from the exponential decay of solutions to classical ODEs, solutions of F-ODEs decay only polynomially, leading to the so-called Mittag-Leffler stability, which was already extended to semi-linear F-ODEs with small perturbations. This work is mainly devoted to the qualitative analysis of the long-time behavior of numerical solutions. By applying the singularity analysis of generating functions developed by Flajolet and Odlyzko (SIAM J. Disc. Math. 3 (1990), 216-240), we are able to prove that both $\mathcal{L}$1 scheme and strong $A$-stable fractional linear multistep methods (F-LMMs) can preserve the numerical Mittag-Leffler stability for linear homogeneous F-ODEs exactly as in the continuous case. Through an improved estimate of the discrete fractional resolvent operator, we show that strong $A$-stable F-LMMs are also Mittag-Leffler stable for semi-linear F-ODEs under small perturbations. For the numerical schemes based on $\alpha$-difference approximation to Caputo derivative, we establish the Mittag-Leffler stability for semi-linear problems by making use of properties of the Poisson transformation and the decay rate of the continuous fractional resolvent operator. Numerical experiments are presented for several typical time fractional evolutional equations, including time fractional sub-diffusion equations, fractional linear system and semi-linear F-ODEs. All the numerical results exhibit the typical long-time polynomial decay rate, which is fully consistent with our theoretical predictions.
Although theoretical properties such as expressive power and over-smoothing of graph neural networks (GNN) have been extensively studied recently, its convergence property is a relatively new direction. In this paper, we investigate the convergence of one powerful GNN, Invariant Graph Network (IGN) over graphs sampled from graphons. We first prove the stability of linear layers for general $k$-IGN (of order $k$) based on a novel interpretation of linear equivariant layers. Building upon this result, we prove the convergence of $k$-IGN under the model of \citet{ruiz2020graphon}, where we access the edge weight but the convergence error is measured for graphon inputs. Under the more natural (and more challenging) setting of \citet{keriven2020convergence} where one can only access 0-1 adjacency matrix sampled according to edge probability, we first show a negative result that the convergence of any IGN is not possible. We then obtain the convergence of a subset of IGNs, denoted as IGN-small, after the edge probability estimation. We show that IGN-small still contains function class rich enough that can approximate spectral GNNs arbitrarily well. Lastly, we perform experiments on various graphon models to verify our statements.
Numerical solving differential equations with fractional derivatives requires elimination of the singularity which is inherent in the standard definition of fractional derivatives. The method of integration by parts to eliminate this singularity is well known. It allows to solve some equations but increases the order of the equation and sometimes leads to wrong numerical results or instability. We suggest another approach: the elimination of singularity by substitution. It does not increase the order of equation and its numerical implementation provides the opportunity to define fractional derivative as the limit of discretization. We present a sufficient condition for the substitution-generated difference approximation to be well-conditioned. We demonstrate how some equations can be solved using this method with full confidence that the solution is accurate with at least second order of approximation.
Spectral Barron spaces have received considerable interest recently as it is the natural function space for approximation theory of two-layer neural networks with a dimension-free convergence rate. In this paper we study the regularity of solutions to the whole-space static Schr\"odinger equation in spectral Barron spaces. We prove that if the source of the equation lies in the spectral Barron space $\mathcal{B}^s(\mathbb{R}^d)$ and the potential function admitting a non-negative lower bound decomposes as a positive constant plus a function in $\mathcal{B}^s(\mathbb{R}^d)$, then the solution lies in the spectral Barron space $\mathcal{B}^{s+2}(\mathbb{R}^d)$.
In this work, we propose a simple yet generic preconditioned Krylov subspace method for a large class of nonsymmetric block Toeplitz all-at-once systems arising from discretizing evolutionary partial differential equations. Namely, our main result is a novel symmetric positive definite preconditioner, which can be efficiently diagonalized by the discrete sine transform matrix. More specifically, our approach is to first permute the original linear system to obtain a symmetric one, and subsequently develop a desired preconditioner based on the spectral symbol of the modified matrix. Then, we show that the eigenvalues of the preconditioned matrix sequences are clustered around $\pm 1$, which entails rapid convergence, when the minimal residual method is devised. Alternatively, when the conjugate gradient method on normal equations is used, we show that our preconditioner is effective in the sense that the eigenvalues of the preconditioned matrix sequence are clustered around the unity. An extension of our proposed preconditioned method is given for high-order backward difference time discretization schemes, which applies on a wide range of time-dependent equations. Numerical examples are given to demonstrate the effectiveness of our proposed preconditioner, which consistently outperforms an existing block circulant preconditioner discussed in the relevant literature.
We present a method for solving linear and nonlinear PDEs based on the variable projection (VarPro) framework and artificial neural networks (ANN). For linear PDEs, enforcing the boundary/initial value problem on the collocation points leads to a separable nonlinear least squares problem about the network coefficients. We reformulate this problem by the VarPro approach to eliminate the linear output-layer coefficients, leading to a reduced problem about the hidden-layer coefficients only. The reduced problem is solved first by the nonlinear least squares method to determine the hidden-layer coefficients, and then the output-layer coefficients are computed by the linear least squares method. For nonlinear PDEs, enforcing the boundary/initial value problem on the collocation points leads to a nonlinear least squares problem that is not separable, which precludes the VarPro strategy for such problems. To enable the VarPro approach for nonlinear PDEs, we first linearize the problem with a Newton iteration, using a particular form of linearization. The linearized system is solved by the VarPro framework together with ANNs. Upon convergence of the Newton iteration, the network coefficients provide the representation of the solution field to the original nonlinear problem. We present ample numerical examples with linear and nonlinear PDEs to demonstrate the performance of the method herein. For smooth field solutions, the errors of the current method decrease exponentially as the number of collocation points or the number of output-layer coefficients increases. We compare the current method with the ELM method from a previous work. Under identical conditions and network configurations, the current method exhibits an accuracy significantly superior to the ELM method.