We consider a generic and explicit tamed Euler--Maruyama scheme for multidimensional time-inhomogeneous stochastic differential equations with multiplicative Brownian noise. The diffusion coefficient is uniformly elliptic, H\"older continuous and weakly differentiable in the spatial variables while the drift satisfies the Ladyzhenskaya--Prodi--Serrin condition, as considered by Krylov and R\"ockner (2005). In the discrete scheme, the drift is tamed by replacing it by an approximation. A strong rate of convergence of the scheme is provided in terms of the approximation error of the drift in a suitable and possibly very weak topology. A few examples of approximating drifts are discussed in detail. The parameters of the approximating drifts can vary and be fine-tuned to achieve the standard $1/2$-strong convergence rate with a logarithmic factor.
This article studies a priori error analysis for linear parabolic interface problems with measure data in time in a bounded convex polygonal domain in $\mathbb{R}^2$. We have used the standard continuous fitted finite element discretization for the space. Due to the low regularity of the data of the problem, the solution possesses very low regularity in the entire domain. A priori error bound in the $L^2(L^2(\Omega))$-norm for the spatially discrete finite element approximations are derived under minimal regularity with the help of the $L^2$ projection operators and the duality argument. The interfaces are assumed to be smooth for our purpose.
We prove that some time Euler schemes for the 3D Navier-Stokes equations modified by adding a Brinkman-Forchheimer term and subject to a random perturbation converge in mean square. This extends previous results about the strong speed of convergence of some time discretization schemes for the 2D Navier Stokes equations. Unlike in the 2D case, in our 3D model the Brinkman-Forchheimer term enables to obtain a strong speed of convergence of order almost 1/2 independent of the viscosity parameter.
We study the optimization landscape and the stability properties of training problems with squared loss for neural networks and general nonlinear conic approximation schemes. It is demonstrated that, if a nonlinear conic approximation scheme is considered that is (in an appropriately defined sense) more expressive than a classical linear approximation approach and if there exist unrealizable label vectors, then a training problem with squared loss is necessarily unstable in the sense that its solution set depends discontinuously on the label vector in the training data. We further prove that the same effects that are responsible for these instability properties are also the reason for the emergence of saddle points and spurious local minima, which may be arbitrarily far away from global solutions, and that neither the instability of the training problem nor the existence of spurious local minima can, in general, be overcome by adding a regularization term to the objective function that penalizes the size of the parameters in the approximation scheme. The latter results are shown to be true regardless of whether the assumption of realizability is satisfied or not. We demonstrate that our analysis in particular applies to training problems for free-knot interpolation schemes and deep and shallow neural networks with variable widths that involve an arbitrary mixture of various activation functions (e.g., binary, sigmoid, tanh, arctan, soft-sign, ISRU, soft-clip, SQNL, ReLU, leaky ReLU, soft-plus, bent identity, SILU, ISRLU, and ELU). In summary, the findings of this paper illustrate that the improved approximation properties of neural networks and general nonlinear conic approximation instruments are linked in a direct and quantifiable way to undesirable properties of the optimization problems that have to be solved in order to train them.
We consider an optimal control problem for the steady-state Kirchhoff equation, a prototype for nonlocal partial differential equations, different from fractional powers of closed operators. Existence and uniqueness of solutions of the state equation, existence of global optimal solutions, differentiability of the control-to-state map and first-order necessary optimality conditions are established. The aforementioned results require the controls to be functions in $H^1$ and subject to pointwise upper and lower bounds. In order to obtain the Newton differentiability of the optimality conditions, we employ a Moreau-Yosida-type penalty approach to treat the control constraints and study its convergence. The first-order optimality conditions of the regularized problems are shown to be Newton diffentiable, and a generalized Newton method is detailed. A discretization of the optimal control problem by piecewise linear finite elements is proposed and numerical results are presented.
Three algorithm are proposed to evaluate volume potentials that arise in boundary element methods for elliptic PDEs. The approach is to apply a modified fast multipole method for a boundary concentrated volume mesh. If $h$ is the meshwidth of the boundary, then the volume is discretized using nearly $O(h^{-2})$ degrees of freedom, and the algorithm computes potentials in nearly $O(h^{-2})$ complexity. Here nearly means that logarithmic terms of $h$ may appear. Thus the complexity of volume potentials calculations is of the same asymptotic order as boundary potentials. For sources and potentials with sufficient regularity the parameters of the algorithm can be designed such that the error of the approximated potential converges at any specified rate $O(h^p)$. The accuracy and effectiveness of the proposed algorithms are demonstrated for potentials of the Poisson equation in three dimensions.
Stochastic partial differential equations (SPDEs) are the mathematical tool of choice to model dynamical systems evolving under the influence of randomness. By formulating the search for a mild solution of an SPDE as a neural fixed-point problem, we introduce the Neural SPDE model to learn solution operators of PDEs with (possibly stochastic) forcing from partially observed data. Our model provides an extension to two classes of physics-inspired neural architectures. On the one hand, it extends Neural CDEs, SDEs, RDEs -- continuous-time analogues of RNNs -- in that it is capable of processing incoming sequential information even when the latter evolves in an infinite dimensional state space. On the other hand, it extends Neural Operators -- generalizations of neural networks to model mappings between spaces of functions -- in that it can be used to learn solution operators $(u_0,\xi) \mapsto u$ of SPDEs depending simultaneously on the initial condition $u_0$ and a realization of the driving noise $\xi$. A Neural SPDE is resolution-invariant, it may be trained using a memory-efficient implicit-differentiation-based backpropagation and, once trained, its evaluation is up to 3 orders of magnitude faster than traditional solvers. Experiments on various semilinear SPDEs, including the 2D stochastic Navier-Stokes equations, demonstrate how Neural SPDEs capable of learning complex spatiotemporal dynamics with better accuracy and using only a modest amount of training data compared to all alternative models.
This work is devoted to find the numerical solutions of several one dimensional second-order ordinary differential equations. In a heuristic way, in such equations the quadratic logistic maps regarded as a local function are inserted within the nonlinear coefficient of the function as well as within the independent term. We apply the Numerov algorithm to solve these equations and we discuss the role of the initial conditions of the logistic maps in such solutions.
Identification methods for dynamic networks typically require prior knowledge of the network and disturbance topology, and often rely on solving poorly scalable non-convex optimization problems. While methods for estimating network topology are available in the literature, less attention has been paid to estimating the disturbance topology, i.e., the (spatial) noise correlation structure and the noise rank in a filtered white noise representation of the disturbance signal. In this work we present an identification method for dynamic networks, in which an estimation of the disturbance topology precedes the identification of the full dynamic network with known network topology. To this end we extend the multi-step Sequential Linear Regression and Weighted Null Space Fitting methods to deal with reduced rank noise, and use these methods to estimate the disturbance topology and the network dynamics in the full measurement situation. As a result, we provide a multi-step least squares algorithm with parallel computation capabilities and that rely only on explicit analytical solutions, thereby avoiding the usual non-convex optimizations involved. Consequently we consistently estimate dynamic networks of Box Jenkins model structure, while keeping the computational burden low. We provide a consistency proof that includes path-based data informativity conditions for allocation of excitation signals in the experimental design. Numerical simulations performed on a dynamic network with reduced rank noise clearly illustrate the potential of this method.
We present a numerical stability analysis of the immersed boundary(IB) method for a special case which is constructed so that Fourier analysis is applicable. We examine the stability of the immersed boundary method with the discrete Fourier transforms defined differently on the fluid grid and the boundary grid. This approach gives accurate theoretical results about the stability boundary since it takes the effects of the spreading kernel of the immersed boundary method on the numerical stability into account. In this paper, the spreading kernel is the standard 4-point IB delta function. A three-dimensional incompressible viscous flow and a no-slip planar boundary are considered. The case of a planar elastic membrane is also analyzed using the same analysis framework and it serves as an example of many possible generalizations of our theory. We present some numerical results and show that the observed stability behaviors are consistent with what are predicted by our theory.
We introduce a new family of deep neural network models. Instead of specifying a discrete sequence of hidden layers, we parameterize the derivative of the hidden state using a neural network. The output of the network is computed using a black-box differential equation solver. These continuous-depth models have constant memory cost, adapt their evaluation strategy to each input, and can explicitly trade numerical precision for speed. We demonstrate these properties in continuous-depth residual networks and continuous-time latent variable models. We also construct continuous normalizing flows, a generative model that can train by maximum likelihood, without partitioning or ordering the data dimensions. For training, we show how to scalably backpropagate through any ODE solver, without access to its internal operations. This allows end-to-end training of ODEs within larger models.