We explore an application of the Physics Informed Neural Networks (PINNs) in conjunction with Airy stress functions and Fourier series to find optimal solutions to a few reference biharmonic problems of elasticity and elastic plate theory. Biharmonic relations are fourth-order partial differential equations (PDEs) that are challenging to solve using classical numerical methods, and have not been addressed using PINNs. Our work highlights a novel application of classical analytical methods to guide the construction of efficient neural networks with the minimal number of parameters that are very accurate and fast to evaluate. In particular, we find that enriching feature space using Airy stress functions can significantly improve the accuracy of PINN solutions for biharmonic PDEs.
Seismic wave propagation forms the basis for most aspects of seismological research, yet solving the wave equation is a major computational burden that inhibits the progress of research. This is exacerbated by the fact that new simulations must be performed when the velocity structure or source location is perturbed. Here, we explore a prototype framework for learning general solutions using a recently developed machine learning paradigm called Neural Operator. A trained Neural Operator can compute a solution in negligible time for any velocity structure or source location. We develop a scheme to train Neural Operators on an ensemble of simulations performed with random velocity models and source locations. As Neural Operators are grid-free, it is possible to evaluate solutions on higher resolution velocity models than trained on, providing additional computational efficiency. We illustrate the method with the 2D acoustic wave equation and demonstrate the method's applicability to seismic tomography, using reverse mode automatic differentiation to compute gradients of the wavefield with respect to the velocity structure. The developed procedure is nearly an order of magnitude faster than using conventional numerical methods for full waveform inversion.
In this paper, we construct a class of Mixed Generalized Multiscale Finite Element Methods for the approximation on a coarse grid for an elliptic problem in thin two-dimensional domains. We consider the elliptic equation with homogeneous boundary conditions on the domain walls. For reference solution of the problem, we use a Mixed Finite Element Method on a fine grid that resolves complex geometry on the grid level. To construct a lower dimensional model, we use the Mixed Generalized Multiscale Finite Element Method, which is based on some multiscale basis functions for velocity fields. The construction of the basis functions is based on the local snapshot space that takes all possible flows on the interface between coarse cells into account. In order to reduce the size of the snapshot space and obtain the multiscale approximation, we solve a local spectral problem to identify dominant modes in the snapshot space. We present a convergence analysis of the presented multiscale method. Numerical results are presented for two-dimensional problems in three testing geometries along with the errors associated to different numbers of the multiscale basis functions used for the velocity field. Numerical investigations are conducted for problems with homogeneous and heterogeneous properties respectively.
We introduce Disease Informed Neural Networks (DINNs) -- neural networks capable of learning how diseases spread, forecasting their progression, and finding their unique parameters (e.g. death rate). Here, we used DINNs to identify the dynamics of 11 highly infectious and deadly diseases. These systems vary in their complexity, ranging from 3D to 9D ODEs, and from a few parameters to over a dozen. The diseases include COVID, Anthrax, HIV, Zika, Smallpox, Tuberculosis, Pneumonia, Ebola, Dengue, Polio, and Measles. Our contribution is three fold. First, we extend the recent physics informed neural networks (PINNs) approach to a large number of infectious diseases. Second, we perform an extensive analysis of the capabilities and shortcomings of PINNs on diseases. Lastly, we show the ease at which one can use DINN to effectively learn COVID's spread dynamics and forecast its progression a month into the future from real-life data. Code and data can be found here: //github.com/Shaier/DINN.
One of the major issues in the computational mechanics is to take into account the geometrical complexity. To overcome this difficulty and to avoid the expensive mesh generation, geometrically unfitted methods, i.e. the numerical methods using the simple computational meshes that do not fit the boundary of the domain, and/or the internal interfaces, have been widely developed. In the present work, we investigate the performances of an unfitted method called $\phi$-FEM that converges optimally and uses classical finite element spaces so that it can be easily implemented using general FEM libraries. The main idea is to take into account the geometry thanks to a level set function describing the boundary or the interface. Up to now, the $\phi$-FEM approach has been proposed, tested and substantiated mathematically only in some simplest settings: Poisson equation with Dirichlet/Neumann/Robin boundary conditions. Our goal here is to demonstrate its applicability to some more sophisticated governing equations arising in the computational mechanics. We consider the linear elasticity equations accompanied by either pure Dirichlet boundary conditions or by the mixed ones (Dirichlet and Neumann boundary conditions co-existing on parts of the boundary), an interface problem (linear elasticity with material coefficients abruptly changing over an internal interface), a model of elastic structures with cracks, and finally the heat equation. In all these settings, we derive an appropriate variant of $\phi$-FEM and then illustrate it by numerical tests on manufactured solutions. We also compare the accuracy and efficiency of $\phi$-FEM with those of the standard fitted FEM on the meshes of similar size, revealing the substantial gains that can be achieved by $\phi$-FEM in both the accuracy and the computational time.
We propose heavy ball neural ordinary differential equations (HBNODEs), leveraging the continuous limit of the classical momentum accelerated gradient descent, to improve neural ODEs (NODEs) training and inference. HBNODEs have two properties that imply practical advantages over NODEs: (i) The adjoint state of an HBNODE also satisfies an HBNODE, accelerating both forward and backward ODE solvers, thus significantly reducing the number of function evaluations (NFEs) and improving the utility of the trained models. (ii) The spectrum of HBNODEs is well structured, enabling effective learning of long-term dependencies from complex sequential data. We verify the advantages of HBNODEs over NODEs on benchmark tasks, including image classification, learning complex dynamics, and sequential modeling. Our method requires remarkably fewer forward and backward NFEs, is more accurate, and learns long-term dependencies more effectively than the other ODE-based neural network models. Code is available at \url{//github.com/hedixia/HeavyBallNODE}.
This paper is on the construction of structure-preserving, online-efficient reduced models for the barotropic Euler equations with a friction term on networks. The nonlinear flow problem finds broad application in the context of gas distribution networks. We propose a snapshot-based reduction approach that consists of a mixed variational Galerkin approximation combined with quadrature-type complexity reduction. Its main feature is that certain compatibility conditions are assured during the training phase, which make our approach structure-preserving. The resulting reduced models are locally mass conservative and inherit an energy-bound and port-Hamiltonian structure. We also derive a well-posedness result for them. In the training phase, the compatibility conditions pose challenges, we face constrained data approximation problems as opposed to the unconstrained training problems in the conventional reduction methods. The training of our model order reduction consists of a principal component analysis under a compatibility constraint and, notably, yields reduced models that fulfill an optimality condition for the snapshot data. The training of our quadrature-type complexity reduction involves a semi-definite program with combinatorial aspects, which we approach by a greedy procedure.
We consider a Johnson-N\'ed\'elec FEM-BEM coupling, which is a direct and non-symmetric coupling of finite and boundary element methods, in order to solve interface problems for the magnetostatic Maxwell's equations with the magnetic vector potential ansatz. In the FEM-domain, equations may be non-linear, whereas they are exclusively linear in the BEM-part to guarantee the existence of a fundamental solution. First, the weak problem is formulated in quotient spaces to avoid resolving to a saddle point problem. Second, we establish in this setting well-posedness of the arising problem using the framework of Lipschitz and strongly monotone operators as well as a stability result for a special type of non-linearity, which is typically considered in magnetostatic applications. Then, the discretization is performed in the isogeometric context, i.e., the same type of basis functions that are used for geometry design are considered as ansatz functions for the discrete setting. In particular, NURBS are employed for geometry considerations, and B-Splines, which can be understood as a special type of NURBS, for analysis purposes. In this context, we derive a priori estimates w.r.t. h-refinement, and point out to an interesting behavior of BEM, which consists in an amelioration of the convergence rates, when a functional of the solution is evaluated in the exterior BEM-domain. This improvement may lead to a doubling of the convergence rate under certain assumptions. Finally, we end the paper with a numerical example to illustrate the theoretical results, along with a conclusion and an outlook.
An interesting observation in artificial neural networks is their favorable generalization error despite typically being extremely overparameterized. It is well known that the classical statistical learning methods often result in vacuous generalization errors in the case of overparameterized neural networks. Adopting the recently developed Neural Tangent (NT) kernel theory, we prove uniform generalization bounds for overparameterized neural networks in kernel regimes, when the true data generating model belongs to the reproducing kernel Hilbert space (RKHS) corresponding to the NT kernel. Importantly, our bounds capture the exact error rates depending on the differentiability of the activation functions. In order to establish these bounds, we propose the information gain of the NT kernel as a measure of complexity of the learning problem. Our analysis uses a Mercer decomposition of the NT kernel in the basis of spherical harmonics and the decay rate of the corresponding eigenvalues. As a byproduct of our results, we show the equivalence between the RKHS corresponding to the NT kernel and its counterpart corresponding to the Mat\'ern family of kernels, showing the NT kernels induce a very general class of models. We further discuss the implications of our analysis for some recent results on the regret bounds for reinforcement learning and bandit algorithms, which use overparameterized neural networks.
Solving for detailed chemical kinetics remains one of the major bottlenecks for computational fluid dynamics simulations of reacting flows using a finite-rate-chemistry approach. This has motivated the use of fully connected artificial neural networks to predict stiff chemical source terms as functions of the thermochemical state of the combustion system. However, due to the nonlinearities and multi-scale nature of combustion, the predicted solution often diverges from the true solution when these deep learning models are coupled with a computational fluid dynamics solver. This is because these approaches minimize the error during training without guaranteeing successful integration with ordinary differential equation solvers. In the present work, a novel neural ordinary differential equations approach to modeling chemical kinetics, termed as ChemNODE, is developed. In this deep learning framework, the chemical source terms predicted by the neural networks are integrated during training, and by computing the required derivatives, the neural network weights are adjusted accordingly to minimize the difference between the predicted and ground-truth solution. A proof-of-concept study is performed with ChemNODE for homogeneous autoignition of hydrogen-air mixture over a range of composition and thermodynamic conditions. It is shown that ChemNODE accurately captures the correct physical behavior and reproduces the results obtained using the full chemical kinetic mechanism at a fraction of the computational cost.
We introduce a new family of deep neural network models. Instead of specifying a discrete sequence of hidden layers, we parameterize the derivative of the hidden state using a neural network. The output of the network is computed using a black-box differential equation solver. These continuous-depth models have constant memory cost, adapt their evaluation strategy to each input, and can explicitly trade numerical precision for speed. We demonstrate these properties in continuous-depth residual networks and continuous-time latent variable models. We also construct continuous normalizing flows, a generative model that can train by maximum likelihood, without partitioning or ordering the data dimensions. For training, we show how to scalably backpropagate through any ODE solver, without access to its internal operations. This allows end-to-end training of ODEs within larger models.