In this paper, we study how the Koopman operator framework can be combined with kernel methods to effectively control nonlinear dynamical systems. While kernel methods have typically large computational requirements, we show how random subspaces (Nystr\"om approximation) can be used to achieve huge computational savings while preserving accuracy. Our main technical contribution is deriving theoretical guarantees on the effect of the Nystr\"om approximation. More precisely, we study the linear quadratic regulator problem, showing that both the approximated Riccati operator and the regulator objective, for the associated solution of the optimal control problem, converge at the rate $m^{-1/2}$, where $m$ is the random subspace size. Theoretical findings are complemented by numerical experiments corroborating our results.
The Koopman operator presents an attractive approach to achieve global linearization of nonlinear systems, making it a valuable method for simplifying the understanding of complex dynamics. While data-driven methodologies have exhibited promise in approximating finite Koopman operators, they grapple with various challenges, such as the judicious selection of observables, dimensionality reduction, and the ability to predict complex system behaviors accurately. This study presents a novel approach termed Mori-Zwanzig autoencoder (MZ-AE) to robustly approximate the Koopman operator in low-dimensional spaces. The proposed method leverages a nonlinear autoencoder to extract key observables for approximating a finite invariant Koopman subspace and integrates a non-Markovian correction mechanism using the Mori-Zwanzig formalism. Consequently, this approach yields a closed representation of dynamics within the latent manifold of the nonlinear autoencoder, thereby enhancing the precision and stability of the Koopman operator approximation. Demonstrations showcase the technique's ability to capture regime transitions in the flow around a cylinder. It also provides a low dimensional approximation for Kuramoto-Sivashinsky with promising short-term predictability and robust long-term statistical performance. By bridging the gap between data-driven techniques and the mathematical foundations of Koopman theory, MZ-AE offers a promising avenue for improved understanding and prediction of complex nonlinear dynamics.
The rise of machine learning has fueled the discovery of new materials and, especially, metamaterials--truss lattices being their most prominent class. While their tailorable properties have been explored extensively, the design of truss-based metamaterials has remained highly limited and often heuristic, due to the vast, discrete design space and the lack of a comprehensive parameterization. We here present a graph-based deep learning generative framework, which combines a variational autoencoder and a property predictor, to construct a reduced, continuous latent representation covering an enormous range of trusses. This unified latent space allows for the fast generation of new designs through simple operations (e.g., traversing the latent space or interpolating between structures). We further demonstrate an optimization framework for the inverse design of trusses with customized mechanical properties in both the linear and nonlinear regimes, including designs exhibiting exceptionally stiff, auxetic, pentamode-like, and tailored nonlinear behaviors. This generative model can predict manufacturable (and counter-intuitive) designs with extreme target properties beyond the training domain.
Motivated by applications to group synchronization and quadratic assignment on random data, we study a general problem of Bayesian inference of an unknown ``signal'' belonging to a high-dimensional compact group, given noisy pairwise observations of a featurization of this signal. We establish a quantitative comparison between the signal-observation mutual information in any such problem with that in a simpler model with linear observations, using interpolation methods. For group synchronization, our result proves a replica formula for the asymptotic mutual information and Bayes-optimal mean-squared-error. Via analyses of this replica formula, we show that the conjectural phase transition threshold for computationally-efficient weak recovery of the signal is determined by a classification of the real-irreducible components of the observed group representation(s), and we fully characterize the information-theoretic limits of estimation in the example of angular/phase synchronization over $SO(2)$/$U(1)$. For quadratic assignment, we study observations given by a kernel matrix of pairwise similarities and a randomly permutated and noisy counterpart, and we show in a bounded signal-to-noise regime that the asymptotic mutual information coincides with that in a Bayesian spiked model with i.i.d. signal prior.
Simulation of realistic classical mechanical systems is of great importance to many areas of engineering such as robotics, dynamics of rotating machinery and control theory. In this work, we develop quantum algorithms to estimate quantities of interest such as the kinetic energy in a given classical mechanical system in the presence of friction or damping as well as forcing or source terms, which makes the algorithm of practical interest. We show that for such systems, the quantum algorithm scales polynomially with the logarithm of the dimension of the system. We cast this problem in terms of Hamilton's equations of motion (equivalent to the first variation of the Lagrangian) and solve them using quantum algorithms for differential equations. We then consider the hardness of estimating the kinetic energy of a damped coupled oscillator system. We show that estimating the kinetic energy at a given time of this system to within additive precision is BQP hard when the strength of the damping term is bounded by an inverse polynomial in the number of qubits. We then consider the problem of designing optimal control of classical systems, which can be cast as the second variation of the Lagrangian. In this direction, we first consider the Riccati equation, which is a nonlinear differential equation ubiquitous in control theory. We give an efficient quantum algorithm to solve the Riccati differential equation well into the nonlinear regime. To our knowledge, this is the first example of any nonlinear differential equation that can be solved when the strength of the nonlinearity is asymptotically greater than the amount of dissipation. We then show how to use this algorithm to solve the linear quadratic regulator problem, which is an example of the Hamilton-Jacobi-Bellman equation.
In this study, our main objective is to address the challenge of solving elliptic equations with quasiperiodic coefficients. To achieve accurate and efficient computation, we introduce the projection method, which enables the embedding of quasiperiodic systems into higher-dimensional periodic systems. To enhance the computational efficiency, we propose a compressed storage strategy for the stiffness matrix, reducing memory requirements while preserving accuracy. Furthermore, we design a diagonal preconditioner to efficiently solve the resulting high-dimensional linear system by reducing the condition number of the stiffness matrix. These techniques collectively contribute to the computational effectiveness of our proposed approach. We demonstrate the effectiveness and accuracy of our approach through a series of numerical examples. Moreover, we apply our method to achieve a highly accurate computation of the homogenized coefficients for a quasiperiodic multiscale elliptic equation.
Transformers are neural networks that revolutionized natural language processing and machine learning. They process sequences of inputs, like words, using a mechanism called self-attention, which is trained via masked language modeling (MLM). In MLM, a word is randomly masked in an input sequence, and the network is trained to predict the missing word. Despite the practical success of transformers, it remains unclear what type of data distribution self-attention can learn efficiently. Here, we show analytically that if one decouples the treatment of word positions and embeddings, a single layer of self-attention learns the conditionals of a generalized Potts model with interactions between sites and Potts colors. Moreover, we show that training this neural network is exactly equivalent to solving the inverse Potts problem by the so-called pseudo-likelihood method, well known in statistical physics. Using this mapping, we compute the generalization error of self-attention in a model scenario analytically using the replica method.
This work presents a nonintrusive physics-preserving method to learn reduced-order models (ROMs) of Lagrangian systems, which includes nonlinear wave equations. Existing intrusive projection-based model reduction approaches construct structure-preserving Lagrangian ROMs by projecting the Euler-Lagrange equations of the full-order model (FOM) onto a linear subspace. This Galerkin projection step requires complete knowledge about the Lagrangian operators in the FOM and full access to manipulate the computer code. In contrast, the proposed Lagrangian operator inference approach embeds the mechanics into the operator inference framework to develop a data-driven model reduction method that preserves the underlying Lagrangian structure. The proposed approach exploits knowledge of the governing equations (but not their discretization) to define the form and parametrization of a Lagrangian ROM which can then be learned from projected snapshot data. The method does not require access to FOM operators or computer code. The numerical results demonstrate Lagrangian operator inference on an Euler-Bernoulli beam model, the sine-Gordon (nonlinear) wave equation, and a large-scale discretization of a soft robot fishtail with 779,232 degrees of freedom. The learned Lagrangian ROMs generalize well, as they can accurately predict the physical solutions both far outside the training time interval, as well as for unseen initial conditions.
In this paper, we present a hybrid neural-network and MAC (Marker-And-Cell) scheme for solving Stokes equations with singular forces on an embedded interface in regular domains. As known, the solution variables (the pressure and velocity) exhibit non-smooth behaviors across the interface so extra discretization efforts must be paid near the interface in order to have small order of local truncation errors in finite difference schemes. The present hybrid approach avoids such additional difficulty. It combines the expressive power of neural networks with the convergence of finite difference schemes to ease the code implementation and to achieve good accuracy at the same time. The key idea is to decompose the solution into singular and regular parts. The neural network learning machinery incorporating the given jump conditions finds the singular part solution, while the standard MAC scheme is used to obtain the regular part solution with associated boundary conditions. The two- and three-dimensional numerical results show that the present hybrid method converges with second-order accuracy for the velocity and first-order accuracy for the pressure, and it is comparable with the traditional immersed interface method in literature.
In this paper, we investigate an optimal control problem governed by parabolic equations with measure-valued controls over time. We establish the well-posedness of the optimal control problem and derive the first-order optimality condition using Clarke's subgradients, revealing a sparsity structure in time for the optimal control. Consequently, these optimal control problems represent a generalization of impulse control for evolution equations. To discretize the optimal control problem, we employ the space-time finite element method. Here, the state equation is approximated using piecewise linear and continuous finite elements in space, alongside a Petrov-Galerkin method utilizing piecewise constant trial functions and piecewise linear and continuous test functions in time. The control variable is discretized using the variational discretization concept. For error estimation, we initially derive a priori error estimates and stabilities for the finite element discretizations of the state and adjoint equations. Subsequently, we establish weak-* convergence for the control under the norm $\mathcal{M}(\bar I_c;L^2(\omega))$, with a convergence order of $O(h^\frac{1}{2}+\tau^\frac{1}{4})$ for the state.
We consider the computation of model-free bounds for multi-asset options in a setting that combines dependence uncertainty with additional information on the dependence structure. More specifically, we consider the setting where the marginal distributions are known and partial information, in the form of known prices for multi-asset options, is also available in the market. We provide a fundamental theorem of asset pricing in this setting, as well as a superhedging duality that allows to transform the maximization problem over probability measures in a more tractable minimization problem over trading strategies. The latter is solved using a penalization approach combined with a deep learning approximation using artificial neural networks. The numerical method is fast and the computational time scales linearly with respect to the number of traded assets. We finally examine the significance of various pieces of additional information. Empirical evidence suggests that "relevant" information, i.e. prices of derivatives with the same payoff structure as the target payoff, are more useful that other information, and should be prioritized in view of the trade-off between accuracy and computational efficiency.