We consider the gradient descent flow widely used for the minimization of the $\mathcal{L}^2$ cost function in Deep Learning networks, and introduce two modified versions; one adapted for the overparametrized setting, and the other for the underparametrized setting. Both have a clear and natural invariant geometric meaning, taking into account the pullback vector bundle structure in the overparametrized, and the pushforward vector bundle structure in the underparametrized setting. In the overparametrized case, we prove that, provided that a rank condition holds, all orbits of the modified gradient descent drive the $\mathcal{L}^2$ cost to its global minimum at a uniform exponential convergence rate; one thereby obtains an a priori stopping time for any prescribed proximity to the global minimum. We point out relations of the latter to sub-Riemannian geometry.
相關內容
We prove that the values of a generalized $\psi$-estimator (introduced by Barczy and P\'ales in 2022) on samples of arbitrary length but having only two different observations uniquely determine the values of the estimator on any sample of arbitrary length without any restriction on the number of different observations. In other words, samples of arbitrary length but having only two different observations form a determining class for generalized $\psi$-estimators. We also obtain a similar statement for the comparison of generalized $\psi$-estimators using comparative functions, and, as a corollary of this result, we derive the Schweitzer's inequality (also called Kantorovich's inequality).
The uniform interpolation property in a given logic can be understood as the definability of propositional quantifiers. We mechanise the computation of these quantifiers and prove correctness in the Coq proof assistant for three modal logics, namely: (1) the modal logic K, for which a pen-and-paper proof exists; (2) G\"odel-L\"ob logic GL, for which our formalisation clarifies an important point in an existing, but incomplete, sequent-style proof; and (3) intuitionistic strong L\"ob logic iSL, for which this is the first proof-theoretic construction of uniform interpolants. Our work also yields verified programs that allow one to compute the propositional quantifiers on any formula in this logic.
Wavelet compressed, modified Hilbert transform in the space-time discretization of the heat equation
On a finite time interval $(0,T)$, we consider the multiresolution Galerkin discretization of a modified Hilbert transform $\mathcal H_T$ which arises in the space-time Galerkin discretization of the linear diffusion equation. To this end, we design spline-wavelet systems in $(0,T)$ consisting of piecewise polynomials of degree $\geq 1$ with sufficiently many vanishing moments which constitute Riesz bases in the Sobolev spaces $ H^{s}_{0,}(0,T)$ and $ H^{s}_{,0}(0,T)$. These bases provide multilevel splittings of the temporal discretization spaces into "increment" or "detail" spaces of direct sum type. Via algebraic tensor-products of these temporal multilevel discretizations with standard, hierarchic finite element spaces in the spatial domain (with standard Lagrangian FE bases), sparse space-time tensor-product spaces are obtained, which afford a substantial reduction in the number of the degrees of freedom as compared to time-marching discretizations. In addition, temporal spline-wavelet bases allow to compress certain nonlocal integrodifferential operators which appear in stable space-time variational formulations of initial-boundary value problems, such as the heat equation and the acoustic wave equation. An efficient preconditioner is proposed that affords linear complexity solves of the linear system of equations which results from the sparse space-time Galerkin discretization.
We show that every $3$-connected $K_{2,\ell}$-minor free graph with minimum degree at least $4$ has maximum degree at most $7\ell$. As a consequence, we show that every 3-connected $K_{2,\ell}$-minor free graph with minimum degree at least $5$ and no twins of degree $5$ has bounded size. Our proofs use Steiner trees and nested cuts; in particular, they do not rely on Ding's characterization of $K_{2,\ell}$-minor free graphs.
We consider the problem of sketching a set valuation function, which is defined as the expectation of a valuation function of independent random item values. We show that for monotone subadditive or submodular valuation functions satisfying a weak homogeneity condition, or certain other conditions, there exist discretized distributions of item values with $O(k\log(k))$ support sizes that yield a sketch valuation function which is a constant-factor approximation, for any value query for a set of items of cardinality less than or equal to $k$. The discretized distributions can be efficiently computed by an algorithm for each item's value distribution separately. Our results hold under conditions that accommodate a wide range of valuation functions arising in applications, such as the value of a team corresponding to the best performance of a team member, constant elasticity of substitution production functions exhibiting diminishing returns used in economics and consumer theory, and others. Sketch valuation functions are particularly valuable for finding approximate solutions to optimization problems such as best set selection and welfare maximization. They enable computationally efficient evaluation of approximate value oracle queries and provide an approximation guarantee for the underlying optimization problem.
We study the problem of approximating a matrix $\mathbf{A}$ with a matrix that has a fixed sparsity pattern (e.g., diagonal, banded, etc.), when $\mathbf{A}$ is accessed only by matrix-vector products. We describe a simple randomized algorithm that returns an approximation with the given sparsity pattern with Frobenius-norm error at most $(1+\varepsilon)$ times the best possible error. When each row of the desired sparsity pattern has at most $s$ nonzero entries, this algorithm requires $O(s/\varepsilon)$ non-adaptive matrix-vector products with $\mathbf{A}$. We proceed to prove a matching lower-bound. Specifically, we show that for any $s\geq 1$, there are matrices $\mathbf{A}$ such that, for any sparsity pattern with $\Theta(s)$ nonzeros per row and column, any algorithm which obtains a $(1+\varepsilon)$ accurate approximation of the given sparsity from matrix-vector products requires at least $\Omega(s/\varepsilon)$ matrix-vector products. Our bounds therefore resolve the matrix-vector product query complexity of the problem up to constant factors, even for the well-studied case of diagonal approximation, for which no previous lower bounds were known.
{We analyze a general Implicit-Explicit (IMEX) time discretization for the compressible Euler equations of gas dynamics, showing that they are asymptotic-preserving (AP) in the low Mach number limit. The analysis is carried out for a general equation of state (EOS). We consider both a single asymptotic length scale and two length scales. We then show that, when coupling these time discretizations with a Discontinuous Galerkin (DG) space discretization with appropriate fluxes, an all Mach number numerical method is obtained. A number of relevant benchmarks for ideal gases and their non-trivial extension to non-ideal EOS validate the performed analysis.
For the numerical solution of Dirichlet-type boundary value problems associated to nonlinear fractional differential equations of order $\alpha \in (1,2)$ that use Caputo derivatives, we suggest to employ shooting methods. In particular, we demonstrate that the so-called proportional secting technique for selecting the required initial values leads to numerical schemes that converge to high accuracy in a very small number of shooting iterations, and we provide an explanation of the analytical background for this favourable numerical behaviour.
Two Latin squares of order $n$ are $r$-orthogonal if, when superimposed, there are exactly $r$ distinct ordered pairs. The spectrum of all values of $r$ for Latin squares of order $n$ is known. A Latin square $A$ of order $n$ is $r$-self-orthogonal if $A$ and its transpose are $r$-orthogonal. The spectrum of all values of $r$ is known for all orders $n\ne 14$. We develop randomized algorithms for computing pairs of $r$-orthogonal Latin squares of order $n$ and algorithms for computing $r$-self-orthogonal Latin squares of order $n$.
Partial differential equations (PDEs) have become an essential tool for modeling complex physical systems. Such equations are typically solved numerically via mesh-based methods, such as finite element methods, with solutions over the spatial domain. However, obtaining these solutions are often prohibitively costly, limiting the feasibility of exploring parameters in PDEs. In this paper, we propose an efficient emulator that simultaneously predicts the solutions over the spatial domain, with theoretical justification of its uncertainty quantification. The novelty of the proposed method lies in the incorporation of the mesh node coordinates into the statistical model. In particular, the proposed method segments the mesh nodes into multiple clusters via a Dirichlet process prior and fits Gaussian process models with the same hyperparameters in each of them. Most importantly, by revealing the underlying clustering structures, the proposed method can provide valuable insights into qualitative features of the resulting dynamics that can be used to guide further investigations. Real examples are demonstrated to show that our proposed method has smaller prediction errors than its main competitors, with competitive computation time, and identifies interesting clusters of mesh nodes that possess physical significance, such as satisfying boundary conditions. An R package for the proposed methodology is provided in an open repository.
注冊地址： 北京市海淀區羊坊店路18號2幢3層301-191