We consider the gradient descent flow widely used for the minimization of the $\mathcal{L}^2$ cost function in Deep Learning networks, and introduce two modified versions; one adapted for the overparametrized setting, and the other for the underparametrized setting. Both have a clear and natural invariant geometric meaning, taking into account the pullback vector bundle structure in the overparametrized, and the pushforward vector bundle structure in the underparametrized setting. In the overparametrized case, we prove that, provided that a rank condition holds, all orbits of the modified gradient descent drive the $\mathcal{L}^2$ cost to its global minimum at a uniform exponential convergence rate. We point out relations of the latter to sub-Riemannian geometry.
相關內容
The most popular method for computing the matrix logarithm is a combination of the inverse scaling and squaring method in conjunction with a Pad\'e approximation, sometimes accompanied by the Schur decomposition. The main computational effort lies in matrix-matrix multiplications and left matrix division. In this work we illustrate that the number of such operations can be substantially reduced, by using a graph based representation of an efficient polynomial evaluation scheme. A technique to analyze the rounding error is proposed, and backward error analysis is adapted. We provide substantial simulations illustrating competitiveness both in terms of computation time and rounding errors.
We generalize the Poisson limit theorem to binary functions of random objects whose law is invariant under the action of an amenable group. Examples include stationary random fields, exchangeable sequences, and exchangeable graphs. A celebrated result of E. Lindenstrauss shows that normalized sums over certain increasing subsets of such groups approximate expectations. Our results clarify that the corresponding unnormalized sums of binary statistics are asymptotically Poisson, provided suitable mixing conditions hold. They extend further to randomly subsampled sums and also show that strict invariance of the distribution is not needed if the requisite mixing condition defined by the group holds. We illustrate the results with applications to random fields, Cayley graphs, and Poisson processes on groups.
We study discretizations of fractional fully nonlinear equations by powers of discrete Laplacians. Our problems are parabolic and of order $\sigma\in(0,2)$ since they involve fractional Laplace operators $(-\Delta)^{\sigma/2}$. They arise e.g.~in control and game theory as dynamic programming equations, and solutions are non-smooth in general and should be interpreted as viscosity solutions. Our approximations are realized as finite-difference quadrature approximations and are 2nd order accurate for all values of $\sigma$. The accuracy of previous approximations depend on $\sigma$ and are worse when $\sigma$ is close to $2$. We show that the schemes are monotone, consistent, $L^\infty$-stable, and convergent using a priori estimates, viscosity solutions theory, and the method of half-relaxed limits. We present several numerical examples.
This paper studies the convergence of a spatial semidiscretization of a three-dimensional stochastic Allen-Cahn equation with multiplicative noise. For non-smooth initial values, the regularity of the mild solution is investigated, and an error estimate is derived with the spatial $ L^2 $-norm. For smooth initial values, two error estimates with the general spatial $ L^q $-norms are established.
The structured $\varepsilon$-stability radius is introduced as a quantity to assess the robustness of transient bounds of solutions to linear differential equations under structured perturbations of the matrix. This applies to general linear structures such as complex or real matrices with a given sparsity pattern or with restricted range and corange, or special classes such as Toeplitz matrices. The notion conceptually combines unstructured and structured pseudospectra in a joint pseudospectrum, allowing for the use of resolvent bounds as with unstructured pseudospectra and for structured perturbations as with structured pseudospectra. We propose and study an algorithm for computing the structured $\varepsilon$-stability radius. This algorithm solves eigenvalue optimization problems via suitably discretized rank-1 matrix differential equations that originate from a gradient system. The proposed algorithm has essentially the same computational cost as the known rank-1 algorithms for computing unstructured and structured stability radii. Numerical experiments illustrate the behavior of the algorithm.
Due to the complex behavior arising from non-uniqueness, symmetry, and bifurcations in the solution space, solving inverse problems of nonlinear differential equations (DEs) with multiple solutions is a challenging task. To address this, we propose homotopy physics-informed neural networks (HomPINNs), a novel framework that leverages homotopy continuation and neural networks (NNs) to solve inverse problems. The proposed framework begins with the use of NNs to simultaneously approximate unlabeled observations across diverse solutions while adhering to DE constraints. Through homotopy continuation, the proposed method solves the inverse problem by tracing the observations and identifying multiple solutions. The experiments involve testing the performance of the proposed method on one-dimensional DEs and applying it to solve a two-dimensional Gray-Scott simulation. Our findings demonstrate that the proposed method is scalable and adaptable, providing an effective solution for solving DEs with multiple solutions and unknown parameters. Moreover, it has significant potential for various applications in scientific computing, such as modeling complex systems and solving inverse problems in physics, chemistry, biology, etc.
SARRIGUREN, a new complete algorithm for SAT based on counting clauses (which is valid also for Unique-SAT and #SAT) is described, analyzed and tested. Although existing complete algorithms for SAT perform slower with clauses with many literals, that is an advantage for SARRIGUREN, because the more literals are in the clauses the bigger is the probability of overlapping among clauses, a property that makes the clause counting process more efficient. Actually, it provides a $O(m^2 \times n/k)$ time complexity for random $k$-SAT instances of $n$ variables and $m$ relatively dense clauses, where that density level is relative to the number of variables $n$, that is, clauses are relatively dense when $k\geq7\sqrt{n}$. Although theoretically there could be worst-cases with exponential complexity, the probability of those cases to happen in random $k$-SAT with relatively dense clauses is practically zero. The algorithm has been empirically tested and that polynomial time complexity maintains also for $k$-SAT instances with less dense clauses ($k\geq5\sqrt{n}$). That density could, for example, be of only 0.049 working with $n=20000$ variables and $k=989$ literals. In addition, they are presented two more complementary algorithms that provide the solutions to $k$-SAT instances and valuable information about number of solutions for each literal. Although this algorithm does not solve the NP=P problem (it is not a polynomial algorithm for 3-SAT), it broads the knowledge about that subject, because $k$-SAT with $k>3$ and dense clauses is not harder than 3-SAT. Moreover, the Python implementation of the algorithms, and all the input datasets and obtained results in the experiments are made available.
We prove discrete versions of the first and second Weber inequalities on $\boldsymbol{H}(\mathbf{curl})\cap\boldsymbol{H}(\mathrm{div}_{\eta})$-like hybrid spaces spanned by polynomials attached to the faces and to the cells of a polyhedral mesh. The proven hybrid Weber inequalities are optimal in the sense that (i) they are formulated in terms of $\boldsymbol{H}(\mathbf{curl})$- and $\boldsymbol{H}(\mathrm{div}_{\eta})$-like hybrid semi-norms designed so as to embed optimally (polynomially) consistent face penalty terms, and (ii) they are valid for face polynomials in the smallest possible stability-compatible spaces. Our results are valid on domains with general, possibly non-trivial topology. In a second part we also prove, within a general topological setting, related discrete Maxwell compactness properties.
We present approximation algorithms for the Fault-tolerant $k$-Supplier with Outliers ($\mathsf{F}k\mathsf{SO}$) problem. This is a common generalization of two known problems -- $k$-Supplier with Outliers, and Fault-tolerant $k$-Supplier -- each of which generalize the well-known $k$-Supplier problem. In the $k$-Supplier problem the goal is to serve $n$ clients $C$, by opening $k$ facilities from a set of possible facilities $F$; the objective function is the farthest that any client must travel to access an open facility. In $\mathsf{F}k\mathsf{SO}$, each client $v$ has a fault-tolerance $\ell_v$, and now desires $\ell_v$ facilities to serve it; so each client $v$'s contribution to the objective function is now its distance to the $\ell_v^{\text{th}}$ closest open facility. Furthermore, we are allowed to choose $m$ clients that we will serve, and only those clients contribute to the objective function, while the remaining $n-m$ are considered outliers. Our main result is a $\min\{4t-1,2^t+1\}$-approximation for the $\mathsf{F}k\mathsf{SO}$ problem, where $t$ is the number of distinct values of $\ell_v$ that appear in the instance. At $t=1$, i.e. in the case where the $\ell_v$'s are uniformly some $\ell$, this yields a $3$-approximation, improving upon the $11$-approximation given for the uniform case by Inamdar and Varadarajan [2020], who also introduced the problem. Our result for the uniform case matches tight $3$-approximations that exist for $k$-Supplier, $k$-Supplier with Outliers, and Fault-tolerant $k$-Supplier. Our key technical contribution is an application of the round-or-cut schema to $\mathsf{F}k\mathsf{SO}$. Guided by an LP relaxation, we reduce to a simpler optimization problem, which we can solve to obtain distance bounds for the "round" step, and valid inequalities for the "cut" step.
Dirac delta distributionally sourced differential equations emerge in many dynamical physical systems from neuroscience to black hole perturbation theory. Most of these lack exact analytical solutions and are thus best tackled numerically. This work describes a generic numerical algorithm which constructs discontinuous spatial and temporal discretisations by operating on discontinuous Lagrange and Hermite interpolation formulae recovering higher order accuracy. It is shown by solving the distributionally sourced wave equation, which has analytical solutions, that numerical weak-form solutions can be recovered to high order accuracy by solving a first-order reduced system of ordinary differential equations. The method-of-lines framework is applied to the DiscoTEX algorithm i.e through discontinuous collocation with implicit-turned-explicit (IMTEX) integration methods which are symmetric and conserve symplectic structure. Furthermore, the main application of the algorithm is proved, for the first-time, by calculating the amplitude at any desired location within the numerical grid, including at the position (and at its right and left limit) where the wave- (or wave-like) equation is discontinuous via interpolation using DiscoTEX. This is shown, firstly by solving the wave- (or wave-like) equation and comparing the numerical weak-form solution to the exact solution. Finally, one shows how to reconstruct the scalar and gravitational metric perturbations from weak-form numerical solutions of a non-rotating black hole, which do not have known exact analytical solutions, and compare against state-of-the-art frequency domain results. One concludes by motivating how DiscoTEX, and related algorithms, open a promising new alternative Extreme-Mass-Ratio-Inspiral (EMRI)s waveform generation route via a self-consistent evolution for the gravitational self-force programme in the time-domain.
注冊地址： 北京市海淀區羊坊店路18號2幢3層301-191