It is of significant interest in many applications to sample from a high-dimensional target distribution $\pi$ with the density $\pi(\text{d} x) \propto e^{-U(x)} (\text{d} x) $, based on the temporal discretization of the Langevin stochastic differential equations (SDEs). In this paper, we propose an explicit projected Langevin Monte Carlo (PLMC) algorithm with non-convex potential $U$ and super-linear gradient of $U$ and investigate the non-asymptotic analysis of its sampling error in total variation distance. Equipped with time-independent regularity estimates for the corresponding Kolmogorov equation, we derive the non-asymptotic bounds on the total variation distance between the target distribution of the Langevin SDEs and the law induced by the PLMC scheme with order $\mathcal{O}(h |\ln h|)$. Moreover, for a given precision $\epsilon$, the smallest number of iterations of the classical Langevin Monte Carlo (LMC) scheme with the non-convex potential $U$ and the globally Lipshitz gradient of $U$ can be guaranteed by order ${\mathcal{O}}\big(\tfrac{d^{3/2}}{\epsilon} \cdot \ln (\tfrac{d}{\epsilon}) \cdot \ln (\tfrac{1}{\epsilon}) \big)$. Numerical experiments are provided to confirm the theoretical findings.
相關內容
In this paper, we examine the relationship between the stability of the dynamical system $x^{\prime}=f(x)$ and the computability of its basins of attraction. We present a computable $C^{\infty}$ system $x^{\prime}=f(x)$ that possesses a computable and stable equilibrium point, yet whose basin of attraction is robustly non-computable in a neighborhood of $f$ in the sense that both the equilibrium point and the non-computability of its associated basin of attraction persist when $f$ is slightly perturbed. This indicates that local stability near a stable equilibrium point alone is insufficient to guarantee the computability of its basin of attraction. However, we also demonstrate that the basins of attraction associated with a structurally stable - globally stable - planar system defined on a compact set are computable. Our findings suggest that the global stability of a system and the compactness of the domain play a pivotal role in determining the computability of its basins of attraction.
We develop a new `subspace layered least squares' interior point method (IPM) for solving linear programs. Applied to an $n$-variable linear program in standard form, the iteration complexity of our IPM is up to an $O(n^{1.5} \log n)$ factor upper bounded by the \emph{straight line complexity} (SLC) of the linear program. This term refers to the minimum number of segments of any piecewise linear curve that traverses the \emph{wide neighborhood} of the central path, a lower bound on the iteration complexity of any IPM that follows a piecewise linear trajectory along a path induced by a self-concordant barrier. In particular, our algorithm matches the number of iterations of any such IPM up to the same factor $O(n^{1.5}\log n)$. As our second contribution, we show that the SLC of any linear program is upper bounded by $2^{n + o(1)}$, which implies that our IPM's iteration complexity is at most exponential. This in contrast to existing iteration complexity bounds that depend on either bit-complexity or condition measures; these can be unbounded in the problem dimension. We achieve our upper bound by showing that the central path is well-approximated by a combinatorial proxy we call the \emph{max central path}, which consists of $2n$ shadow vertex simplex paths. Our upper bound complements the lower bounds of Allamigeon, Benchimol, Gaubert, and Joswig (SIAGA 2018), and Allamigeon, Gaubert, and Vandame (STOC 2022), who constructed linear programs with exponential SLC. Finally, we show that each iteration of our IPM can be implemented in strongly polynomial time. Along the way, we develop a deterministic algorithm that approximates the singular value decomposition of a matrix in strongly polynomial time to high accuracy, which may be of independent interest.
We analyze a Discontinuous Galerkin method for a problem with linear advection-reaction and $p$-type diffusion, with Sobolev indices $p\in (1, \infty)$. The discretization of the diffusion term is based on the full gradient including jump liftings and interior-penalty stabilization while, for the advective contribution, we consider a strengthened version of the classical upwind scheme. The developed error estimates track the dependence of the local contributions to the error on local P\'eclet numbers. A set of numerical tests supports the theoretical derivations.
We consider the problem of sketching a set valuation function, which is defined as the expectation of a valuation function of independent random item values. We show that for monotone subadditive or submodular valuation functions satisfying a weak homogeneity condition, or certain other conditions, there exist discretized distributions of item values with $O(k\log(k))$ support sizes that yield a sketch valuation function which is a constant-factor approximation, for any value query for a set of items of cardinality less than or equal to $k$. The discretized distributions can be efficiently computed by an algorithm for each item's value distribution separately. Our results hold under conditions that accommodate a wide range of valuation functions arising in applications, such as the value of a team corresponding to the best performance of a team member, constant elasticity of substitution production functions exhibiting diminishing returns used in economics and consumer theory, and others. Sketch valuation functions are particularly valuable for finding approximate solutions to optimization problems such as best set selection and welfare maximization. They enable computationally efficient evaluation of approximate value oracle queries and provide an approximation guarantee for the underlying optimization problem.
Using a generating function approach, a computationally tractable expression is derived to predict the frame error rate arising at the output of the binary symmetric channel when a number of outer Reed-Solomon codes are concatenated with a number of inner Bose-Ray-Chaudhuri-Hocquenghem codes, thereby obviating the need for time-consuming Monte Carlo simulations. Measuring (a) code performance via the gap to the Shannon limit, (b) decoding complexity via an estimate of the number of operations per decoded bit, and (c) decoding latency by the overall frame length, a code search is performed to determine the Pareto frontier for performance-complexity-latency trade-offs.
{We analyze a general Implicit-Explicit (IMEX) time discretization for the compressible Euler equations of gas dynamics, showing that they are asymptotic-preserving (AP) in the low Mach number limit. The analysis is carried out for a general equation of state (EOS). We consider both a single asymptotic length scale and two length scales. We then show that, when coupling these time discretizations with a Discontinuous Galerkin (DG) space discretization with appropriate fluxes, an all Mach number numerical method is obtained. A number of relevant benchmarks for ideal gases and their non-trivial extension to non-ideal EOS validate the performed analysis.
We prove that functions over the reals computable in polynomial time can be characterised using discrete ordinary differential equations (ODE), also known as finite differences. We also provide a characterisation of functions computable in polynomial space over the reals. In particular, this covers space complexity, while existing characterisations were only able to cover time complexity, and were restricted to functions over the integers. We prove furthermore that no artificial sign or test function is needed even for time complexity. At a technical level, this is obtained by proving that Turing machines can be simulated with analytic discrete ordinary differential equations. We believe this result opens the way to many applications, as it opens the possibility of programming with ODEs, with an underlying well-understood time and space complexity.
We consider a model selection problem for structural equation modeling (SEM) with latent variables for diffusion processes based on high-frequency data. First, we propose the quasi-Akaike information criterion of the SEM and study the asymptotic properties. Next, we consider the situation where the set of competing models includes some misspecified parametric models. It is shown that the probability of choosing the misspecified models converges to zero. Furthermore, examples and simulation results are given.
Inner products of neural network feature maps arises in a wide variety of machine learning frameworks as a method of modeling relations between inputs. This work studies the approximation properties of inner products of neural networks. It is shown that the inner product of a multi-layer perceptron with itself is a universal approximator for symmetric positive-definite relation functions. In the case of asymmetric relation functions, it is shown that the inner product of two different multi-layer perceptrons is a universal approximator. In both cases, a bound is obtained on the number of neurons required to achieve a given accuracy of approximation. In the symmetric case, the function class can be identified with kernels of reproducing kernel Hilbert spaces, whereas in the asymmetric case the function class can be identified with kernels of reproducing kernel Banach spaces. Finally, these approximation results are applied to analyzing the attention mechanism underlying Transformers, showing that any retrieval mechanism defined by an abstract preorder can be approximated by attention through its inner product relations. This result uses the Debreu representation theorem in economics to represent preference relations in terms of utility functions.
We introduce PaLEnTIR, a significantly enhanced parametric level-set (PaLS) method addressing the restoration and reconstruction of piecewise constant objects. Our key contribution involves a unique PaLS formulation utilizing a single level-set function to restore scenes containing multi-contrast piecewise-constant objects without requiring knowledge of the number of objects or their contrasts. Unlike standard PaLS methods employing radial basis functions (RBFs), our model integrates anisotropic basis functions (ABFs), thereby expanding its capacity to represent a wider class of shapes. Furthermore, PaLEnTIR improves the conditioning of the Jacobian matrix, required as part of the parameter identification process, and consequently accelerates optimization methods. We validate PaLEnTIR's efficacy through diverse experiments encompassing sparse and limited angle of view X-ray computed tomography (2D and 3D), nonlinear diffuse optical tomography (DOT), denoising, and deconvolution tasks using both real and simulated data sets.
小貼士
登錄享
注冊地址： 北京市海淀區羊坊店路18號2幢3層301-191