Recently, Arjevani et al. [1] established a lower bound of iteration complexity for the first-order optimization under an $L$-smooth condition and a bounded noise variance assumption. However, a thorough review of existing literature on Adam's convergence reveals a noticeable gap: none of them meet the above lower bound. In this paper, we close the gap by deriving a new convergence guarantee of Adam, with only an $L$-smooth condition and a bounded noise variance assumption. Our results remain valid across a broad spectrum of hyperparameters. Especially with properly chosen hyperparameters, we derive an upper bound of the iteration complexity of Adam and show that it meets the lower bound for first-order optimizers. To the best of our knowledge, this is the first to establish such a tight upper bound for Adam's convergence. Our proof utilizes novel techniques to handle the entanglement between momentum and adaptive learning rate and to convert the first-order term in the Descent Lemma to the gradient norm, which may be of independent interest.
相關內容
We consider relational semantics (R-models) for the Lambek calculus extended with intersection and explicit constants for zero and unit. For its variant without constants and a restriction which disallows empty antecedents, Andreka and Mikulas (1994) prove strong completeness. We show that it fails without this restriction, but, on the other hand, prove weak completeness for non-standard interpretation of constants. For the standard interpretation, even weak completeness fails. The weak completeness result extends to an infinitary setting, for so-called iterative divisions (Kleene star under division). We also prove strong completeness results for product-free fragments.
While the problem of computing the genus of a knot is now fairly well understood, no algorithm is known for its four-dimensional variants, both in the smooth and in the topological locally flat category. In this article, we investigate a class of knots and links called Hopf arborescent links, which are obtained as the boundaries of some iterated plumbings of Hopf bands. We show that for such links, computing the genus defects, which measure how much the four-dimensional genera differ from the classical genus, is decidable. Our proof is non-constructive, and is obtained by proving that Seifert surfaces of Hopf arborescent links under a relation of minors defined by containment of their Seifert surfaces form a well-quasi-order.
Current quantum computers can only solve optimization problems of a very limited size. For larger problems, decomposition methods are required in which the original problem is broken down into several smaller sub-problems. These are then solved on the quantum computer and their solutions are merged into a final solution for the original problem. Often, these decomposition methods do not take the specific problem structure into account. In this paper, we present a tailored method using a divide-and-conquer strategy to solve the number partitioning problem (NPP) with a large number of variables. The idea is to perform a specialized decomposition into smaller NPPs, which can be solved on a quantum computer, and then recombine the results into another small auxiliary NPP. Solving this auxiliary problem yields an approximate solution of the original larger problem. We experimentally verify that our method allows to solve NPPs with over a thousand variables using a D-Wave quantum annealer.
We study the seeded domino problem, the recurring domino problem and the $k$-SAT problem on finitely generated groups. These problems are generalization of their original versions on $\mathbb{Z}^2$ that were shown to be undecidable using the domino problem. We show that the seeded and recurring domino problems on a group are invariant under changes in the generating set, are many-one reduced from the respective problems on subgroups, and are positive equivalent to the problems on finite index subgroups. This leads to showing that the recurring domino problem is decidable for free groups. Coupled with the invariance properties, we conjecture that the only groups in which the seeded and recurring domino problems are decidable are virtually free groups. In the case of the $k$-SAT problem, we introduce a new generalization that is compatible with decision problems on finitely generated groups. We show that the subgroup membership problem many-one reduces to the $2$-SAT problem, that in certain cases the $k$-SAT problem many one reduces to the domino problem, and finally that the domino problem reduces to $3$-SAT for the class of scalable groups.
Given a set of $n$ points in the Euclidean plane, the $k$-MinSumRadius problem asks to cover this point set using $k$ disks with the objective of minimizing the sum of the radii of the disks. After a long line of research on related problems, it was finally discovered that this problem admits a polynomial time algorithm [GKKPV~'12]; however, the running time of this algorithm is $O(n^{881})$, and its relevance is thereby mostly of theoretical nature. A practically and structurally interesting special case of the $k$-MinSumRadius problem is that of small $k$. For the $2$-MinSumRadius problem, a near-quadratic time algorithm with expected running time $O(n^2 \log^2 n \log^2 \log n)$ was given over 30 years ago [Eppstein~'92]. We present the first improvement of this result, namely, a near-linear time algorithm to compute the $2$-MinSumRadius that runs in expected $O(n \log^2 n \log^2 \log n)$ time. We generalize this result to any constant dimension $d$, for which we give an $O(n^{2-1/(\lceil d/2\rceil + 1) + \varepsilon})$ time algorithm. Additionally, we give a near-quadratic time algorithm for $3$-MinSumRadius in the plane that runs in expected $O(n^2 \log^2 n \log^2 \log n)$ time. All of these algorithms rely on insights that uncover a surprisingly simple structure of optimal solutions: we can specify a linear number of lines out of which one separates one of the clusters from the remaining clusters in an optimal solution.
The major advantage of reduced magnetic vector potential formulations (RMVPs) is that complicated coil structures do not need to be resolved by a computational mesh. Instead, they are modeled by thin wires, whose source field is included into the simulation model along Biot-Savart's law. Such an approach has already been successfully employed in ROXIE for the simulation of superconducting Large Hadron Collider magnets at CERN. This work presents an updated RMVP approach, which significantly outperforms the original method. The updated formulation is postulated, implemented, verified, compared to the original formulation, and applied for the simulation of a quadrupole magnet. The promising results of this work encourage further investigation towards an updated simulation framework for next-generation accelerator magnets.
Macro-level modeling is still the dominant approach in many demographic applications because of its simplicity. Individual-level models, on the other hand, provide a more comprehensive understanding of observed patterns; however, their estimation using real data has remained a challenge. The approach we introduce in this article attempts to overcome this limitation. Using likelihood-free inference techniques, we show that it is possible to estimate the parameters of a simple but demographically interpretable individual-level model of the reproductive process from a set of aggregate fertility rates. By estimating individual-level quantities from widely available aggregate data, this approach can contribute to a better understanding of reproductive behavior and its driving mechanisms. It also allows for a more direct link between individual-level and population-level processes. We illustrate our approach using data from three natural fertility populations.
A robust authentication and authorization mechanism is imperative in modular system development, where modularity and modular thinking are pivotal. Traditional systems often employ identity modules responsible for authentication and token issuance. Tokens, representing user credentials, offer advantages such as reduced reliance on passwords, limited lifespan, and scoped access. Despite these benefits, the "bearer token" problem persists, leaving systems vulnerable to abuse if tokens are compromised. We propose a token-based authentication mechanism addressing modular systems' critical bearer token problem. The proposed mechanism includes a novel RAF (Recursive Augmented Fernet) token, a blacklist component, and a policy enforcer component. RAF tokens are one-time-use tokens, like tickets. They carry commands, and the receiver of an RAF token can issue new tokens using the received RAF token. The blacklist component guarantees an RAF token can not be approved more than once, and the policy enforcer checks the compatibility of commands carried by an RAF token. We introduce two variations of RAF tokens: User-tied RAF, offering simplicity and compatibility, and Fully-tied RAF, providing enhanced security through service-specific secret keys. We thoroughly discuss the security guarantees, technical definitions, and construction of RAF tokens backed by game-based proofs. We demonstrate a proof of concept in the context of OpenStack, involving modifications to Keystone and creating an RAFT library. The experimental results reveal minimal overhead in typical scenarios, establishing the practicality and effectiveness of RAF. Our experiments show that the RAF mechanism beats the idea of using short-life Fernet tokens while providing much better security.
Bayesian Experimental Design (BED), which aims to find the optimal experimental conditions for Bayesian inference, is usually posed as to optimize the expected information gain (EIG). The gradient information is often needed for efficient EIG optimization, and as a result the ability to estimate the gradient of EIG is essential for BED problems. The primary goal of this work is to develop methods for estimating the gradient of EIG, which, combined with the stochastic gradient descent algorithms, result in efficient optimization of EIG. Specifically, we first introduce a posterior expected representation of the EIG gradient with respect to the design variables. Based on this, we propose two methods for estimating the EIG gradient, UEEG-MCMC that leverages posterior samples generated through Markov Chain Monte Carlo (MCMC) to estimate the EIG gradient, and BEEG-AP that focuses on achieving high simulation efficiency by repeatedly using parameter samples. Theoretical analysis and numerical studies illustrate that UEEG-MCMC is robust agains the actual EIG value, while BEEG-AP is more efficient when the EIG value to be optimized is small. Moreover, both methods show superior performance compared to several popular benchmarks in our numerical experiments.
Blended modeling is an emerging paradigm involving seamless interaction between multiple notations for the same underlying modeling language. We focus on a model-driven engineering (MDE) approach based on meta-models to develop textual languages to improve the blended modeling capabilities of modeling tools. In this thesis, we propose an approach that can support the co-evolution of meta-models and grammars as language engineers develop textual languages in a meta-model-based MDE setting. Firstly, we comprehensively report on the challenges and limitations of modeling tools that support blended modeling, as well as opportunities to improve them. Second, we demonstrate how language engineers can extend Xtext's generator capabilities according to their needs. Third, we propose a semi-automatic method to transform a language with a generated grammar into a Python-style language. Finally, we provide a solution (i.e., GrammarOptimizer) that can support rapid prototyping of languages in different styles and the co-evolution of meta-models and grammars of evolving languages.
注冊地址： 北京市海淀區羊坊店路18號2幢3層301-191