In this paper, we consider the decentralized, stochastic nonconvex strongly-concave (NCSC) minimax problem with nonsmooth regularization terms on both primal and dual variables, wherein a network of $m$ computing agents collaborate via peer-to-peer communications. We consider when the coupling function is in expectation or finite-sum form and the double regularizers are convex functions, applied separately to the primal and dual variables. Our algorithmic framework introduces a Lagrangian multiplier to eliminate the consensus constraint on the dual variable. Coupling this with variance-reduction (VR) techniques, our proposed method, entitled VRLM, by a single neighbor communication per iteration, is able to achieve an $\mathcal{O}(\kappa^3\varepsilon^{-3})$ sample complexity under the general stochastic setting, with either a big-batch or small-batch VR option, where $\kappa$ is the condition number of the problem and $\varepsilon$ is the desired solution accuracy. With a big-batch VR, we can additionally achieve $\mathcal{O}(\kappa^2\varepsilon^{-2})$ communication complexity. Under the special finite-sum setting, our method with a big-batch VR can achieve an $\mathcal{O}(n + \sqrt{n} \kappa^2\varepsilon^{-2})$ sample complexity and $\mathcal{O}(\kappa^2\varepsilon^{-2})$ communication complexity, where $n$ is the number of components in the finite sum. All complexity results match the best-known results achieved by a few existing methods for solving special cases of the problem we consider. To the best of our knowledge, this is the first work which provides convergence guarantees for NCSC minimax problems with general convex nonsmooth regularizers applied to both the primal and dual variables in the decentralized stochastic setting. Numerical experiments are conducted on two machine learning problems. Our code is downloadable from //github.com/RPI-OPT/VRLM.
相關內容
In this paper, we study functional approximations where we choose the so-called radial basis function method and more specifically, quasi-interpolation. From the various available approaches to the latter, we form new quasi-Lagrange functions when the orders of the singularities of the radial function's Fourier transforms at zero do not match the parity of the dimension of the space, and therefore new expansions and coefficients are needed to overcome this problem. We develop explicit constructions of infinite Fourier expansions that provide these coefficients and make an extensive comparison of the approximation qualities and - with a particular focus - polynomial precision and uniform approximation order of the various formulae. One of the interesting observations concerns the link between algebraic conditions of expansion coefficients and analytic properties of localness and convergence.
Multiscale constitutive framework of 1D blood flow modeling: Asymptotic limits and numerical methods
In this paper, a multiscale constitutive framework for one-dimensional blood flow modeling is presented and discussed. By analyzing the asymptotic limits of the proposed model, it is shown that different types of blood propagation phenomena in arteries and veins can be described through an appropriate choice of scaling parameters, which are related to distinct characterizations of the fluid-structure interaction mechanism (whether elastic or viscoelastic) that exist between vessel walls and blood flow. In these asymptotic limits, well-known blood flow models from the literature are recovered. Additionally, by analyzing the perturbation of the local elastic equilibrium of the system, a new viscoelastic blood flow model is derived. The proposed approach is highly flexible and suitable for studying the human cardiovascular system, which is composed of vessels with high morphological and mechanical variability. The resulting multiscale hyperbolic model of blood flow is solved using an asymptotic-preserving Implicit-Explicit Runge-Kutta Finite Volume method, which ensures the consistency of the numerical scheme with the different asymptotic limits of the mathematical model without affecting the choice of the time step by restrictions related to the smallness of the scaling parameters. Several numerical tests confirm the validity of the proposed methodology, including a case study investigating the hemodynamics of a thoracic aorta in the presence of a stent.
In this paper, we develop a non-asymptotic local normal approximation for multinomial probabilities. First, we use it to find non-asymptotic total variation bounds between the measures induced by uniformly jittered multinomials and the multivariate normals with the same means and covariances. From the total variation bounds, we also derive a comparison of the cumulative distribution functions and quantile coupling inequalities between Pearson's chi-square statistic (written as the normalized quadratic form of a multinomial vector) and its multivariate normal analogue. We apply our results to find confidence intervals for the negative entropy of discrete distributions. Our method can be applied more generally to find confidence intervals for strictly convex functions of the weights of discrete distributions.
In this paper, we introduce a new first-order mixture integer-valued threshold autoregressive process, based on the binomial and negative binomial thinning operators. Basic probabilistic and statistical properties of this model are discussed. Conditional least squares (CLS) and conditional maximum likelihood (CML) estimators are derived and the asymptotic properties of the estimators are established. The inference for the threshold parameter is obtained based on the CLS and CML score functions. Moreover, the Wald test is applied to detect the existence of the piecewise structure. Simulation studies are considered, along with an application: the number of criminal mischief incidents in the Pittsburgh dataset.
Bayesian binary regression is a prosperous area of research due to the computational challenges encountered by currently available methods either for high-dimensional settings or large datasets, or both. In the present work, we focus on the expectation propagation (EP) approximation of the posterior distribution in Bayesian probit regression under a multivariate Gaussian prior distribution. Adapting more general derivations in Anceschi et al. (2023), we show how to leverage results on the extended multivariate skew-normal distribution to derive an efficient implementation of the EP routine having a per-iteration cost that scales linearly in the number of covariates. This makes EP computationally feasible also in challenging high-dimensional settings, as shown in a detailed simulation study.
This paper proposes a Cartesian grid-based boundary integral method for efficiently and stably solving two representative moving interface problems, the Hele-Shaw flow and the Stefan problem. Elliptic and parabolic partial differential equations (PDEs) are reformulated into boundary integral equations and are then solved with the matrix-free generalized minimal residual (GMRES) method. The evaluation of boundary integrals is performed by solving equivalent and simple interface problems with finite difference methods, allowing the use of fast PDE solvers, such as fast Fourier transform (FFT) and geometric multigrid methods. The interface curve is evolved utilizing the $\theta-L$ variables instead of the more commonly used $x-y$ variables. This choice simplifies the preservation of mesh quality during the interface evolution. In addition, the $\theta-L$ approach enables the design of efficient and stable time-stepping schemes to remove the stiffness that arises from the curvature term. Ample numerical examples, including simulations of complex viscous fingering and dendritic solidification problems, are presented to showcase the capability of the proposed method to handle challenging moving interface problems.
This paper explores an iterative coupling approach to solve thermo-poroelasticity problems, with its application as a high-fidelity discretization utilizing finite elements during the training of projection-based reduced order models. One of the main challenges in addressing coupled multi-physics problems is the complexity and computational expenses involved. In this study, we introduce a decoupled iterative solution approach, integrated with reduced order modeling, aimed at augmenting the efficiency of the computational algorithm. The iterative coupling technique we employ builds upon the established fixed-stress splitting scheme that has been extensively investigated for Biot's poroelasticity. By leveraging solutions derived from this coupled iterative scheme, the reduced order model employs an additional Galerkin projection onto a reduced basis space formed by a small number of modes obtained through proper orthogonal decomposition. The effectiveness of the proposed algorithm is demonstrated through numerical experiments, showcasing its computational prowess.
In this paper, we propose a human trajectory prediction model that combines a Long Short-Term Memory (LSTM) network with an attention mechanism. To do that, we use attention scores to determine which parts of the input data the model should focus on when making predictions. Attention scores are calculated for each input feature, with a higher score indicating the greater significance of that feature in predicting the output. Initially, these scores are determined for the target human position, velocity, and their neighboring individual's positions and velocities. By using attention scores, our model can prioritize the most relevant information in the input data and make more accurate predictions. We extract attention scores from our attention mechanism and integrate them into the trajectory prediction module to predict human future trajectories. To achieve this, we introduce a new neural layer that processes attention scores after extracting them and concatenates them with positional information. We evaluate our approach on the publicly available ETH and UCY datasets and measure its performance using the final displacement error (FDE) and average displacement error (ADE) metrics. We show that our modified algorithm performs better than the Social LSTM in predicting the future trajectory of pedestrians in crowded spaces. Specifically, our model achieves an improvement of 6.2% in ADE and 6.3% in FDE compared to the Social LSTM results in the literature.
We introduce new control-volume finite-element discretization schemes suitable for solving the Stokes problem. Within a common framework, we present different approaches for constructing such schemes. The first and most established strategy employs a non-overlapping partitioning into control volumes. The second represents a new idea by splitting into two sets of control volumes, the first set yielding a partition of the domain and the second containing the remaining overlapping control volumes required for stability. The third represents a hybrid approach where finite volumes are combined with finite elements based on a hierarchical splitting of the ansatz space. All approaches are based on typical finite element function spaces but yield locally mass and momentum conservative discretization schemes that can be interpreted as finite volume schemes. We apply all strategies to the inf-sub stable MINI finite-element pair. Various test cases, including convergence tests and the numerical observation of the boundedness of the number of preconditioned Krylov solver iterations, as well as more complex scenarios of flow around obstacles or through a three-dimensional vessel bifurcation, demonstrate the stability and robustness of the schemes.
Recent advances in 3D fully convolutional networks (FCN) have made it feasible to produce dense voxel-wise predictions of volumetric images. In this work, we show that a multi-class 3D FCN trained on manually labeled CT scans of several anatomical structures (ranging from the large organs to thin vessels) can achieve competitive segmentation results, while avoiding the need for handcrafting features or training class-specific models. To this end, we propose a two-stage, coarse-to-fine approach that will first use a 3D FCN to roughly define a candidate region, which will then be used as input to a second 3D FCN. This reduces the number of voxels the second FCN has to classify to ~10% and allows it to focus on more detailed segmentation of the organs and vessels. We utilize training and validation sets consisting of 331 clinical CT images and test our models on a completely unseen data collection acquired at a different hospital that includes 150 CT scans, targeting three anatomical organs (liver, spleen, and pancreas). In challenging organs such as the pancreas, our cascaded approach improves the mean Dice score from 68.5 to 82.2%, achieving the highest reported average score on this dataset. We compare with a 2D FCN method on a separate dataset of 240 CT scans with 18 classes and achieve a significantly higher performance in small organs and vessels. Furthermore, we explore fine-tuning our models to different datasets. Our experiments illustrate the promise and robustness of current 3D FCN based semantic segmentation of medical images, achieving state-of-the-art results. Our code and trained models are available for download: //github.com/holgerroth/3Dunet_abdomen_cascade.
注冊地址： 北京市海淀區羊坊店路18號2幢3層301-191