In this paper, we develop a gradient recovery based linear (GRBL) finite element method (FEM) and a Hessian recovery based linear (HRBL) FEM for second order elliptic equations in non-divergence form. The elliptic equation is casted into a symmetric non-divergence weak formulation, in which second order derivatives of the unknown function are involved. We use gradient and Hessian recovery operators to calculate the second order derivatives of linear finite element approximations. Although, thanks to low degrees of freedom (DOF) of linear elements, the implementation of the proposed schemes is easy and straightforward, the performances of the methods are competitive. The unique solvability and the $H^2$ seminorm error estimate of the GRBL scheme are rigorously proved. Optimal error estimates in both the $L^2$ norm and the $H^1$ seminorm have been proved when the coefficient is diagonal, which have been confirmed by numerical experiments. Superconvergence in errors has also been observed. Moreover, our methods can handle computational domains with curved boundaries without loss of accuracy from approximation of boundaries. Finally, the proposed numerical methods have been successfully applied to solve fully nonlinear Monge-Amp\`{e}re equations.
相關內容
This paper introduces a novel algorithm, the Perturbed Proximal Preconditioned SPIDER algorithm (3P-SPIDER), designed to solve finite sum non-convex composite optimization. It is a stochastic Variable Metric Forward-Backward algorithm, which allows approximate preconditioned forward operator and uses a variable metric proximity operator as the backward operator; it also proposes a mini-batch strategy with variance reduction to address the finite sum setting. We show that 3P-SPIDER extends some Stochastic preconditioned Gradient Descent-based algorithms and some Incremental Expectation Maximization algorithms to composite optimization and to the case the forward operator can not be computed in closed form. We also provide an explicit control of convergence in expectation of 3P-SPIDER, and study its complexity in order to satisfy the epsilon-approximate stationary condition. Our results are the first to combine the composite non-convex optimization setting, a variance reduction technique to tackle the finite sum setting by using a minibatch strategy and, to allow deterministic or random approximations of the preconditioned forward operator. Finally, through an application to inference in a logistic regression model with random effects, we numerically compare 3P-SPIDER to other stochastic forward-backward algorithms and discuss the role of some design parameters of 3P-SPIDER.
In this paper, we propose a Dimension-Reduced Second-Order Method (DRSOM) for convex and nonconvex (unconstrained) optimization. Under a trust-region-like framework, our method preserves the convergence of the second-order method while using only curvature information in a few directions. Consequently, the computational overhead of our method remains comparable to the first-order such as the gradient descent method. Theoretically, we show that the method has a local quadratic convergence and a global convergence rate of $O(\epsilon^{-3/2})$ to satisfy the first-order and second-order conditions if the subspace satisfies a commonly adopted approximated Hessian assumption. We further show that this assumption can be removed if we perform one \emph{corrector step} (using a Krylov method, for example) periodically at the end stage of the algorithm. The applicability and performance of DRSOM are exhibited by various computational experiments, particularly in machine learning and deep learning. For neural networks, our preliminary implementation seems to gain computational advantages in terms of training accuracy and iteration complexity over state-of-the-art first-order methods such as SGD and ADAM.
We design a monotone meshfree finite difference method for linear elliptic equations in the non-divergence form on point clouds via a nonlocal relaxation method. The key idea is a novel combination of a nonlocal integral relaxation of the PDE problem with a robust meshfree discretization on point clouds. Minimal positive stencils are obtained through a local $l_1$-type optimization procedure that automatically guarantees the stability and, therefore, the convergence of the meshfree discretization for linear elliptic equations. A major theoretical contribution is the existence of consistent and positive stencils for a given point cloud geometry. We provide sufficient conditions for the existence of positive stencils by finding neighbors within an ellipse (2d) or ellipsoid (3d) surrounding each interior point, generalizing the study for Poisson's equation by Seibold in 2008. It is well-known that wide stencils are in general needed for constructing consistent and monotone finite difference schemes for linear elliptic equations. Our result represents a significant improvement in the stencil width estimate for positive-type finite difference methods for linear elliptic equations in the near-degenerate regime (when the ellipticity constant becomes small), compared to previously known works in this area. Numerical algorithms and practical guidance are provided with an eye on the case of small ellipticity constant. At the end, we present numerical results for the performance of our method in both 2d and 3d, examining a range of ellipticity constants including the near-degenerate regime.
On the role of surrogates in the efficient estimation of treatment effects with limited outcome data
In many investigations, the primary outcome of interest is difficult or expensive to collect. Examples include long-term health effects of medical interventions, measurements requiring expensive testing or follow-up, and outcomes only measurable on small panels as in marketing. This reduces effective sample sizes for estimating the average treatment effect (ATE). However, there is often an abundance of observations on surrogate outcomes not of primary interest, such as short-term health effects or online-ad click-through. We study the role of such surrogate observations in the efficient estimation of treatment effects. To quantify their value, we derive the semiparametric efficiency bounds on ATE estimation with and without the presence of surrogates and several intermediary settings. The difference between these characterizes the efficiency gains from optimally leveraging surrogates. We study two regimes: when the number of surrogate observations is comparable to primary-outcome observations and when the former dominates the latter. We take an agnostic missing-data approach circumventing strong surrogate conditions previously assumed. To leverage surrogates' efficiency gains, we develop efficient ATE estimation and inference based on flexible machine-learning estimates of nuisance functions appearing in the influence functions we derive. We empirically demonstrate the gains by studying the long-term earnings effect of job training.
We consider a mixed dimensional elliptic partial differential equation posed in a bulk domain with a large number of embedded interfaces. In particular, we study well-posedness of the problem and regularity of the solution. We also propose a fitted finite element approximation and prove an a priori error bound. For the solution of the arising linear system we propose and analyze an iterative method based on subspace decomposition. Finally, we present numerical experiments and achieve rapid convergence using the proposed preconditioner, confirming our theoretical findings.
Neural operators, which emerge as implicit solution operators of hidden governing equations, have recently become popular tools for learning responses of complex real-world physical systems. Nevertheless, the majority of neural operator applications has thus far been data-driven, which neglects the intrinsic preservation of fundamental physical laws in data. In this paper, we introduce a novel integral neural operator architecture, to learn physical models with fundamental conservation laws automatically guaranteed. In particular, by replacing the frame-dependent position information with its invariant counterpart in the kernel space, the proposed neural operator is by design translation- and rotation-invariant, and consequently abides by the conservation laws of linear and angular momentums. As applications, we demonstrate the expressivity and efficacy of our model in learning complex material behaviors from both synthetic and experimental datasets, and show that, by automatically satisfying these essential physical laws, our learned neural operator is not only generalizable in handling translated and rotated datasets, but also achieves state-of-the-art accuracy and efficiency as compared to baseline neural operator models.
Stochastic gradient methods have enabled variational inference for high-dimensional models. However, the steepest ascent direction in the parameter space of a statistical model is actually given by the natural gradient which premultiplies the widely used Euclidean gradient by the inverse Fisher information. Use of natural gradients can improve convergence, but inverting the Fisher information matrix is daunting in high-dimensions. In Gaussian variational approximation, natural gradient updates of the mean and precision of the normal distribution can be derived analytically, but do not ensure that the precision matrix remains positive definite. To tackle this issue, we consider Cholesky decomposition of the covariance or precision matrix, and derive analytic natural gradient updates of the Cholesky factor, which depend on either the first or second derivative of the log posterior density. Efficient natural gradient updates of the Cholesky factor are also derived under sparsity constraints representing different posterior correlation structures. As Adam's adaptive learning rate does not work well with natural gradients, we propose stochastic normalized natural gradient ascent with momentum. The efficiency of proposed methods are demonstrated using logistic regression and generalized linear mixed models.
We present a new perspective on the use of weighted essentially nonoscillatory (WENO) reconstructions in high-order methods for scalar hyperbolic conservation laws. The main focus of this work is on nonlinear stabilization of continuous Galerkin (CG) approximations. The proposed methodology also provides an interesting alternative to WENO-based limiters for discontinuous Galerkin (DG) methods. Unlike Runge--Kutta DG schemes that overwrite finite element solutions with WENO reconstructions, our approach uses a reconstruction-based smoothness sensor to blend the numerical viscosity operators of high- and low-order stabilization terms. The so-defined WENO approximation introduces low-order nonlinear diffusion in the vicinity of shocks, while preserving the high-order accuracy of a linearly stable baseline discretization in regions where the exact solution is sufficiently smooth. The underlying reconstruction procedure performs Hermite interpolation on stencils consisting of a mesh cell and its neighbors. The amount of numerical dissipation depends on the relative differences between partial derivatives of reconstructed candidate polynomials and those of the underlying finite element approximation. All derivatives are taken into account by the employed smoothness sensor. To assess the accuracy of our CG-WENO scheme, we derive error estimates and perform numerical experiments. In particular, we prove that the consistency error of the nonlinear stabilization is of the order $p+1/2$, where $p$ is the polynomial degree. This estimate is optimal for general meshes. For uniform meshes and smooth exact solutions, the experimentally observed rate of convergence is as high as $p+1$.
Sparse principal component analysis (SPCA) has been widely used for dimensionality reduction and feature extraction in high-dimensional data analysis. Despite there are many methodological and theoretical developments in the past two decades, the theoretical guarantees of the popular SPCA algorithm proposed by Zou, Hastie & Tibshirani (2006) based on the elastic net are still unknown. We aim to close this important theoretical gap in this paper. We first revisit the SPCA algorithm of Zou et al. (2006) and present our implementation. Also, we study a computationally more efficient variant of the SPCA algorithm in Zou et al. (2006) that can be considered as the limiting case of SPCA. We provide the guarantees of convergence to a stationary point for both algorithms. We prove that, under a sparse spiked covariance model, both algorithms can recover the principal subspace consistently under mild regularity conditions. We show that their estimation error bounds match the best available bounds of existing works or the minimax rates up to some logarithmic factors. Moreover, we demonstrate the numerical performance of both algorithms in simulation studies.
Sampling methods (e.g., node-wise, layer-wise, or subgraph) has become an indispensable strategy to speed up training large-scale Graph Neural Networks (GNNs). However, existing sampling methods are mostly based on the graph structural information and ignore the dynamicity of optimization, which leads to high variance in estimating the stochastic gradients. The high variance issue can be very pronounced in extremely large graphs, where it results in slow convergence and poor generalization. In this paper, we theoretically analyze the variance of sampling methods and show that, due to the composite structure of empirical risk, the variance of any sampling method can be decomposed into \textit{embedding approximation variance} in the forward stage and \textit{stochastic gradient variance} in the backward stage that necessities mitigating both types of variance to obtain faster convergence rate. We propose a decoupled variance reduction strategy that employs (approximate) gradient information to adaptively sample nodes with minimal variance, and explicitly reduces the variance introduced by embedding approximation. We show theoretically and empirically that the proposed method, even with smaller mini-batch sizes, enjoys a faster convergence rate and entails a better generalization compared to the existing methods.
注冊地址： 北京市海淀區羊坊店路18號2幢3層301-191