Score-based diffusion models have emerged as one of the most promising frameworks for deep generative modelling, due to their state-of-the art performance in many generation tasks while relying on mathematical foundations such as stochastic differential equations (SDEs) and ordinary differential equations (ODEs). Empirically, it has been reported that ODE based samples are inferior to SDE based samples. In this paper we rigorously describe the range of dynamics and approximations that arise when training score-based diffusion models, including the true SDE dynamics, the neural approximations, the various approximate particle dynamics that result, as well as their associated Fokker--Planck equations and the neural network approximations of these Fokker--Planck equations. We systematically analyse the difference between the ODE and SDE dynamics of score-based diffusion models, and link it to an associated Fokker--Planck equation. We derive a theoretical upper bound on the Wasserstein 2-distance between the ODE- and SDE-induced distributions in terms of a Fokker--Planck residual. We also show numerically that conventional score-based diffusion models can exhibit significant differences between ODE- and SDE-induced distributions which we demonstrate using explicit comparisons. Moreover, we show numerically that reducing the Fokker--Planck residual by adding it as an additional regularisation term leads to closing the gap between ODE- and SDE-induced distributions. Our experiments suggest that this regularisation can improve the distribution generated by the ODE, however that this can come at the cost of degraded SDE sample quality.
相關內容
ACM/IEEE第23屆模型驅動工程語言和系統國際會議，是模型驅動軟件和系統工程的首要會議系列，由ACM-SIGSOFT和IEEE-TCSE支持組織。自1998年以來，模型涵蓋了建模的各個方面，從語言和方法到工具和應用程序。模特的參加者來自不同的背景，包括研究人員、學者、工程師和工業專業人士。MODELS 2019是一個論壇，參與者可以圍繞建模和模型驅動的軟件和系統交流前沿研究成果和創新實踐經驗。今年的版本將為建模社區提供進一步推進建模基礎的機會，并在網絡物理系統、嵌入式系統、社會技術系統、云計算、大數據、機器學習、安全、開源等新興領域提出建模的創新應用以及可持續性。
官網鏈接： ·
Tikhonov正則化
·
INFORMS
·
正則化項
·
操作
·
High-frequency issues have been remarkably challenges in numerical methods for partial differential equations. In this paper, a learning based numerical method (LbNM) is proposed for Helmholtz equation with high frequency. The main novelty is using Tikhonov regularization method to stably learn the solution operator by utilizing relevant information especially the fundamental solutions. Then applying the solution operator to a new boundary input could quickly update the solution. Based on the method of fundamental solutions and the quantitative Runge approximation, we give the error estimate. This indicates interpretability and generalizability of the present method. Numerical results validates the error analysis and demonstrates the high-precision and high-efficiency features.
Kleene's computability theory based on the S1-S9 computation schemes constitutes a model for computing with objects of any finite type and extends Turing's 'machine model' which formalises computing with real numbers. A fundamental distinction in Kleene's framework is between normal and non-normal functionals where the former compute the associated Kleene quantifier $\exists^n$ and the latter do not. Historically, the focus was on normal functionals, but recently new non-normal functionals have been studied based on well-known theorems, the weakest among which seems to be the uncountability of the reals. These new non-normal functionals are fundamentally different from historical examples like Tait's fan functional: the latter is computable from $\exists^2$, while the former are computable in $\exists^3$ but not in weaker oracles. Of course, there is a great divide or abyss separating $\exists^2$ and $\exists^3$ and we identify slight variations of our new non-normal functionals that are again computable in $\exists^2$, i.e. fall on different sides of this abyss. Our examples are based on mainstream mathematical notions, like quasi-continuity, Baire classes, bounded variation, and semi-continuity from real analysis.
In this study, we investigate an anisotropic weakly over-penalised symmetric interior penalty method for the Stokes equation {on convex domains}. Our approach is a simple discontinuous Galerkin method similar to the Crouzeix--Raviart finite element method. As our primary contribution, we show a new proof for the consistency term, which allows us to obtain an estimate of the anisotropic consistency error. The key idea of the proof is to apply the relation between the Raviart--Thomas finite element space and a discontinuous space. While inf-sup stable schemes of the discontinuous Galerkin method on shape-regular mesh partitions have been widely discussed, our results show that the Stokes element satisfies the inf-sup condition on anisotropic meshes. Furthermore, we also provide an error estimate in an energy norm on anisotropic meshes. In numerical experiments, we compare calculation results for standard and anisotropic mesh partitions.
We study the relationship between certain Groebner bases for zero dimensional ideals, and the interpolation condition functionals of ideal interpolation. Ideal interpolation is defined by a linear idempotent projector whose kernel is a polynomial ideal. In this paper, we propose the notion of "reverse" complete reduced basis. Based on the notion, we present a fast algorithm to compute the reduced Groebner basis for the kernel of ideal projector under an arbitrary compatible ordering. As an application, we show that knowing the affine variety makes available information concerning the reduced Groebner basis.
We propose a Lawson-time-splitting extended Fourier pseudospectral (LTSeFP) method for the numerical integration of the Gross-Pitaevskii equation with time-dependent potential that is of low regularity in space. For the spatial discretization of low regularity potential, we use an extended Fourier pseudospectral (eFP) method, i.e., we compute the discrete Fourier transform of the low regularity potential in an extended window. For the temporal discretization, to efficiently implement the eFP method for time-dependent low regularity potential, we combine the standard time-splitting method with a Lawson-type exponential integrator to integrate potential and nonlinearity differently. The LTSeFP method is both accurate and efficient: it achieves first-order convergence in time and optimal-order convergence in space in $L^2$-norm under low regularity potential, while the computational cost is comparable to the standard time-splitting Fourier pseudospectral method. Theoretically, we also prove such convergence orders for a large class of spatially low regularity time-dependent potential. Extensive numerical results are reported to confirm the error estimates and to demonstrate the superiority of our method.
This paper addresses a factorization method for imaging the support of a wave-number-dependent source function from multi-frequency data measured at a finite pair of symmetric receivers in opposite directions. The source function is given by the inverse Fourier transform of a compactly supported time-dependent source whose initial moment or terminal moment for radiating is unknown. Using the multi-frequency far-field data at two opposite observation directions, we provide a computational criterion for characterizing the smallest strip containing the support and perpendicular to the directions. A new parameter is incorporated into the design of test functions for indicating the unknown moment. The data from a finite pair of opposite directions can be used to recover the $\Theta$-convex polygon of the support. Uniqueness in recovering the convex hull of the support is obtained as a by-product of our analysis using all observation directions. Similar results are also discussed with the multi-frequency near-field data from a finite pair of observation positions in three dimensions. We further comment on possible extensions to source functions with two disconnected supports. Extensive numerical tests in both two and three dimensions are implemented to show effectiveness and feasibility of the approach. The theoretical framework explored here should be seen as the frequency-domain analysis for inverse source problems in the time domain.
The hazard function represents one of the main quantities of interest in the analysis of survival data. We propose a general approach for parametrically modelling the dynamics of the hazard function using systems of autonomous ordinary differential equations (ODEs). This modelling approach can be used to provide qualitative and quantitative analyses of the evolution of the hazard function over time. Our proposal capitalises on the extensive literature of ODEs which, in particular, allow for establishing basic rules or laws on the dynamics of the hazard function via the use of autonomous ODEs. We show how to implement the proposed modelling framework in cases where there is an analytic solution to the system of ODEs or where an ODE solver is required to obtain a numerical solution. We focus on the use of a Bayesian modelling approach, but the proposed methodology can also be coupled with maximum likelihood estimation. A simulation study is presented to illustrate the performance of these models and the interplay of sample size and censoring. Two case studies using real data are presented to illustrate the use of the proposed approach and to highlight the interpretability of the corresponding models. We conclude with a discussion on potential extensions of our work and strategies to include covariates into our framework.
We consider nonlinear solvers for the incompressible, steady (or at a fixed time step for unsteady) Navier-Stokes equations in the setting where partial measurement data of the solution is available. The measurement data is incorporated/assimilated into the solution through a nudging term addition to the the Picard iteration that penalized the difference between the coarse mesh interpolants of the true solution and solver solution, analogous to how continuous data assimilation (CDA) is implemented for time dependent PDEs. This was considered in the paper [Li et al. {\it CMAME} 2023], and we extend the methodology by improving the analysis to be in the $L^2$ norm instead of a weighted $H^1$ norm where the weight depended on the coarse mesh width, and to the case of noisy measurement data. For noisy measurement data, we prove that the CDA-Picard method is stable and convergent, up to the size of the noise. Numerical tests illustrate the results, and show that a very good strategy when using noisy data is to use CDA-Picard to generate an initial guess for the classical Newton iteration.
Covariance matrices of random vectors contain information that is crucial for modelling. Certain structures and patterns of the covariances (or correlations) may be used to justify parametric models, e.g., autoregressive models. Until now, there have been only few approaches for testing such covariance structures systematically and in a unified way. In the present paper, we propose such a unified testing procedure, and we will exemplify the approach with a large variety of covariance structure models. This includes common structures such as diagonal matrices, Toeplitz matrices, and compound symmetry but also the more involved autoregressive matrices. We propose hypothesis tests for these structures, and we use bootstrap techniques for better small-sample approximation. The structures of the proposed tests invite for adaptations to other covariance patterns by choosing the hypothesis matrix appropriately. We prove their correctness for large sample sizes. The proposed methods require only weak assumptions. With the help of a simulation study, we assess the small sample properties of the tests. We also analyze a real data set to illustrate the application of the procedure.
Characterizing the solution sets in a problem by closedness under operations is recognized as one of the key aspects of algorithm development, especially in constraint satisfaction. An example from the Boolean satisfiability problem is that the solution set of a Horn conjunctive normal form (CNF) is closed under the minimum operation, and this property implies that minimizing a nonnegative linear function over a Horn CNF can be done in polynomial time. In this paper, we focus on the set of integer points (vectors) in a polyhedron, and study the relation between these sets and closedness under operations from the viewpoint of 2-decomposability. By adding further conditions to the 2-decomposable polyhedra, we show that important classes of sets of integer vectors in polyhedra are characterized by 2-decomposability and closedness under certain operations, and in some classes, by closedness under operations alone. The most prominent result we show is that the set of integer vectors in a unit-two-variable-per-inequality polyhedron can be characterized by closedness under the median and directed discrete midpoint operations, each of these operations was independently considered in constraint satisfaction and discrete convex analysis.
注冊地址： 北京市海淀區羊坊店路18號2幢3層301-191