We propose a regularization for Reduced Order Models (ROMs) of the quasi-geostrophic equations (QGE) to increase accuracy when the Proper Orthogonal Decomposition (POD) modes retained to construct the reduced basis are insufficient to describe the system dynamics. Our regularization is based on the so-called BV-alpha model, which modifies the nonlinear term in the QGE and adds a linear differential filter for the vorticity. To show the effectiveness of the BV-alpha model for ROM closure, we compare the results computed by a POD-Galerkin ROM with and without regularization for the classical double-gyre wind forcing benchmark. Our numerical results show that the solution computed by the regularized ROM is more accurate, even when the retained POD modes account for a small percentage of the eigenvalue energy. Additionally, we show that, although computationally more expensive that the ROM with no regularization, the regularized ROM is still a competitive alternative to full order simulations of the QGE.
Often, polynomials or rational functions, orthogonal for a particular inner product are desired. In practical numerical algorithms these polynomials are not constructed, but instead the associated recurrence relations are computed. Moreover, also typically the inner product is changed to a discrete inner product, which is the finite sum of weighted functions evaluated in specific nodes. For particular applications it is beneficial to have an efficient procedure to update the recurrence relations when adding or removing nodes from the inner product. The construction of the recurrence relations is equivalent to computing a structured matrix (polynomial) or pencil (rational) having prescribed spectral properties. Hence the solution of this problem is often referred to as solving an Inverse Eigenvalue Problem. In Van Buggenhout et al. (2022) we proposed updating techniques to add nodes to the inner product while efficiently updating the recurrences. To complete this study we present in this article manners to efficiently downdate the recurrences when removing nodes from the inner product. The link between removing nodes and the QR algorithm to deflate eigenvalues is exploited to develop efficient algorithms. We will base ourselves on the perfect shift strategy and develop algorithms, both for the polynomial case and the rational function setting. Numerical experiments validate our approach.
In recent years, several swarm intelligence optimization algorithms have been proposed to be applied for solving a variety of optimization problems. However, the values of several hyperparameters should be determined. For instance, although Particle Swarm Optimization (PSO) has been applied for several applications with higher optimization performance, the weights of inertial velocity, the particle's best known position and the swarm's best known position should be determined. Therefore, this study proposes an analytic framework to analyze the optimized average-fitness-function-value (AFFV) based on mathematical models for a variety of fitness functions. Furthermore, the optimized hyperparameter values could be determined with a lower AFFV for minimum cases. Experimental results show that the hyperparameter values from the proposed method can obtain higher efficiency convergences and lower AFFVs.
We develop a novel computational framework to approximate solution operators of evolution partial differential equations (PDEs). By employing a general nonlinear reduced-order model, such as a deep neural network, to approximate the solution of a given PDE, we realize that the evolution of the model parameter is a control problem in the parameter space. Based on this observation, we propose to approximate the solution operator of the PDE by learning the control vector field in the parameter space. From any initial value, this control field can steer the parameter to generate a trajectory such that the corresponding reduced-order model solves the PDE. This allows for substantially reduced computational cost to solve the evolution PDE with arbitrary initial conditions. We also develop comprehensive error analysis for the proposed method when solving a large class of semilinear parabolic PDEs. Numerical experiments on different high-dimensional evolution PDEs with various initial conditions demonstrate the promising results of the proposed method.
Self-supervised learning (SSL) speech models, which can serve as powerful upstream models to extract meaningful speech representations, have achieved unprecedented success in speech representation learning. However, their effectiveness on non-speech datasets is relatively less explored. In this work, we propose an ensemble framework, with a combination of ensemble techniques, to fuse SSL speech models' embeddings. Extensive experiments on speech and non-speech audio datasets are conducted to investigate the representation abilities of our ensemble method and its single constituent model. Ablation studies are carried out to evaluate the performances of different ensemble techniques, such as feature averaging and concatenation. All experiments are conducted during NeurIPS 2021 HEAR Challenge as a standard evaluation pipeline provided by competition officials. Results demonstrate SSL speech models' strong abilities on various non-speech tasks, while we also note that they fail to deal with fine-grained music tasks, such as pitch classification and note onset detection. In addition, feature ensemble is shown to have great potential on producing more holistic representations, as our proposed framework generally surpasses state-of-the-art SSL speech/audio models and has superior performance on various datasets compared with other teams in HEAR Challenge. Our code is available at //github.com/tony10101105/HEAR-2021-NeurIPS-Challenge -- NTU-GURA.
Pin fins are imperative in the cooling of turbine blades. The designs of pin fins, therefore, have seen significant research in the past. With the developments in metal additive manufacturing, novel design approaches toward complex geometries are now feasible. To that end, this article presents a Bayesian optimization approach for designing inline pins that can achieve low pressure loss. The pin-fin shape is defined using featurized (parametrized) piecewise cubic splines in 2D. The complexity of the shape is dependent on the number of splines used for the analysis. From a method development perspective, the study is performed using three splines. Owing to this piece-wise modeling, a unique pin fin design is defined using five features. After specifying the design, a computational fluid dynamics-based model is developed that computes the pressure drop during the flow. Bayesian optimization is carried out on a Gaussian processes-based surrogate to obtain an optimal combination of pin-fin features to minimize the pressure drop. The results show that the optimization tends to approach an aerodynamic design leading to low pressure drop corroborating with the existing knowledge. Furthermore, multiple iterations of optimizations are conducted with varying degree of input data. The results reveal that a convergence to similar optimal design is achieved with a minimum of just twenty five initial design-of-experiments data points for the surrogate. Sensitivity analysis shows that the distance between the rows of the pin fins is the most dominant feature influencing the pressure drop. In summary, the newly developed automated framework demonstrates remarkable capabilities in designing pin fins with superior performance characteristics.
Numerical discretization of the large-scale Maxwell's equations leads to an ill-conditioned linear system that is challenging to solve. The key requirement for successive solutions of this linear system is to choose an efficient solver. In this work we use Perfectly Matched Layers (PML) to increase this efficiency. PML have been widely used to truncate numerical simulations of wave equations due to improving the accuracy of the solution instead of using absorbing boundary conditions (ABCs). Here, we will develop an efficient solver by providing an alternative use of PML as transmission conditions at the interfaces between subdomains in our domain decomposition method. We solve Maxwell's equations and assess the convergence rate of our solutions compared to the situation where absorbing boundary conditions are chosen as transmission conditions.
In this paper, we present a unified and general framework for analyzing the batch updating approach to nonlinear, high-dimensional optimization. The framework encompasses all the currently used batch updating approaches, and is applicable to nonconvex as well as convex functions. Moreover, the framework permits the use of noise-corrupted gradients, as well as first-order approximations to the gradient (sometimes referred to as "gradient-free" approaches). By viewing the analysis of the iterations as a problem in the convergence of stochastic processes, we are able to establish a very general theorem, which includes most known convergence results for zeroth-order and first-order methods. The analysis of "second-order" or momentum-based methods is not a part of this paper, and will be studied elsewhere. However, numerical experiments indicate that momentum-based methods can fail if the true gradient is replaced by its first-order approximation. This requires further theoretical analysis.
Dynamical models described by ordinary differential equations (ODEs) are a fundamental tool in the sciences and engineering. Exact reduction aims at producing a lower-dimensional model in which each macro-variable can be directly related to the original variables, and it is thus a natural step towards the model's formal analysis and mechanistic understanding. We present an algorithm which, given a polynomial ODE model, computes a longest possible chain of exact linear reductions of the model such that each reduction refines the previous one, thus giving a user control of the level of detail preserved by the reduction. This significantly generalizes over the existing approaches which compute only the reduction of the lowest dimension subject to an approach-specific constraint. The algorithm reduces finding exact linear reductions to a question about representations of finite-dimensional algebras. We provide an implementation of the algorithm, demonstrate its performance on a set of benchmarks, and illustrate the applicability via case studies. Our implementation is freely available at //github.com/x3042/ExactODEReduction.jl
The geometric optimisation of crystal structures is a procedure widely used in Chemistry that changes the geometrical placement of the particles inside a structure. It is called structural relaxation and constitutes a local minimization problem with a non-convex objective function whose domain complexity increases along with the number of particles involved. In this work we study the performance of the two most popular first order optimisation methods, Gradient Descent and Conjugate Gradient, in structural relaxation. The respective pseudocodes can be found in Section 6. Although frequently employed, there is a lack of their study in this context from an algorithmic point of view. In order to accurately define the problem, we provide a thorough derivation of all necessary formulae related to the crystal structure energy function and the function's differentiation. We run each algorithm in combination with a constant step size, which provides a benchmark for the methods' analysis and direct comparison. We also design dynamic step size rules and study how these improve the two algorithms' performance. Our results show that there is a trade-off between convergence rate and the possibility of an experiment to succeed, hence we construct a function to assign utility to each method based on our respective preference. The function is built according to a recently introduced model of preference indication concerning algorithms with deadline and their run time. Finally, building on all our insights from the experimental results, we provide algorithmic recipes that best correspond to each of the presented preferences and select one recipe as the optimal for equally weighted preferences.
Quick simulations for iterative evaluations of multi-design variables and boundary conditions are essential to find the optimal acoustic conditions in building design. We propose to use the reduced basis method (RBM) for realistic room acoustic scenarios where the surfaces have inhomogeneous acoustic properties, which enables quick evaluations of changing absorption materials for different surfaces in room acoustic simulations. The RBM has shown its benefit to speed up room acoustic simulations by three orders of magnitude for uniform boundary conditions. This study investigates the RBM with two main focuses, 1) various source positions in diverse geometries, e.g., square, rectangular, L-shaped, and disproportionate room. 2) Inhomogeneous surface absorption in 2D and 3D by parameterizing numerous acoustic parameters of surfaces, e.g., the thickness of a porous material, cavity depth, switching between a frequency independent (e.g., hard surface) and frequency dependent boundary condition. Results of numerical experiments show speedups of more than two orders of magnitude compared to a high fidelity numerical solver in a 3D case where reverberation time varies within one just noticeable difference in all the frequency octave bands.