One of the well-known challenges in optimal experimental design is how to efficiently estimate the nested integrations of the expected information gain. The Gaussian approximation and associated importance sampling have been shown to be effective at reducing the numerical costs. However, they may fail due to the non-negligible biases and the numerical instabilities. A new approach is developed to compute the expected information gain, when the posterior distribution is multimodal - a situation previously ignored by the methods aiming at accelerating the nested numerical integrations. Specifically, the posterior distribution is approximated using a mixture distribution constructed by multiple runs of global search for the modes and weighted local Laplace approximations. Under any given probability of capturing all the modes, we provide an estimation of the number of runs of searches, which is dimension independent. It is shown that the novel global-local multimodal approach can be significantly more accurate and more efficient than the other existing approaches, especially when the number of modes is large. The methods can be applied to the designs of experiments with both calibrated and uncalibrated observation noises.
相關內容
Accelerated degradation tests are used to provide accurate estimation of lifetime properties of highly reliable products within a relatively short testing time. There data from particular tests at high levels of stress (e.\,g.\ temperature, voltage, or vibration) are extrapolated, through a physically meaningful model, to obtain estimates of lifetime quantiles under normal use conditions. In this work, we consider repeated measures accelerated degradation tests with multiple stress variables, where the degradation paths are assumed to follow a linear mixed effects model which is quite common in settings when repeated measures are made. We derive optimal experimental designs for minimizing the asymptotic variance for estimating the median failure time under normal use conditions when the time points for measurements are either fixed in advance or are also to be optimized.
Analytical conditions are available for the optimum design of impact absorbers for the case where the host structure is well described as rigid body. Accordingly, the analysis relies on the assumption that the impacts cause immediate dissipation in the contact region, which is modeled in terms of a known coefficient of restitution. When a flexible host structure is considered instead, the impact absorber not only dissipates energy at the time instances of impact, but it inflicts nonlinear energy scattering between structural modes. Hence, it is crucial to account for such nonlinear energy transfers yielding energy redistribution within the modal space of the structure. In the present work, we develop a design approach for reonantly-driven, flexible host structures. We demonstrate decoupling of the time scales of the impact and the resonant vibration. On the long time scale, the dynamics can be properly reduced to the fundamental harmonic of the resonant mode. A light impact absorber responds to this enforced motion, and we recover the Slow Invariant Manifold of the dynamics for the regime of two impacts per period. On the short time scale, the contact mechanics and elasto-dynamics must be finely resolved. We show that it is sufficient to run a numerical simulation of a single impact event with adequate pre-impact velocity. From this simulation, we derive a modal coefficient of restitution and the properties of the contact force pulse, needed to approximate the behavior on the long time scale. We establish that the design problem can be reduced to four dimensionless parameters and demonstrate the approach for the numerical example of a cantilevered beam with an impact absorber. We conclude that the proposed semi-analytical procedure enables deep qualitative understanding of the problem and, at the same time, yields a quantitatively accurate prediction of the optimum design.
R\'enyi's information provides a theoretical foundation for tractable and data-efficient non-parametric density estimation, based on pair-wise evaluations in a reproducing kernel Hilbert space (RKHS). This paper extends this framework to parametric probabilistic modeling, motivated by the fact that R\'enyi's information can be estimated in closed-form for Gaussian mixtures. Based on this special connection, a novel generative model framework called the structured generative model (SGM) is proposed that makes straightforward optimization possible, because costs are scale-invariant, avoiding high gradient variance while imposing less restrictions on absolute continuity, which is a huge advantage in parametric information theoretic optimization. The implementation employs a single neural network driven by an orthonormal input appended to a single white noise source adapted to learn an infinite Gaussian mixture model (IMoG), which provides an empirically tractable model distribution in low dimensions. To train SGM, we provide three novel variational cost functions, based on R\'enyi's second-order entropy and divergence, to implement minimization of cross-entropy, minimization of variational representations of $f$-divergence, and maximization of the evidence lower bound (conditional probability). We test the framework for estimation of mutual information and compare the results with the mutual information neural estimation (MINE), for density estimation, for conditional probability estimation in Markov models as well as for training adversarial networks. Our preliminary results show that SGM significantly improves MINE estimation in terms of data efficiency and variance, conventional and variational Gaussian mixture models, as well as the performance of generative adversarial networks.
PAC-Bayesian bounds are known to be tight and informative when studying the generalization ability of randomized classifiers. However, when applied to some family of deterministic models such as neural networks, they require a loose and costly derandomization step. As an alternative to this step, we introduce new PAC-Bayesian generalization bounds that have the originality to provide disintegrated bounds, i.e., they give guarantees over one single hypothesis instead of the usual averaged analysis. Our bounds are easily optimizable and can be used to design learning algorithms. We illustrate the interest of our result on neural networks and show a significant practical improvement over the state-of-the-art framework.
We study the conformal capacity by using novel computational algorithms based on implementations of the fast multipole method, and analytic techniques. Especially, we apply domain functionals to study the capacities of condensers $(G,E)$ where $G$ is a simply connected domain in the complex plane and $E$ is a compact subset of $G$. Due to conformal invariance, our main tools are the hyperbolic geometry and functionals such as the hyperbolic perimeter of $E$. Our computational experiments demonstrate, for instance, sharpness of established inequalities. In the case of model problems with known analytic solutions, very high precision of computation is observed.
We consider a perimeter defense problem in a planar conical environment in which a single vehicle, having a finite capture radius, aims to defend a concentric perimeter from mobile intruders. The intruders are arbitrarily released at the circumference of the environment and move radially toward the perimeter with fixed speed. We present a competitive analysis approach to this problem by measuring the performance of multiple online algorithms for the vehicle against arbitrary inputs, relative to an optimal offline algorithm that has access to all future inputs. In particular, we first establish a necessary condition on the parameter space to guarantee finite competitiveness of any algorithm, and then characterize a parameter regime in which the competitive ratio is guaranteed to be at least 2 for any algorithm. We then design and analyze three online algorithms and characterize parameter regimes for which they have finite competitive ratios. Specifically, our first two algorithms are provably 1, and 2-competitive, respectively, whereas our third algorithm exhibits a finite competitive ratio that depends on the problem parameters. Finally, we provide numerous parameter space plots providing insights into the relative performance of our algorithms.
An increasing body of research focuses on using neural networks to model time series. A common assumption in training neural networks via maximum likelihood estimation on time series is that the errors across time steps are uncorrelated. However, errors are actually autocorrelated in many cases due to the temporality of the data, which makes such maximum likelihood estimations inaccurate. In this paper, in order to adjust for autocorrelated errors, we propose to learn the autocorrelation coefficient jointly with the model parameters. In our experiments, we verify the effectiveness of our approach on time series forecasting. Results across a wide range of real-world datasets with various state-of-the-art models show that our method enhances performance in almost all cases. Based on these results, we suggest empirical critical values to determine the severity of autocorrelated errors. We also analyze several aspects of our method to demonstrate its advantages. Finally, other time series tasks are also considered to validate that our method is not restricted to only forecasting.
The Covid-19 pandemic presents a serious threat to people's health, resulting in over 250 million confirmed cases and over 5 million deaths globally. In order to reduce the burden on national health care systems and to mitigate the effects of the outbreak, accurate modelling and forecasting methods for short- and long-term health demand are needed to inform government interventions aiming at curbing the pandemic. Current research on Covid-19 is typically based on a single source of information, specifically on structured historical pandemic data. Other studies are exclusively focused on unstructured online retrieved insights, such as data available from social media. However, the combined use of structured and unstructured information is still uncharted. This paper aims at filling this gap, by leveraging historical as well as social media information with a novel data integration methodology. The proposed approach is based on vine copulas, which allow us to improve predictions by exploiting the dependencies between different sources of information. We apply the methodology to combine structured datasets retrieved from official sources and to a big unstructured dataset of information collected from social media. The results show that the proposed approach, compared to traditional approaches, yields more accurate estimations and predictions of the evolution of the Covid-19 pandemic.
Multimodal sentiment analysis is a very actively growing field of research. A promising area of opportunity in this field is to improve the multimodal fusion mechanism. We present a novel feature fusion strategy that proceeds in a hierarchical fashion, first fusing the modalities two in two and only then fusing all three modalities. On multimodal sentiment analysis of individual utterances, our strategy outperforms conventional concatenation of features by 1%, which amounts to 5% reduction in error rate. On utterance-level multimodal sentiment analysis of multi-utterance video clips, for which current state-of-the-art techniques incorporate contextual information from other utterances of the same clip, our hierarchical fusion gives up to 2.4% (almost 10% error rate reduction) over currently used concatenation. The implementation of our method is publicly available in the form of open-source code.
Recently, generative adversarial networks (GANs) have shown promising performance in generating realistic images. However, they often struggle in learning complex underlying modalities in a given dataset, resulting in poor-quality generated images. To mitigate this problem, we present a novel approach called mixture of experts GAN (MEGAN), an ensemble approach of multiple generator networks. Each generator network in MEGAN specializes in generating images with a particular subset of modalities, e.g., an image class. Instead of incorporating a separate step of handcrafted clustering of multiple modalities, our proposed model is trained through an end-to-end learning of multiple generators via gating networks, which is responsible for choosing the appropriate generator network for a given condition. We adopt the categorical reparameterization trick for a categorical decision to be made in selecting a generator while maintaining the flow of the gradients. We demonstrate that individual generators learn different and salient subparts of the data and achieve a multiscale structural similarity (MS-SSIM) score of 0.2470 for CelebA and a competitive unsupervised inception score of 8.33 in CIFAR-10.
注冊地址： 北京市海淀區羊坊店路18號2幢3層301-191