In matrix-valued datasets the sampled matrices often exhibit correlations among both their rows and their columns. A useful and parsimonious model of such dependence is the matrix normal model, in which the covariances among the elements of a random matrix are parameterized in terms of the Kronecker product of two covariance matrices, one representing row covariances and one representing column covariance. An appealing feature of such a matrix normal model is that the Kronecker covariance structure allows for standard likelihood inference even when only a very small number of data matrices is available. For instance, in some cases a likelihood ratio test of dependence may be performed with a sample size of one. However, more generally the sample size required to ensure boundedness of the matrix normal likelihood or the existence of a unique maximizer depends in a complicated way on the matrix dimensions. This motivates the study of how large a sample size is needed to ensure that maximum likelihood estimators exist, and exist uniquely with probability one. Our main result gives precise sample size thresholds in the paradigm where the number of rows and the number of columns of the data matrices differ by at most a factor of two. Our proof uses invariance properties that allow us to consider data matrices in canonical form, as obtained from the Kronecker canonical form for matrix pencils.
相關內容
Inference is the task of drawing conclusions about unobserved variables given observations of related variables. Applications range from identifying diseases from symptoms to classifying economic regimes from price movements. Unfortunately, performing exact inference is intractable in general. One alternative is variational inference, where a candidate probability distribution is optimized to approximate the posterior distribution over unobserved variables. For good approximations a flexible and highly expressive candidate distribution is desirable. In this work, we propose quantum Born machines as variational distributions over discrete variables. We apply the framework of operator variational inference to achieve this goal. In particular, we adopt two specific realizations: one with an adversarial objective and one based on the kernelized Stein discrepancy. We demonstrate the approach numerically using examples of Bayesian networks, and implement an experiment on an IBM quantum computer. Our techniques enable efficient variational inference with distributions beyond those that are efficiently representable on a classical computer.
Exposure mappings facilitate investigations of complex causal effects when units interact in experiments. Current methods assume that the exposures are correctly specified, but such an assumption cannot be verified, and its validity is often questionable. This paper describes conditions under which one can draw inferences about exposure effects when the exposures are misspecified. The main result is a proof of consistency under mild conditions on the errors introduced by the misspecification. The rate of convergence is determined by the dependence between units' specification errors, and consistency is achieved even if the errors are large as long as they are sufficiently weakly dependent. In other words, exposure effects can be precisely estimated also under misspecification as long as the units' exposures are not misspecified in the same way. The limiting distribution of the estimator is discussed. Asymptotic normality is achieved under stronger conditions than those needed for consistency. Similar conditions also facilitate conservative variance estimation.
We consider high-dimensional multivariate linear regression models, where the joint distribution of covariates and response variables is a multivariate normal distribution with a bandable covariance matrix. The main goal of this paper is to estimate the regression coefficient matrix, which is a function of the bandable covariance matrix. Although the tapering estimator of covariance has the minimax optimal convergence rate for the class of bandable covariances, we show that it has a sub-optimal convergence rate for the regression coefficient; that is, a minimax estimator for the class of bandable covariances may not be a minimax estimator for its functionals. We propose the blockwise tapering estimator of the regression coefficient, which has the minimax optimal convergence rate for the regression coefficient under the bandable covariance assumption. We also propose a Bayesian procedure called the blockwise tapering post-processed posterior of the regression coefficient and show that the proposed Bayesian procedure has the minimax optimal convergence rate for the regression coefficient under the bandable covariance assumption. We show that the proposed methods outperform the existing methods via numerical studies.
A generalized method of moments (GMM) estimator is unreliable for a large number of moment conditions, that is, it is comparable, or larger than the sample size. While classical GMM literature proposes several provisions to this problem, its Bayesian counterpart (i.e., Bayesian inference using a GMM criterion as a quasi-likelihood) almost totally ignores it. This study bridges this gap by proposing an adaptive Markov Chain Monte Carlo (MCMC) approach to a GMM inference with many moment conditions. Particularly, this study focuses on the adaptive tuning of a weighting matrix on the fly. Our proposal consists of two elements. The first is the use of the nonparametric eigenvalue-regularized precision matrix estimator, which contributes to numerical stability. The second is the random update of a weighting matrix, which substantially reduces computational cost, while maintaining the accuracy of the estimation. We then present a simulation study and real data application to compare the performance of the proposed approach with existing approaches.
The rigidity of a matrix $A$ for target rank $r$ is the minimum number of entries of $A$ that need to be changed in order to obtain a matrix of rank at most $r$. Matrix rigidity was introduced by Valiant in 1977 as a tool to prove circuit lower bounds for linear functions and since then this notion has also found applications in other areas of complexity theory. Recently (arXiv 2021), Alman proved that for any field $\mathbb{F}$, $d\geq 2$ and arbitrary matrices $M_1, \ldots, M_n \in \mathbb{F}^{d\times d}$, one can get a $d^n\times d^n$ matrix of rank $\le d^{(1-\gamma)n}$ over $\mathbb{F}$ by changing only $d^{(1+\varepsilon) n}$ entries of the Kronecker product $M = M_1\otimes M_2\otimes \ldots\otimes M_n$, where $1/\gamma$ is roughly $2^d/\varepsilon^2$. In this note we improve this result in two directions. First, we do not require the matrices $M_i$ to have equal size. Second, we reduce $1/\gamma$ from exponential in $d$ to roughly $d^{3/2}/\varepsilon^2$ (where $d$ is the maximum size of the matrices), and to nearly linear (roughly $d/\varepsilon^2$) for matrices $M_i$ of sizes within a constant factor of each other. For the case of matrices of equal size, our bound matches the bound given by Dvir and Liu (\textit{Theory of Computing, 2020}) for the rigidity of generalized Walsh--Hadamard matrices (Kronecker powers of DFT matrices), and improves their bounds for DFT matrices of abelian groups that are direct products of small groups.
A differential geometric framework to construct an asymptotically unbiased estimator of a function of a parameter is presented. The derived estimator asymptotically coincides with the uniformly minimum variance unbiased estimator, if a complete sufficient statistic exists. The framework is based on the maximum a posteriori estimation, where the prior is chosen such that the estimator is unbiased. The framework is demonstrated for the second-order asymptotic unbiasedness (unbiased up to $O(n^{-1})$ for a sample of size $n$). The condition of the asymptotic unbiasedness leads the choice of the prior such that the departure from a kind of harmonicity of the estimand is canceled out at each point of the model manifold. For a given estimand, the prior is given as an integral. On the other hand, for a given prior, we can address the bias of what estimator can be reduced by solving an elliptic partial differential equation. A family of invariant priors, which generalizes the Jeffreys prior, is mentioned as a specific example. Some illustrative examples of applications of the proposed framework are provided.
The conditional value-at-risk (CVaR) is a useful risk measure in fields such as machine learning, finance, insurance, energy, etc. When measuring very extreme risk, the commonly used CVaR estimation method of sample averaging does not work well due to limited data above the value-at-risk (VaR), the quantile corresponding to the CVaR level. To mitigate this problem, the CVaR can be estimated by extrapolating above a lower threshold than the VaR using a generalized Pareto distribution (GPD), which is often referred to as the peaks-over-threshold (POT) approach. This method often requires a very high threshold to fit well, leading to high variance in estimation, and can induce significant bias if the threshold is chosen too low. In this paper, we derive a new expression for the GPD approximation error of the CVaR, a bias term induced by the choice of threshold, as well as a bias correction method for the estimated GPD parameters. This leads to the derivation of a new estimator for the CVaR that we prove to be asymptotically unbiased. In a practical setting, we show through experiments that our estimator provides a significant performance improvement compared with competing CVaR estimators in finite samples. As a consequence of our bias correction method, it is also shown that a much lower threshold can be selected without introducing significant bias. This allows a larger portion of data to be be used in CVaR estimation compared with the typical POT approach, leading to more stable estimates. As secondary results, a new estimator for a second-order parameter of heavy-tailed distributions is derived, as well as a confidence interval for the CVaR which enables quantifying the level of variability in our estimator.
By using permutation representations of maps, one obtains a bijection between all maps whose underlying graph is isomorphic to a graph $G$ and products of permutations of given cycle types. By using statistics on cycle distributions in products of permutations, one can derive information on the set of all $2$-cell embeddings of $G$. In this paper, we study multistars -- loopless multigraphs in which there is a vertex incident with all the edges. The well known genus distribution of the two-vertex multistar, also known as a dipole, can be used to determine the expected genus of the dipole. We then use a result of Stanley to show that, in general, the expected genus of every multistar with $n$ nonleaf edges lies in an interval of length $2/(n+1)$ centered at the expected genus of an $n$-edge dipole. As an application, we show that the face distribution of the multistar is the same as the face distribution gained when adding a new vertex to a $2$-cell embedded graph, and use this to obtain a general upper bound for the expected number of faces in random embeddings of graphs.
In this work, we compare three different modeling approaches for the scores of soccer matches with regard to their predictive performances based on all matches from the four previous FIFA World Cups 2002 - 2014: Poisson regression models, random forests and ranking methods. While the former two are based on the teams' covariate information, the latter method estimates adequate ability parameters that reflect the current strength of the teams best. Within this comparison the best-performing prediction methods on the training data turn out to be the ranking methods and the random forests. However, we show that by combining the random forest with the team ability parameters from the ranking methods as an additional covariate we can improve the predictive power substantially. Finally, this combination of methods is chosen as the final model and based on its estimates, the FIFA World Cup 2018 is simulated repeatedly and winning probabilities are obtained for all teams. The model slightly favors Spain before the defending champion Germany. Additionally, we provide survival probabilities for all teams and at all tournament stages as well as the most probable tournament outcome.
We develop an approach to risk minimization and stochastic optimization that provides a convex surrogate for variance, allowing near-optimal and computationally efficient trading between approximation and estimation error. Our approach builds off of techniques for distributionally robust optimization and Owen's empirical likelihood, and we provide a number of finite-sample and asymptotic results characterizing the theoretical performance of the estimator. In particular, we show that our procedure comes with certificates of optimality, achieving (in some scenarios) faster rates of convergence than empirical risk minimization by virtue of automatically balancing bias and variance. We give corroborating empirical evidence showing that in practice, the estimator indeed trades between variance and absolute performance on a training sample, improving out-of-sample (test) performance over standard empirical risk minimization for a number of classification problems.
小貼士
登錄享
相關主題
注冊地址： 北京市海淀區羊坊店路18號2幢3層301-191