We study the estimation problem for linear time-invariant (LTI) state-space models with Gaussian excitation of an unknown covariance. We provide non asymptotic lower bounds for the expected estimation error and the mean square estimation risk of the least square estimator, and the minimax mean square estimation risk. These bounds are sharp with explicit constants when the matrix of the dynamics has no eigenvalues on the unit circle and are rate-optimal when they do. Our results extend and improve existing lower bounds to lower bounds in expectation of the mean square estimation risk and to systems with a general noise covariance. Instrumental to our derivation are new concentration results for rescaled sample covariances and deviation results for the corresponding multiplication processes of the covariates, a differential geometric construction of a prior on the unit operator ball of small Fisher information, and an extension of the Cram\'er-Rao and van Treesinequalities to matrix-valued estimators.
We give the first polynomial-time, polynomial-sample, differentially private estimator for the mean and covariance of an arbitrary Gaussian distribution $\mathcal{N}(\mu,\Sigma)$ in $\mathbb{R}^d$. All previous estimators are either nonconstructive, with unbounded running time, or require the user to specify a priori bounds on the parameters $\mu$ and $\Sigma$. The primary new technical tool in our algorithm is a new differentially private preconditioner that takes samples from an arbitrary Gaussian $\mathcal{N}(0,\Sigma)$ and returns a matrix $A$ such that $A \Sigma A^T$ has constant condition number.
We derive information-theoretic lower bounds on the Bayes risk and generalization error of realizable machine learning models. In particular, we employ an analysis in which the rate-distortion function of the model parameters bounds the required mutual information between the training samples and the model parameters in order to learn a model up to a Bayes risk constraint. For realizable models, we show that both the rate distortion function and mutual information admit expressions that are convenient for analysis. For models that are (roughly) lower Lipschitz in their parameters, we bound the rate distortion function from below, whereas for VC classes, the mutual information is bounded above by $d_\mathrm{vc}\log(n)$. When these conditions match, the Bayes risk with respect to the zero-one loss scales no faster than $\Omega(d_\mathrm{vc}/n)$, which matches known outer bounds and minimax lower bounds up to logarithmic factors. We also consider the impact of label noise, providing lower bounds when training and/or test samples are corrupted.
Quantum state tomography (QST) for reconstructing pure states requires exponentially increasing resources and measurements with the number of qubits by using state-of-the-art quantum compressive sensing (CS) methods. In this article, QST reconstruction for any pure state composed of the superposition of $K$ different computational basis states of $n$ qubits in a specific measurement set-up, i.e., denoted as $K$-sparse, is achieved without any initial knowledge and with quantum polynomial-time complexity of resources based on the assumption of the existence of polynomial size quantum circuits for implementing exponentially large powers of a specially designed unitary operator. The algorithm includes $\mathcal{O}(2 \, / \, \vert c_{k}\vert^2)$ repetitions of conventional phase estimation algorithm depending on the probability $\vert c_{k}\vert^2$ of the least possible basis state in the superposition and $\mathcal{O}(d \, K \,(log K)^c)$ measurement settings with conventional quantum CS algorithms independent from the number of qubits while dependent on $K$ for constant $c$ and $d$. Quantum phase estimation algorithm is exploited based on the favorable eigenstructure of the designed operator to represent any pure state as a superposition of eigenvectors. Linear optical set-up is presented for realizing the special unitary operator which includes beam splitters and phase shifters where propagation paths of single photon are tracked with which-path-detectors. Quantum circuit implementation is provided by using only CNOT, phase shifter and $- \pi \, / \, 2$ rotation gates around X-axis in Bloch sphere, i.e., $R_{X}(- \pi \, / \, 2)$, allowing to be realized in NISQ devices. Open problems are discussed regarding the existence of the unitary operator and its practical circuit implementation.
Diagnostic classification models (DCMs) offer statistical tools to inspect the fined-grained attribute of respondents' strengths and weaknesses. However, the diagnosis accuracy deteriorates when misspecification occurs in the predefined item-attribute relationship, which is encoded into a Q-matrix. To prevent such misspecification, methodologists have recently developed several Bayesian Q-matrix estimation methods for greater estimation flexibility. However, these methods become infeasible in the case of large-scale assessments with a large number of attributes and items. In this study, we focused on the deterministic inputs, noisy ``and'' gate (DINA) model and proposed a new framework for the Q-matrix estimation to find the Q-matrix with the maximum marginal likelihood. Based on this framework, we developed a scalable estimation algorithm for the DINA Q-matrix by constructing an iteration algorithm that utilizes stochastic optimization and variational inference. The simulation and empirical studies reveal that the proposed method achieves high-speed computation, good accuracy, and robustness to potential misspecifications, such as initial value's choices and hyperparameter settings. Thus, the proposed method can be a useful tool for estimating a Q-matrix in large-scale settings.
Gaussian covariance graph model is a popular model in revealing underlying dependency structures among random variables. A Bayesian approach to the estimation of covariance structures uses priors that force zeros on some off-diagonal entries of covariance matrices and put a positive definite constraint on matrices. In this paper, we consider a spike and slab prior on off-diagonal entries, which uses a mixture of point-mass and normal distribution. The point-mass naturally introduces sparsity to covariance structures so that the resulting posterior from this prior renders covariance structure learning. Under this prior, we calculate posterior model probabilities of covariance structures using Laplace approximation. We show that the error due to Laplace approximation becomes asymptotically marginal at some rate depending on the posterior convergence rate of covariance matrix under the Frobenius norm. With the approximated posterior model probabilities, we propose a new framework for estimating a covariance structure. Since the Laplace approximation is done around the mode of conditional posterior of covariance matrix, which cannot be obtained in the closed form, we propose a block coordinate descent algorithm to find the mode and show that the covariance matrix can be estimated using this algorithm once the structure is chosen. Through a simulation study based on five numerical models, we show that the proposed method outperforms graphical lasso and sample covariance matrix in terms of root mean squared error, max norm, spectral norm, specificity, and sensitivity. Also, the advantage of the proposed method is demonstrated in terms of accuracy compared to our competitors when it is applied to linear discriminant analysis (LDA) classification to breast cancer diagnostic dataset.
We consider a potential outcomes model in which interference may be present between any two units but the extent of interference diminishes with spatial distance. The causal estimand is the global average treatment effect, which compares counterfactual outcomes when all units are treated to outcomes when none are. We study a class of designs in which space is partitioned into clusters that are randomized into treatment and control. For each design, we estimate the treatment effect using a Horovitz-Thompson estimator that compares the average outcomes of units with all neighbors treated to units with no neighbors treated, where the neighborhood radius is of the same order as the cluster size dictated by the design. We derive the estimator's rate of convergence as a function of the design and degree of interference and use this to obtain estimator-design pairs in this class that achieve near-optimal rates of convergence under relatively minimal assumptions on interference. We prove that the estimators are asymptotically normal and provide a variance estimator. Finally, we discuss practical implementation of the designs by partitioning space using clustering algorithms.
Random fields are mathematical structures used to model the spatial interaction of random variables along time, with applications ranging from statistical physics and thermodynamics to system's biology and the simulation of complex systems. Despite being studied since the 19th century, little is known about how the dynamics of random fields are related to the geometric properties of their parametric spaces. For example, how can we quantify the similarity between two random fields operating in different regimes using an intrinsic measure? In this paper, we propose a numerical method for the computation of geodesic distances in Gaussian random field manifolds. First, we derive the metric tensor of the underlying parametric space (the 3 x 3 first-order Fisher information matrix), then we derive the 27 Christoffel symbols required in the definition of the system of non-linear differential equations whose solution is a geodesic curve starting at the initial conditions. The fourth-order Runge-Kutta method is applied to numerically solve the non-linear system through an iterative approach. The obtained results show that the proposed method can estimate the geodesic distances for several different initial conditions. Besides, the results reveal an interesting pattern: in several cases, the geodesic curve obtained by reversing the system of differential equations in time does not match the original curve, suggesting the existence of irreversible geometric deformations in the trajectory of a moving reference traveling along a geodesic curve.
The Arnold-Beltrami-Childress (ABC) flow and the Kolmogorov flow are three dimensional periodic divergence free velocity fields that exhibit chaotic streamlines. We are interested in front speed enhancement in G-equation of turbulent combustion by large intensity ABC and Kolmogorov flows. We give a quantitative construction of the ballistic orbits of ABC and Kolmogorov flows, namely those with maximal large time asymptotic speeds in a coordinate direction. Thanks to the optimal control theory of G-equation (a convex but non-coercive Hamilton-Jacobi equation), the ballistic orbits serve as admissible trajectories for front speed estimates. To study the tightness of the estimates, we compute the front speeds of G-equation based on a semi-Lagrangian (SL) scheme with Strang splitting and weighted essentially non-oscillatory (WENO) interpolation. Time step size is chosen so that the Courant number grows sublinearly with the flow intensity. Numerical results show that the front speed growth rate in terms of the flow intensity may approach the analytical bounds from the ballistic orbits.
In classical statistics, a statistical experiment consisting of $n$ i.i.d observations from d-dimensional multinomial distributions can be well approximated by a $d-1$ dimensional Gaussian distribution. In a quantum version of the result it has been shown that a collection of $n$ qudits of full rank can be well approximated by a quantum system containing a classical part, which is a $d-1$ dimensional Gaussian distribution, and a quantum part containing an ensemble of $d(d-1)/2$ shifted thermal states. In this paper, we obtain a generalization of this result when the qudits are not of full rank. We show that when the rank of the qudits is $r$, then the limiting experiment consists of an $r-1$ dimensional Gaussian distribution and an ensemble of both shifted pure and shifted thermal states. We also outline a two-stage procedure for the estimation of the low-rank qudit, where we obtain an estimator which is sharp minimax optimal. For the estimation of a linear functional of the quantum state, we construct an estimator, analyze the risk and use quantum LAN to show that our estimator is also optimal in the minimax sense.
We study the least square estimator, in the framework of simple linear regression, when the deviance term $\varepsilon$ with respect to the linear model is modeled by a uniform distribution. In particular, we give the law of this estimator, and prove some convergence properties.