Exponential family models, generalized linear models (GLMs), generalized linear mixed models (GLMMs) and generalized additive models (GAMs) are widely used methods in statistics. However, many scientific applications necessitate constraints be placed on model parameters such as shape and linear inequality constraints. Constrained estimation and inference of parameters remains a pervasive problem in statistics where many methods rely on modifying rigid large sample theory assumptions for inference. We propose a flexible slice sampler Gibbs algorithm for Bayesian GLMMs and GAMs with linear inequality and shape constraints. We prove our posterior samples follow a Markov chain central limit theorem (CLT) by proving uniform ergodicity of our Markov chain and existence of the a moment generating function for our posterior distributions. We use our CLT results to derive joint bands and multiplicity adjusted Bayesian inference for nonparametric functional effects. Our rigorous CLT results address a shortcoming in the literature by obtaining valid estimation and inference on constrained parameters in finite sample settings. Our algorithmic and proof techniques are adaptable to a myriad of important statistical modeling problems. We apply our Bayesian GAM to a real data analysis example involving proportional odds regression for concussion recovery in children with shape constraints and smoothed nonparametric effects. We obtain multiplicity adjusted inference on monotonic nonparametric time effect to elucidate recovery trends in children as a function of time.
相關內容
We consider stochastic approximations of sampling algorithms, such as Stochastic Gradient Langevin Dynamics (SGLD) and the Random Batch Method (RBM) for Interacting Particle Dynamcs (IPD). We observe that the noise introduced by the stochastic approximation is nearly Gaussian due to the Central Limit Theorem (CLT) while the driving Brownian motion is exactly Gaussian. We harness this structure to absorb the stochastic approximation error inside the diffusion process, and obtain improved convergence guarantees for these algorithms. For SGLD, we prove the first stable convergence rate in KL divergence without requiring uniform warm start, assuming the target density satisfies a Log-Sobolev Inequality. Our result implies superior first-order oracle complexity compared to prior works, under significantly milder assumptions. We also prove the first guarantees for SGLD under even weaker conditions such as H\"{o}lder smoothness and Poincare Inequality, thus bridging the gap between the state-of-the-art guarantees for LMC and SGLD. Our analysis motivates a new algorithm called covariance correction, which corrects for the additional noise introduced by the stochastic approximation by rescaling the strength of the diffusion. Finally, we apply our techniques to analyze RBM, and significantly improve upon the guarantees in prior works (such as removing exponential dependence on horizon), under minimal assumptions.
Nonresponse arises frequently in surveys and follow-ups are routinely made to increase the response rate. In order to monitor the follow-up process, callback data have been used in social sciences and survey studies for decades. In modern surveys, the availability of callback data is increasing because the response rate is decreasing and follow-ups are essential to collect maximum information. Although callback data are helpful to reduce the bias in surveys, such data have not been widely used in statistical analysis until recently. We propose a stableness of resistance assumption for nonresponse adjustment with callback data. We establish the identification and the semiparametric efficiency theory under this assumption, and propose a suite of semiparametric estimation methods including a doubly robust one, which generalize existing parametric approaches for callback data analysis. We apply the approach to a Consumer Expenditure Survey dataset. The results suggest an association between nonresponse and high housing expenditures.
AI-driven decision-making can lead to discrimination against certain individuals or social groups based on protected characteristics/attributes such as race, gender, or age. The domain of fairness-aware machine learning focuses on methods and algorithms for understanding, mitigating, and accounting for bias in AI/ML models. Still, thus far, the vast majority of the proposed methods assess fairness based on a single protected attribute, e.g. only gender or race. In reality, though, human identities are multi-dimensional, and discrimination can occur based on more than one protected characteristic, leading to the so-called ``multi-dimensional discrimination'' or ``multi-dimensional fairness'' problem. While well-elaborated in legal literature, the multi-dimensionality of discrimination is less explored in the machine learning community. Recent approaches in this direction mainly follow the so-called intersectional fairness definition from the legal domain, whereas other notions like additive and sequential discrimination are less studied or not considered thus far. In this work, we overview the different definitions of multi-dimensional discrimination/fairness in the legal domain as well as how they have been transferred/ operationalized (if) in the fairness-aware machine learning domain. By juxtaposing these two domains, we draw the connections, identify the limitations, and point out open research directions.
Deep neural networks have achieved tremendous success due to their representation power and adaptation to low-dimensional structures. Their potential for estimating structured regression functions has been recently established in the literature. However, most of the studies require the input dimension to be fixed and consequently ignore the effect of dimension on the rate of convergence and hamper their applications to modern big data with high dimensionality. In this paper, we bridge this gap by analyzing a $k^{th}$ order nonparametric interaction model in both growing dimension scenarios ($d$ grows with $n$ but at a slower rate) and in high dimension ($d \gtrsim n$). In the latter case, sparsity assumptions and associated regularization are required in order to obtain optimal rates of convergence. A new challenge in diverging dimension setting is in calculation mean-square error, the covariance terms among estimated additive components are an order of magnitude larger than those of the variances and they can deteriorate statistical properties without proper care. We introduce a critical debiasing technique to amend the problem. We show that under certain standard assumptions, debiased deep neural networks achieve a minimax optimal rate both in terms of $(n, d)$. Our proof techniques rely crucially on a novel debiasing technique that makes the covariances of additive components negligible in the mean-square error calculation. In addition, we establish the matching lower bounds.
The multivariate adaptive regression spline (MARS) is one of the popular estimation methods for nonparametric multivariate regressions. However, as MARS is based on marginal splines, to incorporate interactions of covariates, products of the marginal splines must be used, which leads to an unmanageable number of basis functions when the order of interaction is high and results in low estimation efficiency. In this paper, we improve the performance of MARS by using linear combinations of the covariates which achieve sufficient dimension reduction. The special basis functions of MARS facilitate calculation of gradients of the regression function, and estimation of the linear combinations is obtained via eigen-analysis of the outer-product of the gradients. Under some technical conditions, the asymptotic theory is established for the proposed estimation method. Numerical studies including both simulation and empirical applications show its effectiveness in dimension reduction and improvement over MARS and other commonly-used nonparametric methods in regression estimation and prediction.
Bayesian inference with nested sampling requires a likelihood-restricted prior sampling method, which draws samples from the prior distribution that exceed a likelihood threshold. For high-dimensional problems, Markov Chain Monte Carlo derivatives have been proposed. We numerically study ten algorithms based on slice sampling, hit-and-run and differential evolution algorithms in ellipsoidal, non-ellipsoidal and non-convex problems from 2 to 100 dimensions. Mixing capabilities are evaluated with the nested sampling shrinkage test. This makes our results valid independent of how heavy-tailed the posteriors are. Given the same number of steps, slice sampling is outperformed by hit-and-run and whitened slice sampling, while whitened hit-and-run does not provide as good results. Proposing along differential vectors of live point pairs also leads to the highest efficiencies, and appears promising for multi-modal problems. The tested proposals are implemented in the UltraNest nested sampling package, enabling efficient low and high-dimensional inference of a large class of practical inference problems relevant to astronomy, cosmology, particle physics and astronomy.
This paper proposes a flexible framework for inferring large-scale time-varying and time-lagged correlation networks from multivariate or high-dimensional non-stationary time series with piecewise smooth trends. Built on a novel and unified multiple-testing procedure of time-lagged cross-correlation functions with a fixed or diverging number of lags, our method can accurately disclose flexible time-varying network structures associated with complex functional structures at all time points. We broaden the applicability of our method to the structure breaks by developing difference-based nonparametric estimators of cross-correlations, achieve accurate family-wise error control via a bootstrap-assisted procedure adaptive to the complex temporal dynamics, and enhance the probability of recovering the time-varying network structures using a new uniform variance reduction technique. We prove the asymptotic validity of the proposed method and demonstrate its effectiveness in finite samples through simulation studies and empirical applications.
We study a class of McKean--Vlasov Stochastic Differential Equations (MV-SDEs) with drifts and diffusions having super-linear growth in measure and space -- the maps have general polynomial form but also satisfy a certain monotonicity condition. The combination of the drift's super-linear growth in measure (by way of a convolution) and the super-linear growth in space and measure of the diffusion coefficient require novel technical elements in order to obtain the main results. We establish wellposedness, propagation of chaos (PoC), and under further assumptions on the model parameters we show an exponential ergodicity property alongside the existence of an invariant distribution. No differentiability or non-degeneracy conditions are required. Further, we present a particle system based Euler-type split-step scheme (SSM) for the simulation of this type of MV-SDEs. The scheme attains, in stepsize, the strong error rate $1/2$ in the non-path-space root-mean-square error metric and we demonstrate the property of mean-square contraction. Our results are illustrated by numerical examples including: estimation of PoC rates across dimensions, preservation of periodic phase-space, and the observation that taming appears to be not a suitable method unless strong dissipativity is present.
The paper addresses the problem of estimation of the model parameters of the logistic exponential distribution based on progressive type-I hybrid censored sample. The maximum likelihood estimates are obtained and computed numerically using Newton-Raphson method. Further, the Bayes estimates are derived under squared error, LINEX and generalized entropy loss functions. Two types (independent and bivariate) of prior distributions are considered for the purpose of Bayesian estimation. It is seen that the Bayes estimates are not of explicit forms.Thus, Lindley's approximation technique is employed to get approximate Bayes estimates. Interval estimates of the parameters based on normal approximate of the maximum likelihood estimates and normal approximation of the log-transformed maximum likelihood estimates are constructed. The highest posterior density credible intervals are obtained by using the importance sampling method. Furthermore, numerical computations are reported to review some of the results obtained in the paper. A real life dataset is considered for the purpose of illustrations.
In the usual Bayesian setting, a full probabilistic model is required to link the data and parameters, and the form of this model and the inference and prediction mechanisms are specified via de Finetti's representation. In general, such a formulation is not robust to model mis-specification of its component parts. An alternative approach is to draw inference based on loss functions, where the quantity of interest is defined as a minimizer of some expected loss, and to construct posterior distributions based on the loss-based formulation; this strategy underpins the construction of the Gibbs posterior. We develop a Bayesian non-parametric approach; specifically, we generalize the Bayesian bootstrap, and specify a Dirichlet process model for the distribution of the observables. We implement this using direct prior-to-posterior calculations, but also using predictive sampling. We also study the assessment of posterior validity for non-standard Bayesian calculations, and provide an efficient way to calibrate the scaling parameter in the Gibbs posterior so that it can achieve the desired coverage rate. We show that the developed non-standard Bayesian updating procedures yield valid posterior distributions in terms of consistency and asymptotic normality under model mis-specification. Simulation studies show that the proposed methods can recover the true value of the parameter efficiently and achieve frequentist coverage even when the sample size is small. Finally, we apply our methods to evaluate the causal impact of speed cameras on traffic collisions in England.
注冊地址： 北京市海淀區羊坊店路18號2幢3層301-191