Many frameworks exist to infer cause and effect relations in complex nonlinear systems but a complete theory is lacking. A new framework is presented that is fully nonlinear, provides a complete information theoretic disentanglement of causal processes, allows for nonlinear interactions between causes, identifies the causal strength of missing or unknown processes, and can analyze systems that cannot be represented on Directed Acyclic Graphs. The basic building blocks are information theoretic measures such as (conditional) mutual information and a new concept called certainty that monotonically increases with the information available about the target process. The framework is presented in detail and compared with other existing frameworks, and the treatment of confounders is discussed. While there are systems with structures that the framework cannot disentangle, it is argued that any causal framework that is based on integrated quantities will miss out potentially important information of the underlying probability density functions. The framework is tested on several highly simplified stochastic processes to demonstrate how blocking and gateways are handled, and on the chaotic Lorentz 1963 system. We show that the framework provides information on the local dynamics, but also reveals information on the larger scale structure of the underlying attractor. Furthermore, by applying it to real observations related to the El-Nino-Southern-Oscillation system we demonstrate its power and advantage over other methodologies.
相關內容
Identifying cause-effect relations among variables is a key step in the decision-making process. While causal inference requires randomized experiments, researchers and policymakers are increasingly using observational studies to test causal hypotheses due to the wide availability of observational data and the infeasibility of experiments. The matching method is the most used technique to make causal inference from observational data. However, the pair assignment process in one-to-one matching creates uncertainty in the inference because of different choices made by the experimenter. Recently, discrete optimization models are proposed to tackle such uncertainty. Although a robust inference is possible with discrete optimization models, they produce nonlinear problems and lack scalability. In this work, we propose greedy algorithms to solve the robust causal inference test instances from observational data with continuous outcomes. We propose a unique framework to reformulate the nonlinear binary optimization problems as feasibility problems. By leveraging the structure of the feasibility formulation, we develop greedy schemes that are efficient in solving robust test problems. In many cases, the proposed algorithms achieve global optimal solutions. We perform experiments on three real-world datasets to demonstrate the effectiveness of the proposed algorithms and compare our result with the state-of-the-art solver. Our experiments show that the proposed algorithms significantly outperform the exact method in terms of computation time while achieving the same conclusion for causal tests. Both numerical experiments and complexity analysis demonstrate that the proposed algorithms ensure the scalability required for harnessing the power of big data in the decision-making process.
The fundamental communication problem in the wireless Internet of Things (IoT) is to discover a massive number of devices and to allow them reliable access to shared channels. Oftentimes these devices transmit short messages randomly and sporadically. This paper proposes a novel signaling scheme for grant-free massive access, where each device encodes its identity and/or information in a sparse set of tones. Such transmissions are implemented in the form of orthogonal frequency-division multiple access (OFDMA). Under some mild conditions and assuming device delays to be bounded unknown multiples of symbol intervals, sparse OFDMA is proved to enable arbitrarily reliable asynchronous device identification and message decoding with a codelength that is O(K(log K + log S + log N)), where N denotes the device population, K denotes the actual number of active devices, and log S is essentially equal to the number of bits a device can send (including its identity). By exploiting the Fast Fourier Transform (FFT), the computational complexity for discovery and decoding can be made to be sub-linear in the total device population. To prove the concept, a specific design is proposed to identify up to 100 active devices out of $2^{38}$ possible devices with up to 20 symbols of delay and moderate signal-to-noise ratios and fading. The codelength compares much more favorably with those of standard slotted ALOHA and carrier-sensing multiple access (CSMA) schemes.
Riemannian manifold Hamiltonian Monte Carlo (RMHMC) is a sampling algorithm that seeks to adapt proposals to the local geometry of the posterior distribution. The specific form of the Hamiltonian used in RMHMC necessitates {\it implicitly-defined} numerical integrators in order to sustain reversibility and volume-preservation, two properties that are necessary to establish detailed balance of RMHMC. In practice, these implicit equations are solved to a non-zero convergence tolerance via fixed-point iteration. However, the effect of these convergence thresholds on the ergodicity and computational efficiency properties of RMHMC are not well understood. The purpose of this research is to elucidate these relationships through numerous case studies. Our analysis reveals circumstances wherein the RMHMC algorithm is sensitive, and insensitive, to these convergence tolerances. Our empirical analysis examines several aspects of the computation: (i) we examine the ergodicity of the RMHMC Markov chain by employing statistical methods for comparing probability measures based on collections of samples; (ii) we investigate the degree to which detailed balance is violated by measuring errors in reversibility and volume-preservation; (iii) we assess the efficiency of the RMHMC Markov chain in terms of time-normalized ESS. In each of these cases, we investigate the sensitivity of these metrics to the convergence threshold and further contextualize our results in terms of comparison against Euclidean HMC. We propose a method by which one may select the convergence tolerance within a Bayesian inference application using techniques of stochastic approximation and we examine Newton's method, an alternative to fixed point iterations, which can eliminate much of the sensitivity of RMHMC to the convergence threshold.
For multivariate spatial Gaussian process (GP) models, customary specifications of cross-covariance functions do not exploit relational inter-variable graphs to ensure process-level conditional independence among the variables. This is undesirable, especially for highly multivariate settings, where popular cross-covariance functions such as the multivariate Mat\'ern suffer from a "curse of dimensionality" as the number of parameters and floating point operations scale up in quadratic and cubic order, respectively, in the number of variables. We propose a class of multivariate "Graphical Gaussian Processes" using a general construction called "stitching" that crafts cross-covariance functions from graphs and ensures process-level conditional independence among variables. For the Mat\'ern family of functions, stitching yields a multivariate GP whose univariate components are Mat\'ern GPs, and conforms to process-level conditional independence as specified by the graphical model. For highly multivariate settings and decomposable graphical models, stitching offers massive computational gains and parameter dimension reduction. We demonstrate the utility of the graphical Mat\'ern GP to jointly model highly multivariate spatial data using simulation examples and an application to air-pollution modelling.
Millimeter wave (mmWave) and terahertz (THz) radio access technologies (RAT) are expected to become a critical part of the future cellular ecosystem providing an abundant amount of bandwidth in areas with high traffic demands. However, extremely directional antenna radiation patterns that need to be utilized at both transmit and receive sides of a link to overcome severe path losses, dynamic blockage of propagation paths by large static and small dynamic objects, macro- and micromobility of user equipment (UE) makes provisioning of reliable service over THz/mmWave RATs an extremely complex task. This challenge is further complicated by the type of applications envisioned for these systems inherently requiring guaranteed bitrates at the air interface. This tutorial aims to introduce a versatile mathematical methodology for assessing performance reliability improvement algorithms for mmWave and THz systems. Our methodology accounts for both radio interface specifics as well as service process of sessions at mmWave/THz base stations (BS) and is capable of evaluating the performance of systems with multiconnectivity operation, resource reservation mechanisms, priorities between multiple traffic types having different service requirements. The framework is logically separated into two parts: (i) parameterization part that abstracts the specifics of deployment and radio mechanisms, and (ii) queuing part, accounting for details of the service process at mmWave/THz BSs. The modular decoupled structure of the framework allows for further extensions to advanced service mechanisms in prospective mmWave/THz cellular deployments while keeping the complexity manageable and thus making it attractive for system analysts.
We consider the problem of learning stabilizable systems governed by nonlinear state equation $h_{t+1}=\phi(h_t,u_t;\theta)+w_t$. Here $\theta$ is the unknown system dynamics, $h_t $ is the state, $u_t$ is the input and $w_t$ is the additive noise vector. We study gradient based algorithms to learn the system dynamics $\theta$ from samples obtained from a single finite trajectory. If the system is run by a stabilizing input policy, we show that temporally-dependent samples can be approximated by i.i.d. samples via a truncation argument by using mixing-time arguments. We then develop new guarantees for the uniform convergence of the gradients of empirical loss. Unlike existing work, our bounds are noise sensitive which allows for learning ground-truth dynamics with high accuracy and small sample complexity. Together, our results facilitate efficient learning of the general nonlinear system under stabilizing policy. We specialize our guarantees to entry-wise nonlinear activations and verify our theory in various numerical experiments
Assured AI in unrestricted settings is a critical problem. Our framework addresses AI assurance challenges lying at the intersection of domain adaptation, fairness, and counterfactuals analysis, operating via the discovery and intervention on factors of variations in data (e.g. weather or illumination conditions) that significantly affect the robustness of AI models. Robustness is understood here as insensitivity of the model performance to variations in sensitive factors. Sensitive factors are traditionally set in a supervised setting, whereby factors are known a-priori (e.g. for fairness this could be factors like sex or race). In contrast, our motivation is real-life scenarios where less, or nothing, is actually known a-priori about certain factors that cause models to fail. This leads us to consider various settings (unsupervised, domain generalization, semi-supervised) that correspond to different degrees of incomplete knowledge about those factors. Therefore, our two step approach works by a) discovering sensitive factors that cause AI systems to fail in a unsupervised fashion, and then b) intervening models to lessen these factor's influence. Our method considers 3 interventions consisting of Augmentation, Coherence, and Adversarial Interventions (ACAI). We demonstrate the ability for interventions on discovered/source factors to generalize to target/real factors. We also demonstrate how adaptation to real factors of variations can be performed in the semi-supervised case where some target factor labels are known, via automated intervention selection. Experiments show that our approach improves on baseline models, with regard to achieving optimal utility vs. sensitivity/robustness tradeoffs.
The rapid growth of data in the recent years has led to the development of complex learning algorithms that are often used to make decisions in real world. While the positive impact of the algorithms has been tremendous, there is a need to mitigate any bias arising from either training samples or implicit assumptions made about the data samples. This need becomes critical when algorithms are used in automated decision making systems that can hugely impact people's lives. Many approaches have been proposed to make learning algorithms fair by detecting and mitigating bias in different stages of optimization. However, due to a lack of a universal definition of fairness, these algorithms optimize for a particular interpretation of fairness which makes them limited for real world use. Moreover, an underlying assumption that is common to all algorithms is the apparent equivalence of achieving fairness and removing bias. In other words, there is no user defined criteria that can be incorporated into the optimization procedure for producing a fair algorithm. Motivated by these shortcomings of existing methods, we propose the FAIRLEARN procedure that produces a fair algorithm by incorporating user constraints into the optimization procedure. Furthermore, we make the process interpretable by estimating the most predictive features from data. We demonstrate the efficacy of our approach on several real world datasets using different fairness criteria.
Many current applications use recommendations in order to modify the natural user behavior, such as to increase the number of sales or the time spent on a website. This results in a gap between the final recommendation objective and the classical setup where recommendation candidates are evaluated by their coherence with past user behavior, by predicting either the missing entries in the user-item matrix, or the most likely next event. To bridge this gap, we optimize a recommendation policy for the task of increasing the desired outcome versus the organic user behavior. We show this is equivalent to learning to predict recommendation outcomes under a fully random recommendation policy. To this end, we propose a new domain adaptation algorithm that learns from logged data containing outcomes from a biased recommendation policy and predicts recommendation outcomes according to random exposure. We compare our method against state-of-the-art factorization methods, in addition to new approaches of causal recommendation and show significant improvements.
In this paper, we study the optimal convergence rate for distributed convex optimization problems in networks. We model the communication restrictions imposed by the network as a set of affine constraints and provide optimal complexity bounds for four different setups, namely: the function $F(\xb) \triangleq \sum_{i=1}^{m}f_i(\xb)$ is strongly convex and smooth, either strongly convex or smooth or just convex. Our results show that Nesterov's accelerated gradient descent on the dual problem can be executed in a distributed manner and obtains the same optimal rates as in the centralized version of the problem (up to constant or logarithmic factors) with an additional cost related to the spectral gap of the interaction matrix. Finally, we discuss some extensions to the proposed setup such as proximal friendly functions, time-varying graphs, improvement of the condition numbers.
小貼士
登錄享
相關主題
注冊地址： 北京市海淀區羊坊店路18號2幢3層301-191