We consider the problem of secure distributed matrix multiplication (SDMM) in which a user wishes to compute the product of two matrices with the assistance of honest but curious servers. We construct polynomial codes for SDMM by studying a recently introduced combinatorial tool called the degree table. For a fixed partitioning, minimizing the total communication cost of a polynomial code for SDMM is equivalent to minimizing $N$, the number of distinct elements in the corresponding degree table. We propose new constructions of degree tables with a low number of distinct elements. These new constructions lead to a general family of polynomial codes for SDMM, which we call $\mathsf{GASP}_{r}$ (Gap Additive Secure Polynomial codes) parametrized by an integer $r$. $\mathsf{GASP}_{r}$ outperforms all previously known polynomial codes for SDMM under an outer product partitioning. We also present lower bounds on $N$ and prove the optimality or asymptotic optimality of our constructions for certain regimes. Moreover, we formulate the construction of optimal degree tables as an integer linear program and use it to prove the optimality of $\mathsf{GASP}_{r}$ for all the system parameters that we were able to test.
We consider the problem of communication efficient secure distributed matrix multiplication. The previous literature has focused on reducing the number of servers as a proxy for minimizing communication costs. The intuition being, that the more servers used, the higher the communication cost. We show that this is not the case. Our central technique relies on adapting results from the literature on repairing Reed-Solomon codes where instead of downloading the whole of the computing task, a user downloads field traces of these computations. We present field trace polynomial codes, a family of codes, that explore this technique and characterize regimes for which our codes outperform the existing codes in the literature.
Many numerical methods for evaluating matrix functions can be naturally viewed as computational graphs. Rephrasing these methods as directed acyclic graphs (DAGs) is a particularly effective approach to study existing techniques, improve them, and eventually derive new ones. The accuracy of these matrix techniques can be characterized by the accuracy of their scalar counterparts, thus designing algorithms for matrix functions can be regarded as a scalar-valued optimization problem. The derivatives needed during the optimization can be calculated automatically by exploiting the structure of the DAG, in a fashion analogous to backpropagation. This paper describes GraphMatFun.jl, a Julia package that offers the means to generate and manipulate computational graphs, optimize their coefficients, and generate Julia, MATLAB, and C code to evaluate them efficiently at a matrix argument. The software also provides tools to estimate the accuracy of a graph-based algorithm and thus obtain numerically reliable methods. For the exponential, for example, using a particular form (degree-optimal) of polynomials produces implementations that in many cases are cheaper, in terms of computational cost, than the Pad\'e-based techniques typically used in mathematical software. The optimized graphs and the corresponding generated code are available online.
This paper studies a large unitarily invariant system (LUIS) involving a unitarily invariant sensing matrix, an arbitrary signal distribution, and forward error control (FEC) coding. We develop a universal Gram-Schmidt orthogonalization for orthogonal approximate message passing (OAMP). Numerous area properties are established based on the state evolution and minimum mean squared error (MMSE) property of OAMP in an un-coded LUIS. As a byproduct, we provide an alternative derivation for the constrained capacity of a LUIS. Under the assumption that the state evolution for OAMP is correct for the coded system, the achievable rate of OAMP is analyzed. We prove that OAMP achieves the constrained capacity of the LUIS with an arbitrary signal distribution provided that a matching condition is satisfied. Meanwhile, we elaborate a capacity-achieving coding principle for LUIS, based on which irregular low-density parity-check (LDPC) codes are optimized for binary signaling in the numerical results. We show that OAMP with the optimized codes has significant performance improvement over the un-optimized ones and the well-known Turbo linear MMSE algorithm. For quadrature phase-shift keying (QPSK) modulation, capacity-approaching bit error rate (BER) performances are observed under various channel conditions.
This article is about the architecture of a lossless wavelet filter bank with reprogrammable logic. It is based on second generation of wavelets with a reduced of number of operations. A new basic structure for parallel architecture and modules to forward and backward integer discrete wavelet transform is proposed.
Coded computing is an effective technique to mitigate "stragglers" in large-scale and distributed matrix multiplication. In particular, univariate polynomial codes have been shown to be effective in straggler mitigation by making the computation time depend only on the fastest workers. However, these schemes completely ignore the work done by the straggling workers resulting in a waste of computational resources. To reduce the amount of work left unfinished at workers, one can further decompose the matrix multiplication task into smaller sub-tasks, and assign multiple sub-tasks to each worker, possibly heterogeneously, to better fit their particular storage and computation capacities. In this work, we propose a novel family of bivariate polynomial codes to efficiently exploit the work carried out by straggling workers. We show that bivariate polynomial codes bring significant advantages in terms of upload communication costs and storage efficiency, measured in terms of the number of sub-tasks that can be computed per worker. We propose two bivariate polynomial coding schemes. The first one exploits the fact that bivariate interpolation is always possible on a rectangular grid of evaluation points. We obtain such points at the cost of adding some redundant computations. For the second scheme, we relax the decoding constraints and require decodability for almost all choices of the evaluation points. We present interpolation sets satisfying such decodability conditions for certain storage configurations of workers. Our numerical results show that bivariate polynomial coding considerably reduces the average computation time of distributed matrix multiplication. We believe this work opens up a new class of previously unexplored coding schemes for efficient coded distributed computation.
We prove that if $n >k^2$ then a $k$-dimensional linear code of length $n$ over ${\mathbb F}_{q^2}$ has a truncation which is linearly equivalent to a Hermitian self-orthogonal linear code. In the contrary case we prove that truncations of linear codes to codes equivalent to Hermitian self-orthogonal linear codes occur when the columns of a generator matrix of the code do not impose independent conditions on the space of Hermitian forms. In the case that there are more than $n$ common zeros to the set of Hermitian forms which are zero on the columns of a generator matrix of the code, the additional zeros give the extension of the code to a code that has a truncation which is equivalent to a Hermitian self-orthogonal code.
Secure codes are widely-studied combinatorial structures which were introduced for traitor tracing in broadcast encryption. To determine the maximum size of such structures is the main research objective. In this paper, we investigate the lower bounds for secure codes and their related structures. First, we give some improved lower bounds for the rates of $2$-frameproof codes and $\overline{2}$-separable codes for slightly large alphabet size. Then we improve the lower bounds for the rate of some related structures, i.e., strongly $2$-separable matrices and $2$-cancellative set families. Finally, we give a general method to derive new lower bounds for strongly $t$-separable matrices and $t$-cancellative set families for $t\ge 3.$
In this paper we give a survey of methods used to calculate values of resistance distance (also known as effective resistance) in graphs. Resistance distance has played a prominent role not only in circuit theory and chemistry, but also in combinatorial matrix theory and spectral graph theory. Moreover resistance distance has applications ranging from quantifying biological structures, distributed control systems, network analysis, and power grid systems. In this paper we discuss both exact techniques and approximate techniques and for each method discussed we provide an illustrative example of the technique. We also present some open questions and conjectures.
We consider the problem of predicting a response $Y$ from a set of covariates $X$ when test and training distributions differ. Since such differences may have causal explanations, we consider test distributions that emerge from interventions in a structural causal model, and focus on minimizing the worst-case risk. Causal regression models, which regress the response on its direct causes, remain unchanged under arbitrary interventions on the covariates, but they are not always optimal in the above sense. For example, for linear models and bounded interventions, alternative solutions have been shown to be minimax prediction optimal. We introduce the formal framework of distribution generalization that allows us to analyze the above problem in partially observed nonlinear models for both direct interventions on $X$ and interventions that occur indirectly via exogenous variables $A$. It takes into account that, in practice, minimax solutions need to be identified from data. Our framework allows us to characterize under which class of interventions the causal function is minimax optimal. We prove sufficient conditions for distribution generalization and present corresponding impossibility results. We propose a practical method, NILE, that achieves distribution generalization in a nonlinear IV setting with linear extrapolation. We prove consistency and present empirical results.
We consider a mobile edge computing scenario where users want to perform a linear inference operation $\boldsymbol{W} \boldsymbol{x}$ on local data $\boldsymbol{x}$ for some network-side matrix $\boldsymbol{W}$. The inference is performed in a distributed fashion over multiple servers at the network edge. For this scenario, we propose a coding scheme that combines a rateless code to provide resiliency against straggling servers--hence reducing the computation latency--and an irregular-repetition code to provide spatial diversity--hence reducing the communication latency. We further derive a lower bound on the total latency--comprising computation latency, communication latency, and decoding latency. The proposed scheme performs remarkably close to the bound and yields significantly lower latency than the scheme based on maximum distance separable codes recently proposed by Zhang and Simeone.