In view of the extended formulations (EFs) developments (e.g. "Fiorini, S., S. Massar, S. Pokutta, H.R. Tiwary, and R. de Wolf [2015]. Exponential Lower Bounds for Polytopes in Combinatorial Optimization. Journal of the ACM 62:2"), we focus in this paper on the question of whether it is possible to model an NP-Complete problem as a polynomial-sized linear program. For the sake of simplicity of exposition, the discussions are focused on the TSP. We show that a finding that there exists no polynomial-sized extended formulation of "the TSP polytope" does not (necessarily) imply that it is "impossible" for a polynomial-sized linear program to solve the TSP optimization problem. We show that under appropriate conditions the TSP optimization problem can be solved without recourse to the traditional city-to-city ("travel leg") variables, thereby side-stepping/"escaping from" "the TSP polytope" and hence, the barriers. Some illustrative examples are discussed.
We consider the Sobolev embedding operator $E_s : H^s(\Omega) \to L_2(\Omega)$ and its role in the solution of inverse problems. In particular, we collect various properties and investigate different characterizations of its adjoint operator $E_s^*$, which is a common component in both iterative and variational regularization methods. These include variational representations and connections to boundary value problems, Fourier and wavelet representations, as well as connections to spatial filters. Moreover, we consider characterizations in terms of Fourier series, singular value decompositions and frame decompositions, as well as representations in finite dimensional settings. While many of these results are already known to researchers from different fields, a detailed and general overview or reference work containing rigorous mathematical proofs is still missing. Hence, in this paper we aim to fill this gap by collecting, introducing and generalizing a large number of characterizations of $E_s^*$ and discuss their use in regularization methods for solving inverse problems. The resulting compilation can serve both as a reference as well as a useful guide for its efficient numerical implementation in practice.
In this paper, we present splitting algorithms to solve multicomponent transport models with Maxwell-Stefan-diffusion approaches. The multicomponent models are related to transport problems, while we consider plasma processes, in which the local thermodynamic equilibrium and weakly ionized plasma-mixture models are given. Such processes are used for medical and technical applications. These multi-component transport modelling equations are related to convection-diffusion-reactions equations, which are wel-known in transport processes. The multicomponent transport models can be derived from the microscopic multi-component Boltzmann equations with averaging quantities and leads into the macroscopic mass, momentum and energy equations, which are nearly Navier-Stokes-like equations. We discuss the benefits of the decomposition into the convection, diffusion and reaction parts, which allows to use fast numerical solvers for each part. Additional, we concentrate on the nonlinear parts of the multicomponent diffusion, which can be effectively solved with iterative splitting approaches In the numerical experiments, we see the benefit of combining iterative splitting methods with nonlinear solver methods, while these methods can relax the nonlinear terms. In the outview, we discuss the future investigation of the next steps in our multicomponent diffusion approaches.
Tensor train decomposition is widely used in machine learning and quantum physics due to its concise representation of high-dimensional tensors, overcoming the curse of dimensionality. Cross approximation-originally developed for representing a matrix from a set of selected rows and columns-is an efficient method for constructing a tensor train decomposition of a tensor from few of its entries. While tensor train cross approximation has achieved remarkable performance in practical applications, its theoretical analysis, in particular regarding the error of the approximation, is so far lacking. To our knowledge, existing results only provide element-wise approximation accuracy guarantees, which lead to a very loose bound when extended to the entire tensor. In this paper, we bridge this gap by providing accuracy guarantees in terms of the entire tensor for both exact and noisy measurements. Our results illustrate how the choice of selected subtensors affects the quality of the cross approximation and that the approximation error caused by model error and/or measurement error may not grow exponentially with the order of the tensor. These results are verified by numerical experiments, and may have important implications for the usefulness of cross approximations for high-order tensors, such as those encountered in the description of quantum many-body states.
The stochastic block model is a canonical random graph model for clustering and community detection on network-structured data. Decades of extensive study on the problem have established many profound results, among which the phase transition at the Kesten-Stigum threshold is particularly interesting both from a mathematical and an applied standpoint. It states that no estimator based on the network topology can perform substantially better than chance on sparse graphs if the model parameter is below certain threshold. Nevertheless, if we slightly extend the horizon to the ubiquitous semi-supervised setting, such a fundamental limitation will disappear completely. We prove that with arbitrary fraction of the labels revealed, the detection problem is feasible throughout the parameter domain. Moreover, we introduce two efficient algorithms, one combinatorial and one based on optimization, to integrate label information with graph structures. Our work brings a new perspective to stochastic model of networks and semidefinite program research.
The volume penalization (VP) or the Brinkman penalization (BP) method is a diffuse interface method for simulating multiphase fluid-structure interaction (FSI) problems in ocean engineering and/or phase change problems in thermal sciences. The method relies on a penalty factor (which is inversely related to body's permeability $\kappa$) that must be large to enforce rigid body velocity in the solid domain. When the penalty factor is large, the discrete system of equations becomes stiff and difficult to solve numerically. In this paper, we propose a projection method-based preconditioning strategy for solving volume penalized (VP) incompressible and low-Mach Navier-Stokes equations. The projection preconditioner enables the monolithic solution of the coupled velocity-pressure system in both single phase and multiphase flow settings. In this approach, the penalty force is treated implicitly, which is allowed to take arbitrary large values without affecting the solver's convergence rate or causing numerical stiffness/instability. It is made possible by including the penalty term in the pressure Poisson equation. Solver scalability under grid refinement is demonstrated. A manufactured solution in a single phase setting is used to determine the spatial accuracy of the penalized solution. Second-order pointwise accuracy is achieved for both velocity and pressure solutions. Two multiphase fluid-structure interaction (FSI) problems from the ocean engineering literature are also simulated to evaluate the solver's robustness and performance. The proposed solver allows us to investigate the effect of $\kappa$ on the motion of the contact line over the surface of the immersed body. It also allows us to investigate the dynamics of the free surface of a solidifying metal
This paper introduces general methodologies for constructing closed-form solutions to several important partial differential equations (PDEs) with polynomial right-hand sides in two and three spatial dimensions. The covered equations include the isotropic and anisotropic Poisson, Helmholtz, Stokes, and elastostatic equations, as well as the time-harmonic linear elastodynamic and Maxwell equations. Polynomial solutions have recently regained significance in the development of numerical techniques for evaluating volume integral operators and have potential applications in certain kinds of Trefftz finite element methods. Our approach to all of these PDEs relates the particular solution to polynomial solutions of the Poisson and Helmholtz polynomial particular solutions, solutions that can in turn be obtained, respectively, from expansions using homogeneous polynomials and the Neumann series expansion of the operator $(k^2+\Delta)^{-1}$. No matrix inversion is required to compute the solution. The method naturally incorporates divergence constraints on the solution, such as in the case of Maxwell and Stokes flow equations. This work is accompanied by a freely available Julia library, \texttt{PolynomialSolutions.jl}, which implements the proposed methodology in a non-symbolic format and efficiently constructs and provides access to rapid evaluation of the desired solution.
One cannot make truly fair decisions using integer linear programs unless one controls the selection probabilities of the (possibly many) optimal solutions. For this purpose, we propose a unified framework when binary decision variables represent agents with dichotomous preferences, who only care about whether they are selected in the final solution. We develop several general-purpose algorithms to fairly select optimal solutions, for example, by maximizing the Nash product or the minimum selection probability, or by using a random ordering of the agents as a selection criterion (Random Serial Dictatorship). As such, we embed the black-box procedure of solving an integer linear program into a framework that is explainable from start to finish. Moreover, we study the axiomatic properties of the proposed methods by embedding our framework into the rich literature of cooperative bargaining and probabilistic social choice. Lastly, we evaluate the proposed methods on a specific application, namely kidney exchange. We find that while the methods maximizing the Nash product or the minimum selection probability outperform the other methods on the evaluated welfare criteria, methods such as Random Serial Dictatorship perform reasonably well in computation times that are similar to those of finding a single optimal solution.
Given a graph, the $k$-plex is a vertex set in which each vertex is not adjacent to at most $k-1$ other vertices in the set. The maximum $k$-plex problem, which asks for the largest $k$-plex from a given graph, is an important but computationally challenging problem in applications like graph search and community detection. So far, there is a number of empirical algorithms without sufficient theoretical explanations on the efficiency. We try to bridge this gap by defining a novel parameter of the input instance, $g_k(G)$, the gap between the degeneracy bound and the size of maximum $k$-plex in the given graph, and presenting an exact algorithm parameterized by $g_k(G)$. In other words, we design an algorithm with running time polynomial in the size of input graph and exponential in $g_k(G)$ where $k$ is a constant. Usually, $g_k(G)$ is small and bounded by $O(\log{(|V|)})$ in real-world graphs, indicating that the algorithm runs in polynomial time. We also carry out massive experiments and show that the algorithm is competitive with the state-of-the-art solvers. Additionally, for large $k$ values such as $15$ and $20$, our algorithm has superior performance over existing algorithms.
The emergent behavior of a distributed system is conditioned by the information available to the local decision-makers. Therefore, one may expect that providing decision-makers with more information will improve system performance; in this work, we find that this is not necessarily the case. In multi-agent maximum coverage problems, we find that even when agents' objectives are aligned with the global welfare, informing agents about the realization of the resource's random values can reduce equilibrium performance by a factor of 1/2. This affirms an important aspect of designing distributed systems: information need be shared carefully. We further this understanding by providing lower and upper bounds on the ratio of system welfare when information is (fully or partially) revealed and when it is not, termed the value-of-informing. We then identify a trade-off that emerges when optimizing the performance of the best-case and worst-case equilibrium.
The generalization mystery in deep learning is the following: Why do over-parameterized neural networks trained with gradient descent (GD) generalize well on real datasets even though they are capable of fitting random datasets of comparable size? Furthermore, from among all solutions that fit the training data, how does GD find one that generalizes well (when such a well-generalizing solution exists)? We argue that the answer to both questions lies in the interaction of the gradients of different examples during training. Intuitively, if the per-example gradients are well-aligned, that is, if they are coherent, then one may expect GD to be (algorithmically) stable, and hence generalize well. We formalize this argument with an easy to compute and interpretable metric for coherence, and show that the metric takes on very different values on real and random datasets for several common vision networks. The theory also explains a number of other phenomena in deep learning, such as why some examples are reliably learned earlier than others, why early stopping works, and why it is possible to learn from noisy labels. Moreover, since the theory provides a causal explanation of how GD finds a well-generalizing solution when one exists, it motivates a class of simple modifications to GD that attenuate memorization and improve generalization. Generalization in deep learning is an extremely broad phenomenon, and therefore, it requires an equally general explanation. We conclude with a survey of alternative lines of attack on this problem, and argue that the proposed approach is the most viable one on this basis.