In this paper, we study the problem of efficiently and effectively embedding the high-dimensional spatio-spectral information of hyperspectral (HS) images, guided by feature diversity. Specifically, based on the theoretical formulation that feature diversity is correlated with the rank of the unfolded kernel matrix, we rectify 3D convolution by modifying its topology to enhance the rank upper-bound. This modification yields a rank-enhanced spatial-spectral symmetrical convolution set (ReS$^3$-ConvSet), which not only learns diverse and powerful feature representations but also saves network parameters. Additionally, we also propose a novel diversity-aware regularization (DA-Reg) term that directly acts on the feature maps to maximize independence among elements. To demonstrate the superiority of the proposed ReS$^3$-ConvSet and DA-Reg, we apply them to various HS image processing and analysis tasks, including denoising, spatial super-resolution, and classification. Extensive experiments show that the proposed approaches outperform state-of-the-art methods both quantitatively and qualitatively to a significant extent. The code is publicly available at //github.com/jinnh/ReSSS-ConvSet.
相關內容
In this study, we explore the performance of a reconfigurable reflecting surface (RIS)-assisted transmit spatial modulation (SM) system for downlink transmission, wherein the deployment of RIS serves the purpose of blind area coverage within the channel. At the receiving end, we present three detectors, i.e., maximum likelihood (ML) detector, two-stage ML detection, and greedy detector to recover the transmitted signal. By utilizing the ML detector, we initially derive the conditional pair error probability expression for the proposed scheme. Subsequently, we leverage the central limit theorem (CLT) to obtain the probability density function of the combined channel. Following this, the Gaussian-Chebyshev quadrature method is applied to derive a closed-form expression for the unconditional pair error probability and establish the union tight upper bound for the average bit error probability (ABEP). Furthermore, we derive a closed-form expression for the ergodic capacity of the proposed RIS-SM scheme. Monte Carlo simulations are conducted not only to assess the complexity and reliability of the three detection algorithms but also to validate the results obtained through theoretical derivation results.
In this paper, we investigate the negative effect of activation functions on forward and backward propagation and how to counteract this effect. First, We examine how activation functions affect the forward and backward propagation of neural networks and derive a general form for gradient variance that extends the previous work in this area. We try to use mini-batch statistics to dynamically update the normalization factor to ensure the normalization property throughout the training process, rather than only accounting for the state of the neural network after weight initialization. Second, we propose ANAct, a method that normalizes activation functions to maintain consistent gradient variance across layers and demonstrate its effectiveness through experiments. We observe that the convergence rate is roughly related to the normalization property. We compare ANAct with several common activation functions on CNNs and residual networks and show that ANAct consistently improves their performance. For instance, normalized Swish achieves 1.4\% higher top-1 accuracy than vanilla Swish on ResNet50 with the Tiny ImageNet dataset and more than 1.2\% higher with CIFAR-100.
Motivated by the need for communication-efficient distributed learning, we investigate the method for compressing a unit norm vector into the minimum number of bits, while still allowing for some acceptable level of distortion in recovery. This problem has been explored in the rate-distortion/covering code literature, but our focus is exclusively on the "high-distortion" regime. We approach this problem in a worst-case scenario, without any prior information on the vector, but allowing for the use of randomized compression maps. Our study considers both biased and unbiased compression methods and determines the optimal compression rates. It turns out that simple compression schemes are nearly optimal in this scenario. While the results are a mix of new and known, they are compiled in this paper for completeness.
In this paper, we study the stochastic collocation (SC) methods for uncertainty quantification (UQ) in hyperbolic systems of nonlinear partial differential equations (PDEs). In these methods, the underlying PDEs are numerically solved at a set of collocation points in random space. A standard SC approach is based on a generalized polynomial chaos (gPC) expansion, which relies on choosing the collocation points based on the prescribed probability distribution and approximating the computed solution by a linear combination of orthogonal polynomials in the random variable. We demonstrate that this approach struggles to accurately capture discontinuous solutions, often leading to oscillations (Gibbs phenomenon) that deviate significantly from the physical solutions. We explore alternative SC methods, in which one can choose an arbitrary set of collocation points and employ shape-preserving splines to interpolate the solution in a random space. Our study demonstrates the effectiveness of spline-based collocation in accurately capturing and assessing uncertainties while suppressing oscillations. We illustrate the superiority of the spline-based collocation on two numerical examples, including the inviscid Burgers and shallow water equations.
In this research, we present SLYKLatent, a novel approach for enhancing gaze estimation by addressing appearance instability challenges in datasets due to aleatoric uncertainties, covariant shifts, and test domain generalization. SLYKLatent utilizes Self-Supervised Learning for initial training with facial expression datasets, followed by refinement with a patch-based tri-branch network and an inverse explained variance-weighted training loss function. Our evaluation on benchmark datasets achieves an 8.7% improvement on Gaze360, rivals top MPIIFaceGaze results, and leads on a subset of ETH-XGaze by 13%, surpassing existing methods by significant margins. Adaptability tests on RAF-DB and Affectnet show 86.4% and 60.9% accuracies, respectively. Ablation studies confirm the effectiveness of SLYKLatent's novel components. This approach has strong potential in human-robot interaction.
To address the communication bottleneck challenge in distributed learning, our work introduces a novel two-stage quantization strategy designed to enhance the communication efficiency of distributed Stochastic Gradient Descent (SGD). The proposed method initially employs truncation to mitigate the impact of long-tail noise, followed by a non-uniform quantization of the post-truncation gradients based on their statistical characteristics. We provide a comprehensive convergence analysis of the quantized distributed SGD, establishing theoretical guarantees for its performance. Furthermore, by minimizing the convergence error, we derive optimal closed-form solutions for the truncation threshold and non-uniform quantization levels under given communication constraints. Both theoretical insights and extensive experimental evaluations demonstrate that our proposed algorithm outperforms existing quantization schemes, striking a superior balance between communication efficiency and convergence performance.
Despite the plethora of research devoted to analyzing the impact of disability on travel behavior, not enough studies have investigated the varying impact of social and environmental factors on the mode choice of people with disabilities that restrict their ability to use transportation modes efficiently. This research gap can be addressed by investigating the factors influencing the mode choice behavior of people with travel-limiting disabilities, which can inform the development of accessible and sustainable transportation systems. Additionally, such studies can provide insights into the social and economic barriers faced by this population group, which can help policymakers to promote social inclusion and equity. This study utilized a Random Parameters Logit model to identify the individual, trip, and environmental factors that influence mode selection among people with travel-limiting disabilities. Using the 2017 National Household Travel Survey data for New York State, which included information on respondents with travel-limiting disabilities, the analysis focused on a sample of 8,016 people. In addition, climate data from the National Oceanic and Atmospheric Administration were integrated as additional explanatory variables in the modeling process. The results revealed that people with disabilities may be inclined to travel longer distances walking in the absence of suitable accommodation facilities for other transportation modes. Furthermore, people were less inclined to walk during summer and winter, indicating a need to consider weather conditions as a significant determinant of mode choice. Moreover, low-income people with disabilities were more likely to rely on public transport or walking.
In this paper, we study the problem of efficient online reinforcement learning in the infinite horizon setting when there is an offline dataset to start with. We assume that the offline dataset is generated by an expert but with unknown level of competence, i.e., it is not perfect and not necessarily using the optimal policy. We show that if the learning agent models the behavioral policy (parameterized by a competence parameter) used by the expert, it can do substantially better in terms of minimizing cumulative regret, than if it doesn't do that. We establish an upper bound on regret of the exact informed PSRL algorithm that scales as $\tilde{O}(\sqrt{T})$. This requires a novel prior-dependent regret analysis of Bayesian online learning algorithms for the infinite horizon setting. We then propose the Informed RLSVI algorithm to efficiently approximate the iPSRL algorithm.
Because of their excellent asymptotic and finite-length performance, spatially-coupled (SC) codes are a class of low-density parity-check codes that is gaining increasing attention. Multi-dimensional (MD) SC codes are constructed by connecting copies of an SC code via relocations in order to mitigate various sources of non-uniformity and improve performance in many data storage and data transmission systems. As the number of degrees of freedom in the MD-SC code design increases, appropriately exploiting them becomes more difficult because of the complexity growth of the design process. In this paper, we propose a probabilistic framework for the MD-SC code design, which is based on the gradient-descent (GD) algorithm, to design better MD codes and address this challenge. In particular, we express the expected number of short cycles, which we seek to minimize, in the graph representation of the code in terms of entries of a probability-distribution matrix that characterizes the MD-SC code design. We then find a locally-optimal probability distribution, which serves as the starting point of a finite-length algorithmic optimizer that produces the final MD-SC code. We offer the theoretical analysis as well as the algorithms, and we present experimental results demonstrating that our MD codes, conveniently called GD-MD codes, have notably lower short cycle numbers compared with the available state-of-the-art. Moreover, our algorithms converge on solutions in few iterations, which confirms the complexity reduction as a result of limiting the search space via the locally-optimal GD-MD distributions.
In this paper, we provide a theoretical study of noise geometry for minibatch stochastic gradient descent (SGD), a phenomenon where noise aligns favorably with the geometry of local landscape. We propose two metrics, derived from analyzing how noise influences the loss and subspace projection dynamics, to quantify the alignment strength. We show that for (over-parameterized) linear models and two-layer nonlinear networks, when measured by these metrics, the alignment can be provably guaranteed under conditions independent of the degree of over-parameterization. To showcase the utility of our noise geometry characterizations, we present a refined analysis of the mechanism by which SGD escapes from sharp minima. We reveal that unlike gradient descent (GD), which escapes along the sharpest directions, SGD tends to escape from flatter directions and cyclical learning rates can exploit this SGD characteristic to navigate more effectively towards flatter regions. Lastly, extensive experiments are provided to support our theoretical findings.
小貼士
登錄享
相關主題
注冊地址： 北京市海淀區羊坊店路18號2幢3層301-191