Autoencoding is a popular method in representation learning. Conventional autoencoders employ symmetric encoding-decoding procedures and a simple Euclidean latent space to detect hidden low-dimensional structures in an unsupervised way. Some modern approaches to novel data generation such as generative adversarial networks askew this symmetry, but still employ a pair of massive networks--one to generate the image and another to judge the images quality based on priors learned from a training set. This work introduces a chart autoencoder with an asymmetric encoding-decoding process that can incorporate additional semi-supervised information such as class labels. Besides enhancing the capability for handling data with complicated topological and geometric structures, the proposed model can successfully differentiate nearby but disjoint manifolds and intersecting manifolds with only a small amount of supervision. Moreover, this model only requires a low-complexity encoding operation, such as a locally defined linear projection. We discuss the approximation power of such networks and derive a bound that essentially depends on the intrinsic dimension of the data manifold rather than the dimension of ambient space. Next we incorporate bounds for the sampling rate of training data need to faithfully represent a given data manifold. We present numerical experiments that verify that the proposed model can effectively manage data with multi-class nearby but disjoint manifolds of different classes, overlapping manifolds, and manifolds with non-trivial topology. Finally, we conclude with some experiments on computer vision and molecular dynamics problems which showcase the efficacy of our methods on real-world data.
相關內容
Rate splitting multiple access (RSMA) is regarded as an essential and powerful physical-layer (PHY) paradigm for next generation communication systems. Under such a system, users employ successive interference cancellation (SIC), allowing them to decode a portion of the interference and treat the remainder as noise. However, a problem is that current RSMA systems rely on fixed-position antenna arrays, limiting their capacity to fully exploit spatial freedom. This constraint restricts beamforming gain, which substantially degrades RSMA performance. To address this problem, we propose an movable antenna (MA)-aided RSMA scheme that allows the antennas at the base station (BS) to adjust their positions dynamically. Our target is to maximize the system's sum rate of both common and private messages by jointly optimizing the MA positions, beamforming matrix, and common rate allocation. To tackle the formulated non-convex problem, we employ fractional programming (FP) and develop a two-stage, coarse-to-fine-grained search algorithm to obtain suboptimal solutions. Numerical results demonstrate that, with appropriate antenna adjustments, the MA-enabled system significantly enhances the overall performance and reliability of RSMA when employing the proposed algorithm compared to fixed-position antenna configurations.
Large language models (LLMs) have demonstrated remarkable proficiency in machine translation (MT), even without specific training on the languages in question. However, translating rare words in low-resource or domain-specific contexts remains challenging for LLMs. To address this issue, we propose a multi-step prompt chain that enhances translation faithfulness by prioritizing key terms crucial for semantic accuracy. Our method first identifies these keywords and retrieves their translations from a bilingual dictionary, integrating them into the LLM's context using Retrieval-Augmented Generation (RAG). We further mitigate potential output hallucinations caused by long prompts through an iterative self-checking mechanism, where the LLM refines its translations based on lexical and semantic constraints. Experiments using Llama and Qwen as base models on the FLORES-200 and WMT datasets demonstrate significant improvements over baselines, highlighting the effectiveness of our approach in enhancing translation faithfulness and robustness, particularly in low-resource scenarios.
Learning modular object-centric representations is crucial for systematic generalization. Existing methods show promising object-binding capabilities empirically, but theoretical identifiability guarantees remain relatively underdeveloped. Understanding when object-centric representations can theoretically be identified is crucial for scaling slot-based methods to high-dimensional images with correctness guarantees. To that end, we propose a probabilistic slot-attention algorithm that imposes an aggregate mixture prior over object-centric slot representations, thereby providing slot identifiability guarantees without supervision, up to an equivalence relation. We provide empirical verification of our theoretical identifiability result using both simple 2-dimensional data and high-resolution imaging datasets.
3D geometric shape completion hinges on representation learning and a deep understanding of geometric data. Without profound insights into the three-dimensional nature of the data, this task remains unattainable. Our work addresses this challenge of 3D shape completion given partial observations by proposing a transformer operating on the latent space representing Signed Distance Fields (SDFs). Instead of a monolithic volume, the SDF of an object is partitioned into smaller high-resolution patches leading to a sequence of latent codes. The approach relies on a smooth latent space encoding learned via a variational autoencoder (VAE), trained on millions of 3D patches. We employ an efficient masked autoencoder transformer to complete partial sequences into comprehensive shapes in latent space. Our approach is extensively evaluated on partial observations from ShapeNet and the ABC dataset where only fractions of the objects are given. The proposed POC-SLT architecture compares favorably with several baseline state-of-the-art methods, demonstrating a significant improvement in 3D shape completion, both qualitatively and quantitatively.
Lift-and-Embed Learning Methods for Solving Scalar Hyperbolic Equations with Discontinuous Solutions
Unlike traditional mesh-based approximations of differential operators, machine learning methods, which exploit the automatic differentiation of neural networks, have attracted increasing attention for their potential to mitigate stability issues encountered in the numerical simulation of hyperbolic conservation laws. However, solutions to hyperbolic problems are often piecewise smooth, rendering the differential form invalid along discontinuity interfaces and limiting the effectiveness of standard learning approaches. In this work, we propose lift-and-embed learning methods for solving scalar hyperbolic equations with discontinuous solutions, which consist of (i) embedding the Rankine-Hugoniot jump condition within a higher-dimensional space through the inclusion of an augmented variable in the solution ansatz; (ii) utilizing physics-informed neural networks to manage the increased dimensionality and to address both linear and quasi-linear problems within a unified learning framework; and (iii) projecting the trained network solution back onto the original lower-dimensional plane to obtain the approximate solution. Besides, the location of discontinuity can be parametrized as extra model parameters and inferred concurrently with the training of network solution. With collocation points sampled on piecewise surfaces rather than distributed over the entire lifted space, we conduct numerical experiments on various benchmark problems to demonstrate the capability of our methods in resolving discontinuous solutions without spurious numerical smearing and oscillations.
Quantum error-correcting codes (QECCs) are necessary for fault-tolerant quantum computation. Surface codes are a class of topological QECCs that have attracted significant attention due to their exceptional error-correcting capabilities and easy implementation. In the decoding process of surface codes, the syndromes are crucial for error correction, however, they are not always correctly measured. Most of the existing decoding algorithms for surface codes need extra measurements to correct syndromes with errors, which implies a potential increase in inference complexity and decoding latency. In this paper, we propose a high-performance list decoding algorithm for surface codes with erroneous syndromes, where syndrome soft information is incorporated in the decoding, allowing qubits and syndrome to be recovered without needing extra measurements. Precisely, we first use belief propagation (BP) decoding for pre-processing with syndrome soft information, followed by ordered statistics decoding (OSD) for post-processing to list and recover both qubits and syndromes. Numerical results demonstrate that our proposed algorithm efficiently recovers erroneous syndromes and significantly improves the decoding performance of surface codes with erroneous syndromes compared to minimum-weight perfect matching (MWPM), BP and original BP-OSD algorithms.
External knowledge is often useful for natural language understanding tasks. We introduce a contextual text representation model called Conceptual-Contextual (CC) embeddings, which incorporates structured knowledge into text representations. Unlike entity embedding methods, our approach encodes a knowledge graph into a context model. CC embeddings can be easily reused for a wide range of tasks just like pre-trained language models. Our model effectively encodes the huge UMLS database by leveraging semantic generalizability. Experiments on electronic health records (EHRs) and medical text processing benchmarks showed our model gives a major boost to the performance of supervised medical NLP tasks.
Aspect level sentiment classification aims to identify the sentiment expressed towards an aspect given a context sentence. Previous neural network based methods largely ignore the syntax structure in one sentence. In this paper, we propose a novel target-dependent graph attention network (TD-GAT) for aspect level sentiment classification, which explicitly utilizes the dependency relationship among words. Using the dependency graph, it propagates sentiment features directly from the syntactic context of an aspect target. In our experiments, we show our method outperforms multiple baselines with GloVe embeddings. We also demonstrate that using BERT representations further substantially boosts the performance.
External knowledge is often useful for natural language understanding tasks. We introduce a contextual text representation model called Conceptual-Contextual (CC) embeddings, which incorporates structured knowledge into text representations. Unlike entity embedding methods, our approach encodes a knowledge graph into a context model. CC embeddings can be easily reused for a wide range of tasks just like pre-trained language models. Our model effectively encodes the huge UMLS database by leveraging semantic generalizability. Experiments on electronic health records (EHRs) and medical text processing benchmarks showed our model gives a major boost to the performance of supervised medical NLP tasks.
It is important to detect anomalous inputs when deploying machine learning systems. The use of larger and more complex inputs in deep learning magnifies the difficulty of distinguishing between anomalous and in-distribution examples. At the same time, diverse image and text data are available in enormous quantities. We propose leveraging these data to improve deep anomaly detection by training anomaly detectors against an auxiliary dataset of outliers, an approach we call Outlier Exposure (OE). This enables anomaly detectors to generalize and detect unseen anomalies. In extensive experiments on natural language processing and small- and large-scale vision tasks, we find that Outlier Exposure significantly improves detection performance. We also observe that cutting-edge generative models trained on CIFAR-10 may assign higher likelihoods to SVHN images than to CIFAR-10 images; we use OE to mitigate this issue. We also analyze the flexibility and robustness of Outlier Exposure, and identify characteristics of the auxiliary dataset that improve performance.
注冊地址： 北京市海淀區羊坊店路18號2幢3層301-191