This study explores the impact of adversarial perturbations on Convolutional Neural Networks (CNNs) with the aim of enhancing the understanding of their underlying mechanisms. Despite numerous defense methods proposed in the literature, there is still an incomplete understanding of this phenomenon. Instead of treating the entire model as vulnerable, we propose that specific feature maps learned during training contribute to the overall vulnerability. To investigate how the hidden representations learned by a CNN affect its vulnerability, we introduce the Adversarial Intervention framework. Experiments were conducted on models trained on three well-known computer vision datasets, subjecting them to attacks of different nature. Our focus centers on the effects that adversarial perturbations to a model's initial layer have on the overall behavior of the model. Empirical results revealed compelling insights: a) perturbing selected channel combinations in shallow layers causes significant disruptions; b) the channel combinations most responsible for the disruptions are common among different types of attacks; c) despite shared vulnerable combinations of channels, different attacks affect hidden representations with varying magnitudes; d) there exists a positive correlation between a kernel's magnitude and its vulnerability. In conclusion, this work introduces a novel framework to study the vulnerability of a CNN model to adversarial perturbations, revealing insights that contribute to a deeper understanding of the phenomenon. The identified properties pave the way for the development of efficient ad-hoc defense mechanisms in future applications.
相關內容
Despite the wide usage of parametric point processes in theory and applications, a sound goodness-of-fit procedure to test whether a given parametric model is appropriate for data coming from a self-exciting point processes has been missing in the literature. In this work, we establish a bootstrap-based goodness-of-fit test which empirically works for all kinds of self-exciting point processes (and even beyond). In an infill-asymptotic setting we also prove its asymptotic consistency, albeit only in the particular case that the underlying point process is inhomogeneous Poisson.
In this work we study the numerical approximation of a class of ergodic Backward Stochastic Differential Equations. These equations are formulated in an infinite horizon framework and provide a probabilistic representation for elliptic Partial Differential Equations of ergodic type. In order to build our numerical scheme, we put forward a new representation of the PDE solution by using a classical probabilistic representation of the gradient. Then, based on this representation, we propose a fully implementable numerical scheme using a Picard iteration procedure, a grid space discretization and a Monte-Carlo approximation. Up to a limiting technical condition that guarantee the contraction of the Picard procedure, we obtain an upper bound for the numerical error. We also provide some numerical experiments that show the efficiency of this approach for small dimensions.
Motivated by the application of saddlepoint approximations to resampling-based statistical tests, we prove that a Lugananni-Rice style approximation for conditional tail probabilities of averages of conditionally independent random variables has vanishing relative error. We also provide a general condition on the existence and uniqueness of the solution to the corresponding saddlepoint equation. The results are valid under a broad class of distributions involving no restrictions on the smoothness of the distribution function. The derived saddlepoint approximation formula can be directly applied to resampling-based hypothesis tests, including bootstrap, sign-flipping and conditional randomization tests. Our results extend and connect several classical saddlepoint approximation results. On the way to proving our main results, we prove a new conditional Berry-Esseen inequality for the sum of conditionally independent random variables, which may be of independent interest.
The homogenization procedure developed here is conducted on a laminate with periodic space-time modulation on the fine scale: at leading order, this modulation creates convection in the low-wavelength regime if both parameters are modulated. However, if only one parameter is modulated, which is more realistic, this convective term disappears and one recovers a standard diffusion equation with effective homogeneous parameters; this does not describe the non-reciprocity and the propagation of the field observed from exact dispersion diagrams. This inconsistency is corrected here by considering second-order homogenization which results in a non-reciprocal propagation term that is proved to be non-zero for any laminate and verified via numerical simulation. The same methodology is also applied to the case when the density is modulated in the heat equation, leading therefore to a corrective advective term which cancels out non-reciprocity at the leading order but not at the second order.
Complex reflection groups comprise a generalization of Weyl groups of semisimple Lie algebras, and even more generally of finite Coxeter groups. They have been heavily studied since their introduction and complete classification in the 1950s by Shephard and Todd, due to their many applications to combinatorics, representation theory, knot theory, and mathematical physics, to name a few examples. For each given complex reflection group G, we explain a new recipe for producing an integrable system of linear differential equations whose differential Galois group is precisely G. We exhibit these systems explicitly for many (low-rank) irreducible complex reflection groups in the Shephard-Todd classification.
A common method for estimating the Hessian operator from random samples on a low-dimensional manifold involves locally fitting a quadratic polynomial. Although widely used, it is unclear if this estimator introduces bias, especially in complex manifolds with boundaries and nonuniform sampling. Rigorous theoretical guarantees of its asymptotic behavior have been lacking. We show that, under mild conditions, this estimator asymptotically converges to the Hessian operator, with nonuniform sampling and curvature effects proving negligible, even near boundaries. Our analysis framework simplifies the intensive computations required for direct analysis.
This article aims to investigate the impact of noise on parameter fitting for an Ornstein-Uhlenbeck process, focusing on the effects of multiplicative and thermal noise on the accuracy of signal separation. To address these issues, we propose algorithms and methods that can effectively distinguish between thermal and multiplicative noise and improve the precision of parameter estimation for optimal data analysis. Specifically, we explore the impact of both multiplicative and thermal noise on the obfuscation of the actual signal and propose methods to resolve them. First, we present an algorithm that can effectively separate thermal noise with comparable performance to Hamilton Monte Carlo (HMC) but with significantly improved speed. We then analyze multiplicative noise and demonstrate that HMC is insufficient for isolating thermal and multiplicative noise. However, we show that, with additional knowledge of the ratio between thermal and multiplicative noise, we can accurately distinguish between the two types of noise when provided with a sufficiently large sampling rate or an amplitude of multiplicative noise smaller than thermal noise. Thus, we demonstrate the mechanism underlying an otherwise counterintuitive phenomenon: when multiplicative noise dominates the noise spectrum, one can successfully estimate the parameters for such systems after adding additional white noise to shift the noise balance.
This work describes the development and validation of a digital twin for a semi-autogenous grinding (SAG) mill controlled by an expert system. The digital twin consists of three modules emulating a closed-loop system: fuzzy logic for the expert control, a state-space model for regulatory control, and a recurrent neural network for the SAG mill process. The model was trained with 68 hours of data and validated with 8 hours of test data. It predicts the mill's behavior within a 2.5-minute horizon with a 30-second sampling time. The disturbance detection evaluates the need for retraining, and the digital twin shows promise for supervising the SAG mill with the expert control system. Future work will focus on integrating this digital twin into real-time optimization strategies with industrial validation.
In this paper, we introduce the finite difference weighted essentially non-oscillatory (WENO) scheme based on the neural network for hyperbolic conservation laws. We employ the supervised learning and design two loss functions, one with the mean squared error and the other with the mean squared logarithmic error, where the WENO3-JS weights are computed as the labels. Each loss function consists of two components where the first component compares the difference between the weights from the neural network and WENO3-JS weights, while the second component matches the output weights of the neural network and the linear weights. The former of the loss function enforces the neural network to follow the WENO properties, implying that there is no need for the post-processing layer. Additionally the latter leads to better performance around discontinuities. As a neural network structure, we choose the shallow neural network (SNN) for computational efficiency with the Delta layer consisting of the normalized undivided differences. These constructed WENO3-SNN schemes show the outperformed results in one-dimensional examples and improved behavior in two-dimensional examples, compared with the simulations from WENO3-JS and WENO3-Z.
The problem of benign overfitting asks whether it is possible for a model to perfectly fit noisy training data and still generalize well. We study benign overfitting in two-layer leaky ReLU networks trained with the hinge loss on a binary classification task. We consider input data that can be decomposed into the sum of a common signal and a random noise component, that lie on subspaces orthogonal to one another. We characterize conditions on the signal to noise ratio (SNR) of the model parameters giving rise to benign versus non-benign (or harmful) overfitting: in particular, if the SNR is high then benign overfitting occurs, conversely if the SNR is low then harmful overfitting occurs. We attribute both benign and non-benign overfitting to an approximate margin maximization property and show that leaky ReLU networks trained on hinge loss with gradient descent (GD) satisfy this property. In contrast to prior work we do not require the training data to be nearly orthogonal. Notably, for input dimension $d$ and training sample size $n$, while results in prior work require $d = \Omega(n^2 \log n)$, here we require only $d = \Omega\left(n\right)$.
注冊地址： 北京市海淀區羊坊店路18號2幢3層301-191