GLAD: Towards Better Reconstruction with Global and Local Adaptive Diffusion Models for Unsupervised Anomaly Detection
2406.07487
0
0
Abstract
Diffusion models have shown superior performance on unsupervised anomaly detection tasks. Since trained with normal data only, diffusion models tend to reconstruct normal counterparts of test images with certain noises added. However, these methods treat all potential anomalies equally, which may cause two main problems. From the global perspective, the difficulty of reconstructing images with different anomalies is uneven. Therefore, instead of utilizing the same setting for all samples, we propose to predict a particular denoising step for each sample by evaluating the difference between image contents and the priors extracted from diffusion models. From the local perspective, reconstructing abnormal regions differs from normal areas even in the same image. Theoretically, the diffusion model predicts a noise for each step, typically following a standard Gaussian distribution. However, due to the difference between the anomaly and its potential normal counterpart, the predicted noise in abnormal regions will inevitably deviate from the standard Gaussian distribution. To this end, we propose introducing synthetic abnormal samples in training to encourage the diffusion models to break through the limitation of standard Gaussian distribution, and a spatial-adaptive feature fusion scheme is utilized during inference. With the above modifications, we propose a global and local adaptive diffusion model (abbreviated to GLAD) for unsupervised anomaly detection, which introduces appealing flexibility and achieves anomaly-free reconstruction while retaining as much normal information as possible. Extensive experiments are conducted on three commonly used anomaly detection datasets (MVTec-AD, MPDD, and VisA) and a printed circuit board dataset (PCB-Bank) we integrated, showing the effectiveness of the proposed method.
Create account to get full access
Overview
- This paper proposes a new unsupervised anomaly detection method called GLAD (Global and Local Adaptive Diffusion) that leverages diffusion models to better reconstruct and identify anomalies in data.
- GLAD combines a global diffusion model to capture the overall data distribution with local adaptive diffusion models that can better handle anomalous regions.
- The authors demonstrate GLAD's effectiveness on various real-world datasets, showing it outperforms existing state-of-the-art anomaly detection techniques.
Plain English Explanation
Anomaly detection is the process of identifying data points or patterns that deviate significantly from the normal or expected behavior in a dataset. This is an important task with applications in areas like fraud detection, medical diagnosis, and sensor monitoring.
The GLAD method proposed in this paper aims to improve anomaly detection by combining two key components:
- Global Diffusion Model: This component learns a general model of the entire dataset, capturing the overall data distribution.
- Local Adaptive Diffusion Models: These models focus on specific regions of the data, adapting to handle anomalous patterns that may be missed by the global model.
The key idea is that the combination of these global and local models can better reconstruct normal data and identify anomalies compared to using a single global model alone. This is because the local models can adapt to handle peculiarities in different parts of the dataset that the global model may struggle with.
The authors demonstrate GLAD's effectiveness on a variety of real-world datasets, showing that it outperforms other state-of-the-art anomaly detection techniques. This suggests GLAD could be a valuable tool for researchers and practitioners working on anomaly detection problems.
Technical Explanation
The GLAD method builds upon the success of diffusion models, which have shown promising results in tasks like image generation and anomaly detection. Diffusion models work by gradually adding noise to data and then learning to reverse this process to generate new samples.
In GLAD, the authors combine a global diffusion model that captures the overall data distribution with local adaptive diffusion models that focus on specific regions of the data. The local models are trained to better handle anomalous patterns that may be overlooked by the global model.
During the anomaly detection process, GLAD first uses the global model to reconstruct the input data. It then compares the reconstruction to the original input and identifies regions where the reconstruction differs significantly. These regions are then passed to the local adaptive models, which refine the reconstruction and identify anomalies more precisely.
The authors evaluate GLAD on a range of real-world datasets, including images, tabular data, and time series. They show that GLAD outperforms existing state-of-the-art anomaly detection methods, such as Unsupervised Anomaly Detection via Masked Diffusion Posterior, GLADFORMER: Mixed Perspective Graph-Level Anomaly Detection, and GDA: Generalized Diffusion for Robust Test-Time Adaptation.
Critical Analysis
The GLAD method represents a promising advance in unsupervised anomaly detection, leveraging the power of diffusion models to better capture the complexities of real-world data. However, the authors acknowledge several limitations and potential areas for future research:
- Computational Complexity: Training and inference with GLAD can be computationally expensive, especially with large or high-dimensional datasets. The authors suggest exploring ways to improve the efficiency of the method.
- Interpretability: Like many deep learning-based approaches, GLAD can be treated as a "black box," making it difficult to understand the underlying reasons for its decisions. Developing more interpretable anomaly detection methods could be valuable.
- Generalization: While GLAD demonstrates strong performance on the evaluated datasets, its ability to generalize to a wider range of anomaly detection problems is not fully established. Further testing on diverse datasets would be beneficial.
Additionally, researchers may want to consider the following potential issues:
- Sensitivity to Hyperparameters: The performance of GLAD may be sensitive to the choice of hyperparameters, such as the number of local models and the specific architectural choices. Careful tuning and robustness testing would be important.
- Handling of Different Anomaly Types: Anomalies can come in various forms, such as point anomalies, contextual anomalies, and collective anomalies. It's unclear how well GLAD would handle these different types of anomalies compared to other methods.
Overall, GLAD is a promising approach that leverages the strengths of diffusion models for improved unsupervised anomaly detection. However, as with any research, further exploration and validation of the method's capabilities and limitations would be valuable for advancing the field of anomaly detection.
Conclusion
The GLAD method proposed in this paper represents a significant contribution to the field of unsupervised anomaly detection. By combining a global diffusion model with locally adaptive diffusion models, GLAD can better reconstruct normal data and identify anomalies compared to existing techniques.
The authors' experimental results demonstrate GLAD's effectiveness on a range of real-world datasets, suggesting it could be a valuable tool for researchers and practitioners working on anomaly detection problems. While the method has some limitations in terms of computational complexity and interpretability, the core idea of leveraging both global and local models is an intriguing approach that could inspire further advancements in this important area of research.
As the field of anomaly detection continues to evolve, methods like GLAD that can adaptively handle the complexities of real-world data will likely play an increasingly important role in applications spanning fraud detection, medical diagnosis, sensor monitoring, and beyond.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
Dynamic Addition of Noise in a Diffusion Model for Anomaly Detection
Justin Tebbe, Jawad Tayyub
0
0
Diffusion models have found valuable applications in anomaly detection by capturing the nominal data distribution and identifying anomalies via reconstruction. Despite their merits, they struggle to localize anomalies of varying scales, especially larger anomalies such as entire missing components. Addressing this, we present a novel framework that enhances the capability of diffusion models, by extending the previous introduced implicit conditioning approach Meng et al. (2022) in three significant ways. First, we incorporate a dynamic step size computation that allows for variable noising steps in the forward process guided by an initial anomaly prediction. Second, we demonstrate that denoising an only scaled input, without any added noise, outperforms conventional denoising process. Third, we project images in a latent space to abstract away from fine details that interfere with reconstruction of large missing components. Additionally, we propose a fine-tuning mechanism that facilitates the model to effectively grasp the nuances of the target domain. Our method undergoes rigorous evaluation on prominent anomaly detection datasets VisA, BTAD and MVTec yielding strong performance. Importantly, our framework effectively localizes anomalies regardless of their scale, marking a pivotal advancement in diffusion-based anomaly detection.
6/26/2024
Unsupervised Anomaly Detection via Masked Diffusion Posterior Sampling
Di Wu, Shicai Fan, Xue Zhou, Li Yu, Yuzhong Deng, Jianxiao Zou, Baihong Lin
0
0
Reconstruction-based methods have been commonly used for unsupervised anomaly detection, in which a normal image is reconstructed and compared with the given test image to detect and locate anomalies. Recently, diffusion models have shown promising applications for anomaly detection due to their powerful generative ability. However, these models lack strict mathematical support for normal image reconstruction and unexpectedly suffer from low reconstruction quality. To address these issues, this paper proposes a novel and highly-interpretable method named Masked Diffusion Posterior Sampling (MDPS). In MDPS, the problem of normal image reconstruction is mathematically modeled as multiple diffusion posterior sampling for normal images based on the devised masked noisy observation model and the diffusion-based normal image prior under Bayesian framework. Using a metric designed from pixel-level and perceptual-level perspectives, MDPS can effectively compute the difference map between each normal posterior sample and the given test image. Anomaly scores are obtained by averaging all difference maps for multiple posterior samples. Exhaustive experiments on MVTec and BTAD datasets demonstrate that MDPS can achieve state-of-the-art performance in normal image reconstruction quality as well as anomaly detection and localization.
4/30/2024
GLADformer: A Mixed Perspective for Graph-level Anomaly Detection
Fan Xu, Nan Wang, Hao Wu, Xuezhi Wen, Dalin Zhang, Siyang Lu, Binyong Li, Wei Gong, Hai Wan, Xibin Zhao
0
0
Graph-Level Anomaly Detection (GLAD) aims to distinguish anomalous graphs within a graph dataset. However, current methods are constrained by their receptive fields, struggling to learn global features within the graphs. Moreover, most contemporary methods are based on spatial domain and lack exploration of spectral characteristics. In this paper, we propose a multi-perspective hybrid graph-level anomaly detector namely GLADformer, consisting of two key modules. Specifically, we first design a Graph Transformer module with global spectrum enhancement, which ensures balanced and resilient parameter distributions by fusing global features and spectral distribution characteristics. Furthermore, to uncover local anomalous attributes, we customize a band-pass spectral GNN message passing module that further enhances the model's generalization capability. Through comprehensive experiments on ten real-world datasets from multiple domains, we validate the effectiveness and robustness of GLADformer. This demonstrates that GLADformer outperforms current state-of-the-art models in graph-level anomaly detection, particularly in effectively capturing global anomaly representations and spectral characteristics.
6/4/2024
GDA: Generalized Diffusion for Robust Test-time Adaptation
Yun-Yun Tsai, Fu-Chen Chen, Albert Y. C. Chen, Junfeng Yang, Che-Chun Su, Min Sun, Cheng-Hao Kuo
0
0
Machine learning models struggle with generalization when encountering out-of-distribution (OOD) samples with unexpected distribution shifts. For vision tasks, recent studies have shown that test-time adaptation employing diffusion models can achieve state-of-the-art accuracy improvements on OOD samples by generating new samples that align with the model's domain without the need to modify the model's weights. Unfortunately, those studies have primarily focused on pixel-level corruptions, thereby lacking the generalization to adapt to a broader range of OOD types. We introduce Generalized Diffusion Adaptation (GDA), a novel diffusion-based test-time adaptation method robust against diverse OOD types. Specifically, GDA iteratively guides the diffusion by applying a marginal entropy loss derived from the model, in conjunction with style and content preservation losses during the reverse sampling process. In other words, GDA considers the model's output behavior with the semantic information of the samples as a whole, which can reduce ambiguity in downstream tasks during the generation process. Evaluation across various popular model architectures and OOD benchmarks shows that GDA consistently outperforms prior work on diffusion-driven adaptation. Notably, it achieves the highest classification accuracy improvements, ranging from 4.4% to 5.02% on ImageNet-C and 2.5% to 7.4% on Rendition, Sketch, and Stylized benchmarks. This performance highlights GDA's generalization to a broader range of OOD benchmarks.
4/3/2024