Provably Robust Conformal Prediction with Improved Efficiency
2404.19651
0
0
🔮
Abstract
Conformal prediction is a powerful tool to generate uncertainty sets with guaranteed coverage using any predictive model, under the assumption that the training and test data are i.i.d.. Recently, it has been shown that adversarial examples are able to manipulate conformal methods to construct prediction sets with invalid coverage rates, as the i.i.d. assumption is violated. To address this issue, a recent work, Randomized Smoothed Conformal Prediction (RSCP), was first proposed to certify the robustness of conformal prediction methods to adversarial noise. However, RSCP has two major limitations: (i) its robustness guarantee is flawed when used in practice and (ii) it tends to produce large uncertainty sets. To address these limitations, we first propose a novel framework called RSCP+ to provide provable robustness guarantee in evaluation, which fixes the issues in the original RSCP method. Next, we propose two novel methods, Post-Training Transformation (PTT) and Robust Conformal Training (RCT), to effectively reduce prediction set size with little computation overhead. Experimental results in CIFAR10, CIFAR100, and ImageNet suggest the baseline method only yields trivial predictions including full label set, while our methods could boost the efficiency by up to $4.36times$, $5.46times$, and $16.9times$ respectively and provide practical robustness guarantee. Our codes are available at https://github.com/Trustworthy-ML-Lab/Provably-Robust-Conformal-Prediction.
Create account to get full access
Overview
- Conformal prediction is a powerful tool for generating uncertainty sets with guaranteed coverage using any predictive model, assuming the training and test data are independent and identically distributed (i.i.d.).
- However, recent research has shown that adversarial examples can manipulate conformal methods to construct prediction sets with invalid coverage rates, as the i.i.d. assumption is violated.
- To address this issue, a previous work called Randomized Smoothed Conformal Prediction (RSCP) was proposed to certify the robustness of conformal prediction methods to adversarial noise.
- RSCP has two major limitations: (i) its robustness guarantee is flawed when used in practice, and (ii) it tends to produce large uncertainty sets.
- This paper proposes a novel framework called RSCP+ to provide provable robustness guarantees in evaluation, fixing the issues in the original RSCP method.
- The paper also introduces two novel methods, Post-Training Transformation (PTT) and Robust Conformal Training (RCT), to effectively reduce prediction set size with little computational overhead.
Plain English Explanation
Conformal prediction is a technique that can create uncertainty sets with guaranteed coverage using any machine learning model, as long as the training and test data are similar. However, researchers recently discovered that adversarial examples (carefully crafted inputs that can fool models) can break this guarantee, making the uncertainty sets unreliable.
To address this issue, a previous method called Randomized Smoothed Conformal Prediction (RSCP) was proposed. RSCP tries to make conformal prediction robust to adversarial attacks. But RSCP has two major problems: 1) its robustness guarantee doesn't actually work well in practice, and 2) it tends to produce very large uncertainty sets, which are not very useful.
This new paper introduces a few solutions. First, they propose a new framework called RSCP+ that fixes the issues in the original RSCP method and provides a reliable robustness guarantee. Second, they introduce two new techniques, Post-Training Transformation (PTT) and Robust Conformal Training (RCT), that can significantly reduce the size of the uncertainty sets without much extra computation.
The key ideas are to modify the conformal prediction process to make it more robust to adversarial attacks, while also making the resulting uncertainty sets as small and informative as possible. This allows conformal prediction to maintain its reliable coverage guarantees even when the data distribution changes due to adversarial examples.
Technical Explanation
The paper first discusses the limitations of the original Randomized Smoothed Conformal Prediction (RSCP) method. RSCP aimed to make conformal prediction robust to adversarial attacks by adding random noise to the inputs. However, the authors show that RSCP's robustness guarantee is flawed when used in practice, and it tends to produce very large prediction sets.
To address these issues, the paper proposes a new framework called RSCP+. RSCP+ modifies the RSCP procedure to provide a provable robustness guarantee in evaluation, fixing the problems in the original RSCP.
Next, the paper introduces two novel methods to reduce the size of the prediction sets while maintaining robustness:
-
Post-Training Transformation (PTT): This technique applies a learned transformation to the model's outputs after training, which can shrink the prediction sets without affecting the robustness.
-
Robust Conformal Training (RCT): This method modifies the training process of the underlying predictive model to encourage robustness, leading to more efficient conformal predictions.
The authors evaluate their methods on CIFAR-10, CIFAR-100, and ImageNet datasets. They find that the baseline RSCP method only produces trivial predictions (covering the entire label set), while their RSCP+ with PTT or RCT can significantly improve the efficiency of the predictions, up to 16.9 times, while still providing practical robustness guarantees.
Critical Analysis
The paper presents a solid technical contribution in addressing the limitations of the original RSCP method and proposing novel techniques to improve the efficiency of robust conformal prediction.
One potential concern is the reliance on the i.i.d. assumption, which may not always hold in real-world scenarios. The authors acknowledge this limitation and note that relaxing the i.i.d. assumption is an important direction for future research, as discussed in recent works on conformal prediction and metric-guided image reconstruction.
Additionally, the paper focuses on image classification tasks, and it would be valuable to explore the performance of the proposed methods on other types of machine learning tasks and domains.
Overall, this paper makes a significant contribution to the field of conformal prediction by enhancing its robustness and efficiency, which is an important step towards making these reliable prediction methods more practical and widely applicable.
Conclusion
This paper addresses key limitations of the Randomized Smoothed Conformal Prediction (RSCP) method, which was designed to make conformal prediction robust to adversarial attacks. The authors propose a new framework called RSCP+ that fixes the issues in the original RSCP, and introduce two novel techniques, Post-Training Transformation (PTT) and Robust Conformal Training (RCT), to significantly improve the efficiency of the robust conformal predictions.
Experimental results on popular image classification datasets demonstrate the effectiveness of the proposed methods, with up to 16.9 times improvement in prediction set size compared to the baseline RSCP approach, while still providing practical robustness guarantees. This work represents an important advancement in making conformal prediction more robust and practical for real-world applications.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
Verifiably Robust Conformal Prediction
Linus Jeary, Tom Kuipers, Mehran Hosseini, Nicola Paoletti
0
0
Conformal Prediction (CP) is a popular uncertainty quantification method that provides distribution-free, statistically valid prediction sets, assuming that training and test data are exchangeable. In such a case, CP's prediction sets are guaranteed to cover the (unknown) true test output with a user-specified probability. Nevertheless, this guarantee is violated when the data is subjected to adversarial attacks, which often result in a significant loss of coverage. Recently, several approaches have been put forward to recover CP guarantees in this setting. These approaches leverage variations of randomised smoothing to produce conservative sets which account for the effect of the adversarial perturbations. They are, however, limited in that they only support $ell^2$-bounded perturbations and classification tasks. This paper introduces VRCP (Verifiably Robust Conformal Prediction), a new framework that leverages recent neural network verification methods to recover coverage guarantees under adversarial attacks. Our VRCP method is the first to support perturbations bounded by arbitrary norms including $ell^1$, $ell^2$, and $ell^infty$, as well as regression tasks. We evaluate and compare our approach on image classification tasks (CIFAR10, CIFAR100, and TinyImageNet) and regression tasks for deep reinforcement learning environments. In every case, VRCP achieves above nominal coverage and yields significantly more efficient and informative prediction regions than the SotA.
6/7/2024
A Conformal Prediction Score that is Robust to Label Noise
Coby Penso, Jacob Goldberger
0
0
Conformal Prediction (CP) quantifies network uncertainty by building a small prediction set with a pre-defined probability that the correct class is within this set. In this study we tackle the problem of CP calibration based on a validation set with noisy labels. We introduce a conformal score that is robust to label noise. The noise-free conformal score is estimated using the noisy labeled data and the noise level. In the test phase the noise-free score is used to form the prediction set. We applied the proposed algorithm to several standard medical imaging classification datasets. We show that our method outperforms current methods by a large margin, in terms of the average size of the prediction set, while maintaining the required coverage.
5/22/2024
Robust Conformal Prediction Using Privileged Information
Shai Feldman, Yaniv Romano
0
0
We develop a method to generate prediction sets with a guaranteed coverage rate that is robust to corruptions in the training data, such as missing or noisy variables. Our approach builds on conformal prediction, a powerful framework to construct prediction sets that are valid under the i.i.d assumption. Importantly, naively applying conformal prediction does not provide reliable predictions in this setting, due to the distribution shift induced by the corruptions. To account for the distribution shift, we assume access to privileged information (PI). The PI is formulated as additional features that explain the distribution shift, however, they are only available during training and absent at test time. We approach this problem by introducing a novel generalization of weighted conformal prediction and support our method with theoretical coverage guarantees. Empirical experiments on both real and synthetic datasets indicate that our approach achieves a valid coverage rate and constructs more informative predictions compared to existing methods, which are not supported by theoretical guarantees.
6/11/2024
🔮
Conformal Prediction for Class-wise Coverage via Augmented Label Rank Calibration
Yuanjie Shi, Subhankar Ghosh, Taha Belkhouja, Janardhan Rao Doppa, Yan Yan
0
0
Conformal prediction (CP) is an emerging uncertainty quantification framework that allows us to construct a prediction set to cover the true label with a pre-specified marginal or conditional probability. Although the valid coverage guarantee has been extensively studied for classification problems, CP often produces large prediction sets which may not be practically useful. This issue is exacerbated for the setting of class-conditional coverage on imbalanced classification tasks. This paper proposes the Rank Calibrated Class-conditional CP (RC3P) algorithm to reduce the prediction set sizes to achieve class-conditional coverage, where the valid coverage holds for each class. In contrast to the standard class-conditional CP (CCP) method that uniformly thresholds the class-wise conformity score for each class, the augmented label rank calibration step allows RC3P to selectively iterate this class-wise thresholding subroutine only for a subset of classes whose class-wise top-k error is small. We prove that agnostic to the classifier and data distribution, RC3P achieves class-wise coverage. We also show that RC3P reduces the size of prediction sets compared to the CCP method. Comprehensive experiments on multiple real-world datasets demonstrate that RC3P achieves class-wise coverage and 26.25% reduction in prediction set sizes on average.
6/12/2024