The End Of Knowledge - Vault 6/77 - CVPR - 2022 - Stable Conformal Prediction Sets

graph LR classDef conformal fill:#f9d4d4, font-weight:bold, font-size:14px classDef stability fill:#d4f9d4, font-weight:bold, font-size:14px classDef methods fill:#d4d4f9, font-weight:bold, font-size:14px classDef experiments fill:#f9f9d4, font-weight:bold, font-size:14px A[Stable Conformal Prediction
Sets] --> B[Conformal
Prediction] A --> C[Stability
and
Bounds] A --> D[Methods
and
Techniques] A --> E[Experiments
and
Results] B --> B1[Constructs prediction
intervals with
coverage. 1] B --> B2[Data distribution
invariant under
permutations. 2] B --> B3[Guaranteed coverage
without distribution
assumptions. 3] B --> B4[Infeasible exact
conformal
sets. 4] B --> B5[Combines conformal
prediction and
stability. 6] B --> B6[Requires only
one model
fitting. 7] C --> C1[Prediction changes with
data
perturbations. 5] C --> C2[Small data changes,
small prediction
changes. 10] C --> C3[Regularity condition
for stability
bounds. 20] C --> C4[Optimization theory
concept for
stability. 21] C --> C5[Challenge of estimating
bounds for
models. 30] C --> C6[Preserves conformal
prediction coverage
guarantees. 9] D --> D1[Uses all data
for
training. 8] D --> D2[Measures prediction
non-conformity. 11] D --> D3[Quantifies prediction
fit with
data. 12] D --> D4[Assumes knowledge of
target
variable. 14] D --> D5[Approximates predictions
between known
points. 17] D --> D6[Tighter conformal set
approximations. 18] E --> E1[Iterative optimization
method. 22] E --> E2[Neural network
in
experiments. 23] E --> E3[Ensemble learning
method in
experiments. 24] E --> E4[Real-world data
experiments. 28] E --> E5[Controlled experiments
with artificial
data. 29] E --> E6[Execution time relative
to
baseline. 27] class A,B,B1,B2,B3,B4,B5,B6 conformal class C,C1,C2,C3,C4,C5,C6 stability class D,D1,D2,D3,D4,D5,D6 methods class E,E1,E2,E3,E4,E5,E6 experiments

Resume:

1.- Conformal Prediction: A method for constructing prediction intervals with guaranteed coverage by assuming data exchangeability.

2.- Exchangeability: Assumption that the joint distribution of data points is invariant under permutations.

3.- Coverage Guarantee: Conformal prediction sets have guaranteed coverage for finite sample sizes without distributional assumptions.

4.- Computational Challenge: Computing exact conformal sets is often infeasible, especially for continuous outputs.

5.- Stability Bounds: Bounds on how much a model's predictions change when input data is slightly perturbed.

6.- Stable Conformal Prediction (stabCP): Proposed method combining conformal prediction with stability bounds.

7.- Single Model Fit: stabCP requires fitting the model only once, unlike other methods requiring multiple fits.

8.- No Data Splitting: stabCP uses all data for training, unlike split conformal methods.

9.- Maintained Coverage: stabCP preserves the coverage guarantees of standard conformal prediction.

10.- Algorithmic Stability: Assumption that small changes in input data lead to small changes in model predictions.

11.- Score Function: Measures the non-conformity of a prediction, used to construct conformal sets.

12.- Conformity Measure: Quantifies how well a candidate prediction fits with observed data.

13.- Typicalness: Measure of how typical a candidate prediction is compared to observed data.

14.- Oracle Prediction Set: Reference set assuming knowledge of the unknown target variable.

15.- Split Conformal Prediction: Method using data splitting to separate model fitting and calibration steps.

16.- Root-Finding Approach: Method for computing conformal sets by approximating roots of the conformity function.

17.- Interpolation: Technique for approximating the model's predictions between known points.

18.- Batch Approximation: Using multiple candidate points to obtain tighter approximations of conformal sets.

19.- Convex Optimization: Class of problems for which stability bounds are often easier to derive.

20.- Lipschitz Continuity: Regularity condition on functions, often assumed for deriving stability bounds.

21.- Duality: Concept from optimization theory used to derive some stability bounds.

22.- Stochastic Gradient Descent: Iterative optimization method often used in machine learning.

23.- Multi-Layer Perceptron: Type of neural network used in experiments.

24.- Gradient Boosting: Ensemble learning method used in experiments.

25.- Empirical Coverage: Percentage of times a prediction set contains the true value in experiments.

26.- Confidence Interval Length: Measure of the precision of a prediction set.

27.- Computational Efficiency: Measured by execution time relative to a baseline method.

28.- Real Datasets: Experiments conducted on various real-world datasets to evaluate method performance.

29.- Synthetic Datasets: Artificially generated data used for controlled experiments.

30.- Stability Bound Estimation: Challenge of accurately estimating stability bounds for complex models.

Knowledge Vault built byDavid Vivancos 2024