Stable Conformal Prediction Sets

Eugene Ndiaye

**Concept Graph & Resume using Claude 3.5 Sonnet | Chat GPT4o | Llama 3:**

graph LR
classDef conformal fill:#f9d4d4, font-weight:bold, font-size:14px
classDef stability fill:#d4f9d4, font-weight:bold, font-size:14px
classDef methods fill:#d4d4f9, font-weight:bold, font-size:14px
classDef experiments fill:#f9f9d4, font-weight:bold, font-size:14px
A[Stable Conformal Prediction

Sets] --> B[Conformal

Prediction] A --> C[Stability

and

Bounds] A --> D[Methods

and

Techniques] A --> E[Experiments

and

Results] B --> B1[Constructs prediction

intervals with

coverage. 1] B --> B2[Data distribution

invariant under

permutations. 2] B --> B3[Guaranteed coverage

without distribution

assumptions. 3] B --> B4[Infeasible exact

conformal

sets. 4] B --> B5[Combines conformal

prediction and

stability. 6] B --> B6[Requires only

one model

fitting. 7] C --> C1[Prediction changes with

data

perturbations. 5] C --> C2[Small data changes,

small prediction

changes. 10] C --> C3[Regularity condition

for stability

bounds. 20] C --> C4[Optimization theory

concept for

stability. 21] C --> C5[Challenge of estimating

bounds for

models. 30] C --> C6[Preserves conformal

prediction coverage

guarantees. 9] D --> D1[Uses all data

for

training. 8] D --> D2[Measures prediction

non-conformity. 11] D --> D3[Quantifies prediction

fit with

data. 12] D --> D4[Assumes knowledge of

target

variable. 14] D --> D5[Approximates predictions

between known

points. 17] D --> D6[Tighter conformal set

approximations. 18] E --> E1[Iterative optimization

method. 22] E --> E2[Neural network

in

experiments. 23] E --> E3[Ensemble learning

method in

experiments. 24] E --> E4[Real-world data

experiments. 28] E --> E5[Controlled experiments

with artificial

data. 29] E --> E6[Execution time relative

to

baseline. 27] class A,B,B1,B2,B3,B4,B5,B6 conformal class C,C1,C2,C3,C4,C5,C6 stability class D,D1,D2,D3,D4,D5,D6 methods class E,E1,E2,E3,E4,E5,E6 experiments

Sets] --> B[Conformal

Prediction] A --> C[Stability

and

Bounds] A --> D[Methods

and

Techniques] A --> E[Experiments

and

Results] B --> B1[Constructs prediction

intervals with

coverage. 1] B --> B2[Data distribution

invariant under

permutations. 2] B --> B3[Guaranteed coverage

without distribution

assumptions. 3] B --> B4[Infeasible exact

conformal

sets. 4] B --> B5[Combines conformal

prediction and

stability. 6] B --> B6[Requires only

one model

fitting. 7] C --> C1[Prediction changes with

data

perturbations. 5] C --> C2[Small data changes,

small prediction

changes. 10] C --> C3[Regularity condition

for stability

bounds. 20] C --> C4[Optimization theory

concept for

stability. 21] C --> C5[Challenge of estimating

bounds for

models. 30] C --> C6[Preserves conformal

prediction coverage

guarantees. 9] D --> D1[Uses all data

for

training. 8] D --> D2[Measures prediction

non-conformity. 11] D --> D3[Quantifies prediction

fit with

data. 12] D --> D4[Assumes knowledge of

target

variable. 14] D --> D5[Approximates predictions

between known

points. 17] D --> D6[Tighter conformal set

approximations. 18] E --> E1[Iterative optimization

method. 22] E --> E2[Neural network

in

experiments. 23] E --> E3[Ensemble learning

method in

experiments. 24] E --> E4[Real-world data

experiments. 28] E --> E5[Controlled experiments

with artificial

data. 29] E --> E6[Execution time relative

to

baseline. 27] class A,B,B1,B2,B3,B4,B5,B6 conformal class C,C1,C2,C3,C4,C5,C6 stability class D,D1,D2,D3,D4,D5,D6 methods class E,E1,E2,E3,E4,E5,E6 experiments

**Resume: **

**1.-** Conformal Prediction: A method for constructing prediction intervals with guaranteed coverage by assuming data exchangeability.

**2.-** Exchangeability: Assumption that the joint distribution of data points is invariant under permutations.

**3.-** Coverage Guarantee: Conformal prediction sets have guaranteed coverage for finite sample sizes without distributional assumptions.

**4.-** Computational Challenge: Computing exact conformal sets is often infeasible, especially for continuous outputs.

**5.-** Stability Bounds: Bounds on how much a model's predictions change when input data is slightly perturbed.

**6.-** Stable Conformal Prediction (stabCP): Proposed method combining conformal prediction with stability bounds.

**7.-** Single Model Fit: stabCP requires fitting the model only once, unlike other methods requiring multiple fits.

**8.-** No Data Splitting: stabCP uses all data for training, unlike split conformal methods.

**9.-** Maintained Coverage: stabCP preserves the coverage guarantees of standard conformal prediction.

**10.-** Algorithmic Stability: Assumption that small changes in input data lead to small changes in model predictions.

**11.-** Score Function: Measures the non-conformity of a prediction, used to construct conformal sets.

**12.-** Conformity Measure: Quantifies how well a candidate prediction fits with observed data.

**13.-** Typicalness: Measure of how typical a candidate prediction is compared to observed data.

**14.-** Oracle Prediction Set: Reference set assuming knowledge of the unknown target variable.

**15.-** Split Conformal Prediction: Method using data splitting to separate model fitting and calibration steps.

**16.-** Root-Finding Approach: Method for computing conformal sets by approximating roots of the conformity function.

**17.-** Interpolation: Technique for approximating the model's predictions between known points.

**18.-** Batch Approximation: Using multiple candidate points to obtain tighter approximations of conformal sets.

**19.-** Convex Optimization: Class of problems for which stability bounds are often easier to derive.

**20.-** Lipschitz Continuity: Regularity condition on functions, often assumed for deriving stability bounds.

**21.-** Duality: Concept from optimization theory used to derive some stability bounds.

**22.-** Stochastic Gradient Descent: Iterative optimization method often used in machine learning.

**23.-** Multi-Layer Perceptron: Type of neural network used in experiments.

**24.-** Gradient Boosting: Ensemble learning method used in experiments.

**25.-** Empirical Coverage: Percentage of times a prediction set contains the true value in experiments.

**26.-** Confidence Interval Length: Measure of the precision of a prediction set.

**27.-** Computational Efficiency: Measured by execution time relative to a baseline method.

**28.-** Real Datasets: Experiments conducted on various real-world datasets to evaluate method performance.

**29.-** Synthetic Datasets: Artificially generated data used for controlled experiments.

**30.-** Stability Bound Estimation: Challenge of accurately estimating stability bounds for complex models.

Knowledge Vault built byDavid Vivancos 2024