You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/source/markdown/guides/how_to/models/post_processor.md
+114-164
Original file line number
Diff line number
Diff line change
@@ -8,180 +8,132 @@ This guide explains how post-processing works in Anomalib, its integration with
8
8
9
9
## Overview
10
10
11
-
Post-processing in Anomalib is designed to handle model outputs and convert them into meaningful predictions. The post-processor:
11
+
Post-processing in Anomalib refers to any additional operations that are applied after the model generates its raw predictions. Most anomaly detection models do not generate hard classification labels directly. Instead, the models generate an anomaly score, which can be seen as an estimation of the distance from the sample to the learned representation of normality. The raw anomaly scores may consist of a single score per image for anomaly classification, or a pixel-level anomaly map for anomaly localization/segmentation. The raw anomaly scores may be hard to interpret, as they are unbounded, and the range of values may differ between models. To convert the raw anomaly scores into useable predictions, we need to apply a threshold that maps the raw scores to the binary (normal vs. anomalous) classification labels. In addition, we may want to normalize the raw scores to the [0, 1] range for interpretability and visualization.
12
12
13
-
- Computes anomaly scores from raw model outputs
14
-
- Determines optimal thresholds for anomaly detection
15
-
- Generates segmentation masks for pixel-level detection
16
-
- Gets exported with the model for consistent inference
13
+
The thresholding and normalization steps described above are typical post-processing steps in an anomaly detection workflow. The module that is responsible for these operations in Anomalib is the `PostProcessor`. The `PostProcessor` applies a set of post-processing operations on the raw predictions returned by the model. Similar to the {doc}`PreProcessor <./pre_processor>`, the `PostProcessor` also infuses its operations in the model graph during export. This ensures that during deployment:
17
14
18
-
The `PostProcessor` class is an abstract base class that serves two roles:
15
+
- Post-processing is part of the exported model (ONNX, OpenVINO)
16
+
- Users don't need to manually apply post-processing steps such as thresholding and normalization
17
+
- Edge deployment is simplified with automatic post-processing
18
+
19
+
To achieve this, the `PostProcessor` class implements the following components:
19
20
20
21
1. A PyTorch Module for processing model outputs that gets exported with the model
21
22
2. A Lightning Callback for managing thresholds during training
22
23
23
-
Anomalib provides concrete implementations like `OneClassPostProcessor` for specific use cases, such as one-class anomaly detection.
24
-
This is based on the `PostProcessor` class. For any other use case, you can create a custom post-processor by inheriting from the `PostProcessor` class.
24
+
The `PostProcessor` is an abstract base class that can be implemented to suit different post-processing workflows. In addition, Anomalib also provides a default `OneClassPostProcessor` implementation, which suits most one-class learning algorithms. Other learning types, such as zero-shot learning or VLM-based models may require different post-processing steps.
25
25
26
-
## Basic Usage
26
+
## OneClassPostProcessor
27
27
28
-
The most common post-processor is `OneClassPostProcessor`:
28
+
The `OneClassPostProcessor` is Anomalib's default post-processor class which covers the most common anomaly detection workflow. It is responsible for adaptively computing the optimal threshold value for the dataset, applying this threshold during testing/inference, and normalizing the predicted anomaly scores to the [0, 1] range for interpretability. Thresholding and normalization is applied separately for both image- and pixel-level predictions. The following descriptions focus on the image-level predictions, but the same principles apply for the pixel-level predictions.
29
29
30
-
```python
31
-
from anomalib.post_processing import OneClassPostProcessor
30
+
**Thresholding**
32
31
33
-
# Create a post-processor with sensitivity adjustments
The post-processor adaptively computes the optimal threshold value during the validation sequence. The threshold is computed by collecting the raw anomaly scores and the corresponding ground truth labels for all the images in the validation set, and plotting the Precision-Recall (PR) curve for the range of possible threshold values $\mathbf{\theta}$.
The post-processor is automatically integrated into Anomalib models:
40
+
Finally, the optimal threshold value $\theta^*$ is determined as the threshold value that yields the highest the F1-score:
50
41
51
-
```python
52
-
from anomalib.models import Patchcore
53
-
from anomalib.post_processing import OneClassPostProcessor
42
+
$$
43
+
\theta^* = \text{arg}\max_{i} F1_{i}
44
+
$$
54
45
55
-
# Model creates default post-processor (OneClassPostProcessor)
56
-
model = Patchcore()
46
+
During testing and predicting, the post-processor computes the binary classification labels by assigning a positive label (anomalous) to all anomaly scores that are higher than the threshold, and a negative label (normal) to all anomaly scores below the threshold. Given an anomaly score $s_{\text{test},i}$, the binary classifical label $\hat{y}_{\text{test},i}$ is given by:
57
47
58
-
# Or specify custom post-processor
59
-
model = Patchcore(
60
-
post_processor=OneClassPostProcessor(
61
-
image_sensitivity=0.5,
62
-
pixel_sensitivity=0.5
63
-
)
64
-
)
65
-
```
48
+
$$
49
+
\hat{y}_{\text{test},i} =
50
+
\begin{cases}
51
+
1 & \text{if } s_{\text{test},i} \geq \theta^* \\
52
+
0 & \text{if } s_{\text{test},i} < \theta^*
53
+
\end{cases}
54
+
$$
66
55
67
-
## Creating Custom Post-processors
56
+
**Normalization**
68
57
69
-
To create a custom post-processor, inherit from the abstract base class `PostProcessor`:
58
+
During the validation sequence, the post-processor iterates over the raw anomaly score predictions for the validation set, $\mathbf{s}_{\text{val}}$, and keeps track of the lowest and highest observed values, $\min\mathbf{s}_{\text{val}}$ and $\max \mathbf{s}_{\text{val}}$.
70
59
71
-
```python
72
-
from anomalib.post_processing import PostProcessor
During testing and predicting, the post-processor uses the stored min and max values, together with the optimal threshold value, to normalize the values to the [0, 1] range. For a raw anomaly score $s_{\text{test},i}$, the normalized score $\tilde{s}_{\text{test},i}$ is given by:
81
61
82
-
This method must be implemented by all subclasses.
As a last step, the normalized scores are capped between 0 and 1.
89
67
90
-
Here's a simplified version of how `OneClassPostProcessor` is implemented:
91
-
92
-
```python
93
-
from anomalib.post_processing import PostProcessor
94
-
from anomalib.data import InferenceBatch
95
-
from anomalib.metrics import F1AdaptiveThreshold, MinMax
68
+
The $\theta^*$ term in the formula above ensures that the normalized values are centered around the threshold value, such that a value of 0.5 in the normalized domain corresponds to the value of the threshold in the un-normalized domain. This helps with interpretability of the results, as it asserts that normalized values of 0.5 and higher are labeled anomalous, while values below 0.5 are labeled normal.
96
69
97
-
classCustomOneClassPostProcessor(PostProcessor):
98
-
"""Custom one-class post-processor."""
70
+
Centering the threshold value around 0.5 has the additional advantage that it allows us to add a sensitivity parameter $\alpha$ that changes the sensitivity of the anomaly detector. In the normalized domain, the binary classification label is given by:
Where $\alpha$ is a sensitivity parameter that can be varied between 0 and 1, such that a higher sensitivity value lowers the effective anomaly score threshold. The sensitivity parameter can be tuned depending on the use case. For example, use-cases in which false positives should be avoided may benefit from reducing the sensitivity.
Normalization and thresholding only works when your datamodule contains a validation set, preferably cosisting of both normal and anomalous samples. When your validation set only contains normal samples, the threshold will be set to the value of the highest observed anomaly score in your validation set.
138
84
```
139
85
140
-
## Best Practices
141
-
142
-
1.**Score Normalization**:
86
+
## Basic Usage
143
87
144
-
- Normalize scores to [0,1] range
145
-
- Handle numerical stability
146
-
- Consider score distributions
88
+
To use the `OneClassPostProcessor`, simply add it to any Anomalib model when creating the model:
147
89
148
-
2.**Threshold Selection**:
90
+
```python
91
+
from anomalib.models import Padim
92
+
from anomalib.post_processing import OneClassPostProcessor
149
93
150
-
- Use adaptive thresholding when possible
151
-
- Validate thresholds on validation set
152
-
- Consider application requirements
94
+
post_processor = OneClassPostProcessor()
95
+
model = Padim(post_processor=post_processor)
96
+
```
153
97
154
-
3.**Performance**:
98
+
The post-processor can be configured using its constructor arguments. In the case of the `OneClassPostProcessor`, the only configuration parameters are the sensitivity for the thresholding operation on the image- and pixel-level:
155
99
156
-
- Optimize computations for large outputs
157
-
- Handle GPU/CPU transitions efficiently
158
-
- Cache computed thresholds
100
+
```python
101
+
post_processor = OneClassPostProcessor(
102
+
image_sensitivity=0.4,
103
+
pixel_sensitivity=0.4,
104
+
)
105
+
model = Padim(post_processor=post_processor)
106
+
```
159
107
160
-
4.**Validation**:
161
-
- Verify prediction shapes
162
-
- Check threshold computation
163
-
- Test edge cases
108
+
When a post-processor instance is not passed explicitly to the model, the model will automatically configure a default post-processor instance. Let's confirm this by creating a Padim model and printing the `post_processor` attribute:
Each model implementation in Anomalib is required to implement the `configure_post_processor` method, which defines the default post-processor for that model. We can use this method to quickly inspect the default post-processing behaviour of an Anomalib model class:
168
122
169
-
- Not computing thresholds during training
170
-
- Incorrect threshold computation
171
-
- Not handling score distributions
123
+
```python
124
+
print(Padim.configure_post_processor())
125
+
```
172
126
173
-
2.**Normalization Problems**:
127
+
In some cases it may be desirable to disable post-processing entirely. This is done by passing `False` to the model's `post_processor` argument:
174
128
175
-
- Inconsistent normalization
176
-
- Numerical instability
177
-
- Not handling outliers
129
+
```python
130
+
from anomalib.models import Padim
178
131
179
-
3.**Memory Issues**:
180
-
- Large intermediate tensors
181
-
- Unnecessary CPU-GPU transfers
182
-
- Memory leaks in custom implementations
132
+
model = Padim(post_processor=False)
133
+
print(model.post_processor isNone) # True
134
+
```
183
135
184
-
##Edge Deployment
136
+
### Exporting
185
137
186
138
One key advantage of Anomalib's post-processor design is that it becomes part of the model graph during export. This means:
Advanced users may want to define their own post-processing pipeline. This can be useful when the default post-processing behaviour of the `OneClassPostProcessor` is not suitable for the model and its predictions. To create a custom post-processor, inherit from the abstract base class `PostProcessor`:
236
188
237
-
```python
238
-
# Before: Manual post-processing needed
239
-
core = Core()
240
-
model = core.read_model("model_without_postprocessing.xml")
241
-
compiled_model = core.compile_model(model)
242
-
raw_outputs = compiled_model([image])[output_key]
243
-
normalized = normalize_scores(raw_outputs)
244
-
predictions = apply_threshold(normalized)
189
+
```python
190
+
from anomalib.post_processing import PostProcessor
0 commit comments