Task-agnostic Out-of-Distribution Detection Using Kernel Density Estimation

Abstract

In the recent years, researchers proposed a number of successful methods to perform out-of-distribution (OOD) detection in deep neural networks (DNNs). So far the scope of the highly accurate methods has been limited to image level classification tasks. However, attempts for generally applicable methods beyond classification did not attain similar performance. In this paper, we address this limitation by proposing a simple yet effective task-agnostic OOD detection method. We estimate the probability density functions (pdfs) of intermediate features of a pre-trained DNN by performing kernel density estimation (KDE) on the training dataset. As direct application of KDE to feature maps is hindered by their high dimensionality, we use a set of lower-dimensional marginalized KDE models instead of a single high-dimensional one. At test time, we evaluate the pdfs on a test sample and produce a confidence score that indicates the sample is OOD. The use of KDE eliminates the need for making simplifying assumptions about the underlying feature pdfs and makes the proposed method task-agnostic. We perform extensive experiments on classification tasks using benchmark datasets for OOD detection. Additionally, we perform experiments on medical image segmentation tasks using brain MRI datasets. The results demonstrate that the proposed method consistently achieves high OOD detection performance in both classification and segmentation tasks and improves state-of-the-art in almost all cases. Code is available at https://github.com/eerdil/task_agnostic_ood

1 Introduction

Deep neural networks (DNNs) can perform predictions on test images with very high accuracy when the training and testing data come from the same distribution. However, the prediction accuracy decreases rapidly when the test image is sampled from a different distribution than the training one [16, 36]. Furthermore, in such cases, DNNs can make erroneous predictions with very high confidence [11]. This creates a major obstacle when deploying DNNs for real applications, especially for the ones with a low tolerance for error, such as autonomous driving and medical diagnosis. Therefore, it is crucial to improve the robustness of DNN-based methods and prevent them from making big mistakes [1].

Input	Prediction	Ground Truth

Figure 1: A visual example demonstrating the importance of OOD detection on a segmentation task. A network trained on T1w images works well on a T1w test image (first row) while it produces a poor segmentation on the T2w image of the same patient (second row).

Recently, to improve the robustness of DNNs,substantial advances have been made for OOD detection in DNNs trained for image level classification tasks [12, 23, 25, 15]. Although, OOD detection is equally crucial for non-classification tasks (e.g. segmentation), so far, attempts for developing more generic OOD detection methods did not attain similar performance [26]. In Figure 1, we show a visual example in a medical image segmentation task to emphasize the importance of OOD detection in a non-classification task. The example demonstrates that a DNN trained on T1w brain images (in-distribution (InD)) produces poor segmentation results for T2w images (OOD) of the same patient. Let us imagine an automated clinical decision system that makes the diagnosis based on the segmentations. In that case, the poor segmentation of the OOD T2w image could lead to a wrong diagnosis which can have severe consequences. An OOD detection method within this pipeline can play a key role in preventing such mistakes.

1.1 Related Work

Methods that use predicted class probabilities: Hendrycks et al. [12] proposed a baseline OOD detection method that uses maximum predicted class probability in a classification DNN as a confidence score that the sample is OOD. ODIN [24] extends the baseline by applying an adversarial perturbation to the input image (referred to as input pre-processing) and temperature scaling before softmax to to increase the difference between the prediction probabilities of InD and OOD samples. The method proposed by Sastry et al. [29] take the prediction of an input image and compare the intermediate features of the image with the ones extracted from other images of the predicted class using gram matrices. The work of Hsu et al. [15] further extended ODIN, referred to as Generalized ODIN (G-ODIN), by introducing an additional output that indicates whether the input sample belongs to InD or OOD. The penultimate layer of a DNN is decomposed into two branches to model the conditional distribution of this indicator variable and its joint distribution with the class label. The conditional probability of the indicator variable is used as the confidence score while the ratio of the joint probability and the conditional probability serve as the final class prediction for an image after applying input processing.

Methods that propose training strategies: DeVries and Taylor [7] introduced a confidence estimation network branch, and proposed to use the confidence estimates obtained from this branch in order to train the softmax probabilities of the classification network. Lee et al. [22] presented a training method for classifier networks so that they become less confident for OOD examples. They introduced two loss terms in addition to the cross-entropy. The first one encourages the network to become less confident for the OOD examples whereas the second one generates the most optimal OOD examples for the first one. The method of Vyas et al. [35] proposed to use an ensemble of classifiers for OOD detection, where each classifier is trained by leaving-out a different class from the InD training set. OOD detection is then performed based on the ensemble of softmax probabilities of each classifier, after applying temperature scaling and input pre-processing as in ODIN. Hendrycks et al. [13] proposed a method called outlier exposure which exploits existing very large datasets that are known to be OOD during training. Yu et al. [39] proposed a DNN with two classification heads, where one aims to minimize classification loss and the other aims to maximize the discrepancy between the two classifiers. The method, named as MCD, uses a subset of OOD samples along with the InD samples in the discrepancy loss. At test time, the samples with higher discrepancy are labeled as OOD. Very recently, Liu et al. [25] proposed an energy-based method (EBM) which interprets softmax probabilities as energy scores and use for OOD detection.

Methods that perform density estimation: Lee et al. [23] proposed a method named as Mahalanobis, which models the class conditional pdfs of the features at intermediate layers of a DNN with Gaussian densities for InD samples. The parameters of each class conditional Gaussian are estimated by computing the empirical mean and co-variance using InD training samples belonging to that class. At test time, ODIN-style input pre-processing is applied before evaluating the estimated densities to obtain a confidence score, which is expected to be higher for InD samples and lower for OOD samples.

Despite their successful performance, most of the aforementioned methods are designed for OOD detection for classification tasks and their extension to non-classification tasks usually is not trivial. Task-agnostic networks that do not share the same drawback have also been proposed. Hendryks et al. [14] proposed a self-supervised learning (SSL) based OOD detection method. The method trains an auxiliary rotation network, which predicts angle of rotation in discrete categories, on the InD dataset and computes a confidence score for a test image as the maximum of the softmax activation, expecting higher activations for InD samples compared to OOD samples. Kim et al. [18] proposed a method, referred to as RaPP, that is based on the observation that in an autoencoder the internal feature representations of an input image and its reconstructed version are very similar for InD samples and the similarity decreases for OOD samples that are not used for the training of the autoencoder. RaPP defines a confidence score based on this observation for OOD detection. Venkatakrishnan et al. [34] combine the ideas in SSL and RaPP, and propose a method (Multitask_SSL) by jointly training a network for both rotation prediction and reconstruction tasks for OOD detecion in brain images. As both SSL [14], RaPP [18], and Multitask_SSL[34] operate on auxiliary networks that are detached from the main network, they are task-agnostic and therefore can be applied to both classification and non-classification tasks.

1.2 Contribution

In this paper, we propose a simple yet effective task-agnostic OOD detection method. In the proposed method, we estimate feature pdfs of each channel in a DNN using KDE and InD training images. We evaluate the pdfs using a new test sample and obtain a confidence score for each channel. We combine all the scores into a final confidence score using a logistic regression model, which we train using channel-wise confidence scores of training images as InD samples and their adversarially perturbed versions as OOD samples.

We take our motivation from Mahalanobis [23] for developing the proposed method but extend it in multiple ways crucial for building a task-agnostic method that achieves improved detection accuracy. (1) Mahalanobis estimates class conditional densities while the distribution approximation in the proposed method is not conditioned on the class, making it task-agnostic. (2) Direct use of Mahalanobis in the task-agnostic setting is feasible using unconditioned Gaussian distributions to approximate layer-wise feature distributions of InD samples. However, the Gaussian assumption may be too restrictive to model unconditioned feature densities and lead to lower accuracy when the assumption does not hold. Using a nonparametric density estimation method KDEs, we extend the flexibility of the density approximation in the proposed method. (3) Layer-wise approximation is prone to the curse-of-dimensionality. Even though Mahalanobis takes channel-wise mean to reduce the dimension of a channel from $C\times H\times W$ to $C\times 1$ before density estimation, the resulting vector can still be high dimensional in modern architectures. We approximate 1D channel-wise distributions in the proposed method, which are simpler to estimate. This approach ignores dependencies between channels of a layer in the density estimation part but takes them into account in the logistic regression model that combines channel-wise scores.

The use of KDEs for OOD detection is not new. The first use dates back to 1994 when Bishop [2] applied KDE in the input space. In that work, the input was only 12-dimensional and application of KDE was feasible. In modern architectures, the input dimensions are often much larger thereby, making the direct application of Bishop’s method infeasible. The application of a modified version of the method in high dimensional spaces is still possible by applying KDE over the distances between the test image and the training images [17, 4]. Bishop’s method as well as its modified version differs from the proposed method. In our method, we use multiple channel-wise KDEs and aggregate results

Using KDE and estimating channel-wise pdfs are conceptually simple extensions that are very effective and yield substantial gains. We performed extensive comparisons on DNNs trained for classification and segmentation tasks. In the classification experiments, we use the common benchmark that contains 2 different classification networks trained on CIFAR-10 and CIFAR-100 datasets [21] and 6 other OOD datasets. We compare the proposed method with 6 methods in the literature most of which are either very recent or common baselines used in the literature. In the segmentation experiments, we use datasets for brain MRI segmentation and compare with 5 methods. In total, we compare with 10 different OOD detection methods to correctly position the proposed method within the current literature.

2 Method

Let us denote a set of training images with $X_{tr}=\{x_1,x_2,\mathrm{\dots }{x}_M\}\sim P_{in}$ and corresponding labels with $y_{tr}=\{y_1,y_2,\mathrm{\dots },y_M\}$, where $P_{in}$ denotes the InD. Let us also denote a DNN with $f$, trained using $(X_{tr},y_{tr})$. $f$ is more likely to perform good predictions on a test image $x_{test}$ if $x_{test}\sim P_{in}$ and incorrect predictions if $x_{test}\sim P_{out}$, where $P_{out}\ne P_{in}$. In this section, we present the proposed task-agnostic KDE-based approach that identifies test images sampled from $P_{out}$. The application of the proposed model at test time is summarized in Algorithm 1.

Input: Test Image: $x$

Random subset of training images: ${\widehat{X}}_{tr}=\{x_{u_1},x_{u_2},\mathrm{\dots },x_{u_N}\}$

Weights of logistic regression classifier: ${\alpha }_{lc}$

Set of kernel sizes: $𝝈$

Output: Confidence score of $x$: ${ℳ}_x$.

foreach $l\in L$ do

for $c\leftarrow 1$ to $C_l$ do

Perform KDE at channel $c$ of the feature map at layer $l$: ${\widehat{p}}_{lc}(x)=\frac{1}{N}{\sum }_{i=1}^N𝒦(f_{lc}(x)-f_{lc}(x_{u_i});{\sigma }_{lc})$

end for

end foreach

return ${ℳ}_x={\sum }_{l=1}^L{\sum }_{c=1}^{C_l}{\alpha }_{lc}{\widehat{p}}_{lc}(x)$

Algorithm 1 The proposed method

2.1 Computing the confidence scores

The main output of the proposed method is a confidence score that indicates how likely a given sample belongs to OOD for the given DNN $f$. In this section, we describe how we compute this score. Let us assume that $f$ consists of $L$ layers and the feature map in a layer $l$ for a given image $x$ is denoted as $f_l(x)$ and it has dimensions $C_l\times H_l\times W_l$, where $C_l$, $H_l$, and $W_l$ are the number of channels, height, and width of the feature map, respectively. We take the channel-wise mean of the feature map and reduce the dimensionality to $C_l\times 1$, as also done in [23]. We denote the resulting $C_l$-dimensional feature vector by $f_l^{\prime }(x)$. We then estimate the marginal feature pdfs for each channel $c$ using KDE:

$$p_{lc}(x)\approx {\widehat{p}}_{lc}(x)=\frac{1}{M}\sum _{i=1}^M𝒦(f_{lc}^{\prime }(x)-f_{lc}^{\prime }(x_i);{\sigma }_{lc})$$ (1)

where $p_{lc}$ is the true marginal pdf of the features $f_{lc}^{\prime }$ in channel $c$ of layer $l$, ${\widehat{p}}_{lc}$ is the estimate of the pdf, and $𝒦(u,v;{\sigma }_{lc})=e^{-{(u-v)}^2/{\sigma }_{lc}^2}$ is a 1D squared exponential kernel with ${\sigma }_{lc}$ being the kernel size. We describe the procedure to set the kernel size ${\sigma }_{lc}$ in detail later in Section 2.3. When using KDE, we use samples $x_i\in X_{tr}$ and thus model InD channel-wise pdfs with ${\widehat{p}}_{lc}$. For a given sample $x$, ${\widehat{p}}_{lc}(x)$ is the confidence score of channel $c$ in layer $l$.

The advantage of estimating channel-wise pdfs over estimating layer-wise pdfs, as was done in [23], is performing density estimation in 1D space instead of $C_l-$D space. Typically, $C_l$ can be very large in modern networks, and density estimation becomes less accurate in high-dimensions [30], channel-wise estimation avoids this.

In order to evaluate $p_{lc}$ in Eq. (1) for a new sample, ideally we need to store all the InD training images in $X_{tr}$. In real-world applications where $M$ is very large, storing the entire $X_{tr}$ may not be feasible and the summation over $M$ images in Eq. (1) can take very long. Improving the computational and memory efficiency of KDE-based methods are possible by defining an unbiased estimator [9], which simply uses a random subset of $X_{tr}$ such that

$${\widehat{X}}_{tr}=\{x_{u_1},x_{u_2},\mathrm{\dots },x_{u_N}\}\subset X_{tr}$$

where $\{u_1,u_2,\mathrm{\dots },u_N\}\subset \{1,2,\mathrm{\dots },M\}$ is a random subset of indices generated by sampling from a Uniform density, $𝒰(1,M)$, without replacement and . Using the random subset, we replace the summation in Eq. (1) with the computationally more efficient unbiased estimator

$$p_{lc}(x)\approx {\widehat{p}}_{lc}(x)=\frac{1}{N}\sum _{i=1}^N𝒦(f_{lc}^{\prime }(x)-f_{lc}^{\prime }(x_{u_i});{\sigma }_{lc}).$$ (2)

In our experiments, we set $N=5000$. In the supplementary material, we demonstrate results with different choices of $N$.

Estimating marginal pdfs using Eq. (2) does not model dependencies between channels. In the proposed method, we take into account such dependencies and compute the final confidence score using a logistic regression classifier:

$${ℳ}_x=\sum _{l=1}^L\sum _{c=1}^{C_l}{\alpha }_{lc}{\widehat{p}}_{lc}(x)$$ (3)

where ${\alpha }_{lc}$ are the weights that are learned as described next.

2.2 Learning logistic regression weights ${\alpha }_{lc}$

The role of the logistic regression model is to distinguish between InD and OOD samples given the channel-wise confidence scores. Training for the weights ${\alpha }_{lc}$ requires having access to both InD and OOD images. Although the InD images, $X_{tr}$, are already available, it is difficult to capture all possible images in $P_{out}$. Lee et al. [23] propose using adversarial examples obtained by FGSM [10] as samples from $P_{out}$ for hyperparameter tuning. We use adversarial examples as OOD samples to train logistic regression in the proposed method. After obtaining OOD samples by applying adversarial perturbation to the images in $X_{tr}$ using FGSM, the logistic regression classifier is trained by using the confidence scores ${\widehat{p}}_{lc}$ as inputs, and the output labels are provided as positive for InD images and negative for the OOD ones. Note that FGSM can work with any type of label that is for the task, e.g., image-level label for classification, ground truth mask for segmentation and so on. Therefore, using FGSM does not affect the task-agnostic nature of the proposed method. We present further details on the FGSM method in the supplementary materials for completeness.

2.3 Determining size of KDE kernel ${\sigma }_{lc}$

Kernel size, $\sigma$, is a crucial parameter of KDE since it significantly affects the shape of the estimated density. Setting $\sigma$ to a large value leads to very smooth pdfs, reducing the likelihood of the observed samples as well as other samples from the same distribution. Setting $\sigma$ to a very small value leads to very peaky distributions only attributing high probability to observed samples, and assigning very low probability to unobserved samples even from the same distribution. Therefore, finding an optimal $\sigma$ value is quite important in order to capture the underlying pdf of the data [30].

In the proposed method, as given in Eq. (2), we use a different ${\sigma }_{lc}$ for each channel $c$ and layer $l$. To compute ${\sigma }_{lc}$, we use the $k$-nearest neighbor method ($k$NN) [31]. Specifically, we compute the distance between $f_{lc}^{\prime }(x_{u_i})$ and $f_{lc}(x_{u_j})$ for all $i,j\in [1,N]$, $i\ne j$ and set ${\sigma }_{lc}$ as the $k^{th}$ smallest value. We denote the set of all ${\sigma }_{lc}$ values by $𝝈$.

A parameter of the $k$NN method is the $k$ itself. We automatically select the most appropriate $k$ value from a set of candidate values denoted by $𝐤$. To achieve this, we split a validation set $X_{val}$ from $X_{tr}$ by taking the samples that are not used in KDE. Then, we apply adversarial perturbation to the images in $X_{val}$ and obtain $X_{val}^{adv}$ as OOD examples. We select the $k\in 𝐤$ that maximizes the difference between the InD and OOD datasets.

$$k_{lc}=\underset{k\in 𝐤}{\arg \max }\sum _{x\in X_{val}}{\widehat{p}}_{lc}(x)-\sum _{x^{\prime }\in X_{val}^{adv}}{\widehat{p}}_{lc}(x^{\prime })$$ (4)

where $k_{lc}$ indicates the optimum $k$ value in layer $l$. In our experiments, we choose $k_l$ from the candidate set $𝐤=\{1,2,5,10,15,20,50\}$ using the described method.

3 Experiments and results

We evaluate the performance of the proposed approach on DNNs trained for classification and segmentation tasks. The proposed method is implemented in PyTorch and we run all experiments on a Nvidia GeForce Titan X GPU with 12GB memory.

3.1 Datasets and network architectures

In the classification experiments, we use two different ResNet architectures trained on CIFAR-10 and CIFAR-100 [21] datasets that contain images from $10$ and $100$ classes, respectively. Both datasets contain $50000$ training and $10000$ test RGB color images of size $32\times 32$. The pretrained models are used as common benchmarks in the literature and are available at https://github.com/pokaxpoka/deep_Mahalanobis_detector.

We use 6 common benchmark datasets as OOD. SVHN [27] contains $26032$ images of house numbers in Google street-view images. TinyImageNet (TIN) dataset consists of $10000$ $32\times 32$ RGB test images [6]. LSUN dataset contain $10000$ RGB test images with size $32\times 32$ [38]. iSUN is a subset of SUN dataset [37] that contains 8925 RGB images resized to $32\times 32$. Gaussian and Uniform datasets contain 10000 noise images generated from a Gaussian distribution with zero mean and unit variance, and a Uniform distribution, respectively. SVHN dataset is available in Pytorch and TIN, LSUN, and iSUN are available at https://github.com/facebookresearch/odin.

In the segmentation experiments, we use images from 2 publicly available datasets for brain segmentation: Human Connectome Project (HCP) [33] and Autism Brain Imaging Data Exchange (ABIDE) [8]. HCP dataset contains both T1w and T2w images for each subject, while ABIDE dataset consists of T1w image from different imaging sites. HCP_T1w and HCP_T2w datasets contain images from 47 patients and we split 21 for training, 5 for validation, and 21 for testing. There are T1w images from 37 patients in both ABIDE_Caltech_T1w and ABIDE_Stanford_T1w datasets and we split 11, 5, 21 images for train, validation and test.

Using HCP and ABIDE datasets, we design 2 different experiments to evaluate OOD detection performance on segmentation task. In the first experiment, we train a UNet [28] architecture on ABIDE_Caltech_T1w images and use ABIDE_Stanford_T1w, HCP_T1w, and HCP_T2w images as OOD. In the second experiment, we train the UNet on HCP_T1w image and use ABIDE_Caltech_T1w, ABIDE_Stanford_T1w, and HCP_T2w as OOD. We choose UNet as the network architecture since it is the most common choice for medical image segmentation [28, 3, 20]. In both experiments, we segment the following 15 labels: background, cerebellum gray matter, cerebellum white matter, cerebral gray matter, cerebral white matter, thalamus, hippocampus, amygdala, ventricles, caudate, putamen, pallidum, ventral DC, CSF and brain stem.

OOD	FPR at 95% TPR $\downarrow$	AUROC $\uparrow$
	Baseline / ODIN / Mahalanobis / MCD / G-ODIN / EBM / Proposed
SVHN	25.77 / 16.65 / 8.37 / 60.61 / 10.50 / 6.86 / 6.49	89.88 / 95.42 / 98.12 / 72.86 / 97.80 / 98.19 / 98.48
TIN	28.37 / 11.24 / 18.89 / 40.44 / 18.60 / 35.88 / 8.41	90.53 / 96.78 / 96.73 / 89.75 / 96.10 / 86.21 / 98.31
LSUN	28.31 / 10.30 / 19.61 / 34.46 / 9.10 / 21.62 / 3.80	91.09 / 97.06 / 96.77 / 91.15 / 98.00 / 92.50 / 99.01
iSUN	28.02 / 12.37 / 22.46 / 37.72 / 11.20 / 22.52 / 7.31	91.01 / 96.03 / 96.34 / 89.89 / 97.60 / 92.03 / 98.55
Gaussian	6.44 / 2.69 / 0.0 / 4.21 / 0.0 / 0.13 / 0.0	97.11 / 98.45 / 100.0 / 97.14 / 100.0 / 99.96 / 100.0
Uniform	9.24 / 4.16 / 0.0 / 13.17 / 0.0 / 0.0 / 0.0	96.04 / 97.78 / 100.0 / 92.69 / 100.0 / 100.0 / 100.0

Table 1: Quantitative results on distinguishing test set of CIFAR-10 InD dataset from several OOD datasets. $\uparrow$ indicates larger value is better and $\downarrow$ indicates lower value is better. All values are percentages.

OOD	FPR at 95% TPR $\downarrow$	AUROC $\uparrow$
	Baseline / ODIN / Mahalanobis / MCD / G-ODIN / EBM / Proposed
SVHN	55.73 / 24.76 / 15.53 / 73.33 / 44.90 / 45.49 / 17.46	79.34 / 92.13 / 97.01 / 64.92 / 93.20 / 88.93 / 95.44
TIN	58.97 / 33.74 / 24.33 / 56.95 / 23.50 / 70.04 / 7.64	77.01 / 88.32 / 95.04 / 85.53 / 95.90 / 75.11 / 98.38
LSUN	64.71 / 37.09 / 28.68 / 58.40 / 23.20 / 67.99 / 3.73	75.58 / 87.70 / 94.66 / 84.97 / 96.10 / 76.45 / 99.13
iSUN	63.26 / 38.21 / 29.46 / 64.32 / 24.70 / 70.11 / 6.07	75.68 / 86.73 / 94.02 / 83.46 / 95.70 / 76.57 / 98.75
Gaussian	58.43 / 39.41 / 0.0 / 10.78 / 0.0 / 0.0 / 0.0	55.85 / 72.04 / 100.0 / 94.02 / 100.0 / 100.0 / 100.0
Uniform	32.04 / 18.49 / 0.0 / 15.99 / 0.0 / 0.0 / 0.0	85.13 / 89.81 / 100.0 / 92.34 / 100.0 / 100.0 / 100.0

Table 2: Quantitative results on distinguishing test set of CIFAR-100 InD dataset from several OOD datasets. $\uparrow$ indicates larger value is better and $\downarrow$ indicates lower value is better. All values are percentages.

3.2 Details of the methods used in comparisons

In the classification experiments, we compare the proposed method with the Baseline method proposed by Hendryks et al. [12], ODIN [24], Mahalanobis [23], MCD [39], G-ODIN [15], and EBM [25] which are primarily designed for OOD detection in classification tasks.

ODIN and Mahalanobis have user defined parameters that significantly affect their performance. For ODIN, these parameters are temperature scaling and input pre-processing magnitude. For Mahalanobis, it is only the input pre-processing magnitude. In the original papers, these parameters are chosen from a list of values by utilizing a subset of the target OOD dataset as validation. Similarly, MCD requires samples from the target OOD dataset during training. However, in a real use case, we usually do not have access to the target OOD dataset. EBM uses a large-scale dataset that contains 80 Million images [32] as OOD to fine-tune networks for OOD detection to obtain the best results reported in [25]. We argue that using such a large dataset as OOD in the training may not be suitable to evaluate the real performance of the method since the large dataset may contain examples very similar to the target OOD benchmarks used in the experiments. For a fair comparison of the existing methods, we use adversarial images for the methods, including ours, that require OOD examples either for setting the hyperparameters or model fine-tuning. Note that the adversarial images can be obtained with no additional cost using the InD training set. We searched the adversarial magnitude parameter from a set of values $\{0.01,0.1,1.0,2.0,5.0\}$ and choose the best values based on the performance on a hold-out set of adversarial images. The implementation of G-ODIN is not publicly available but as our experimental settings are very similar to theirs [15], including network architectures and OOD datasets, we directly use the results presented in their paper for comparison.

In the segmentation experiments, we compare the proposed method with 5 different OOD detection methods that can work on non-classification tasks: Baseline [12], Bishop [2], SSL [14], RaPP [18], and Multitask_SSL [34]. In the experiments with RaPP, we train an auto-encoder network on the InD dataset (COCO) by using a 10-layers encoder-decoder network architecture as suggested in the original paper [18]. SSL uses a self-supervised rotation network for OOD detection. To compare with SSL, we randomly rotate the training images of the InD dataset by a value from the set $\{0^{\circ },{90}^{\circ },{180}^{\circ },{270}^{\circ }\}$ and train a neural network to predict the rotation angles applied on input images using a ResNet architecture. In Multitask_SSL, we constructed a 12-layers variational autoencoder (VAE) [19]where an additional 2 dense layers are applied on top of the latent representation for rotation prediction as suggested in the original paper. We trained the architecture jointly for both tasks. In Baseline [12], we take the average of the softmax probabilities for the foreground classes as a confidence score for OOD detection, which is expected to be higher for InD and lower for OOD samples. Lastly, we also compare our method with Bishop’s work [2] to demonstrate the value of applying KDE on the lower-dimensional space rather than the high-dimensional input space.

SVHN	TIN	LSUN	iSUN	Adversarial

Figure 2: Trends of channel-wise OOD detection performance evaluated with AUROC for different OOD datasets when the InD dataset is CIFAR-10. The plots show that the AUROC trend of adversarial images is similar to the OOD datasets, indicating that the logistic regression classifier trained using adversarial images can generalize to OOD ones.

3.3 Evaluation methods

We use the evaluation methods that are commonly used for evaluating OOD detection methods in the literature [12]. In all evaluations, we take the InD as the positive class and OOD as the negative class.

FPR at $95\%$ TPR: False positive rate (FPR) is measured when the true positive rate (TPR) of $95\%$ is reached at a certain threshold. The false positive rate is calculated as FPR=FP/(FP+TN) and true positive rate is calculated as TPR=TP/(TP+FN) where TP, FP, TN, and FN represent true positive, false positive, true negative, and false negative, respectively.

AUROC: The Area Under the Receiver Operating Characteristic curve (AUROC) is a threshold independent method that measures the area below Receiver Operating Characteristic (ROC) curve [5]. ROC curve reflects the relationship between TPR and FPR values as the threshold changes. AUROC takes its highest value at $100\%$ when the detection is perfect.

Detection error: This metric measures the probability of wrong classification when TPR is $95\%$. We denote the detection error by $P_{err}$ and compute as $P_{err}$=0.5(1-TPR+FPR). We present the results for this metric in the supplementary material for to save space.

3.4 Results and Analysis

Here we present the main quantitative results for both classification and segmentation tasks and present additional experimental results in the supplementary materials.

3.4.1 Results on classification tasks

We present the OOD detection results when CIFAR-10 and CIFAR-100 datasets are InD in Tables 1 and 2, respectively. The results in CIFAR-10 experiments demonstrate that the proposed methods achieves better OOD detection performance than the existing methods on all OOD datasets. In the experiments on CIFAR-100 dataset, our method produces the best OOD detection results on all datasets except SVHN where it achieves the second best results. The results on the classification tasks suggest that the the proposed method improves the state-of-the-art OOD detection methods in almost all cases.

The logistic regression model in the proposed method is trained with the InD images and their adversarially perturbed versions as OOD samples. Even though the OOD space is vast, training with the perturbed samples seems to perform well. To better understand how the regression model generalizes to real OOD images, we analyzed the channel-wise confidence scores ${\widehat{p}}_{lc}(x)$, which are the inputs to the regression model. We performed OOD detection with each channel’s confidence score separately and plot the AUROC values for SVHN, TIN, LSUN, iSUN datasets, and the adversarial InD images when the InD is CIFAR-10 in Fig. 2. We do not present similar plots for Gaussian and Uniform datasets for the sake of brevity. It can be seen that the AUROC trends across channels for all OOD datasets are similar to the adversarial images, where AUROC results are lower in the earlier channels and higher in the later ones. This demonstrate that channel-wise scores of adversarial images generalize quite well to the ones obtained with the OOD images. This observation holds for CIFAR-100 dataset as well and we present similar plots in the supplementary material.

OOD	FPR at 95% TPR $\downarrow$	AUROC $\uparrow$
	Baseline / Bishop / SSL / RaPP / Multitask_SSL / Proposed
ABIDE_Stanford_T1w	78.90 / 76.21 / 49.60 / 69.57 / 66.48 / 44.25	48.26 / 81.71 / 63.45 / 52.44 / 54.30 / 89.27
HCP_T1w	88.06 / 79.72 / 63.51 / 87.20 / 76.58 / 42.93	39.30 / 75.83 / 55.13 / 40.11 / 45.35 / 93.96
HCP_T2w	80.37 / 41.77 / 81.28 / 57.85 / 70.39 / 40.27	42.19 / 92.93 / 43.82 / 52.78 / 47.88 / 94.62

Table 3: Quantitative results on distinguishing test set of ABIDE_Caltech_T1w InD dataset from several OOD datasets. $\uparrow$ indicates larger value is better and $\downarrow$ indicates lower value is better. All values are percentages.

OOD	FPR at 95% TPR $\downarrow$	AUROC $\uparrow$
	Baseline / Bishop / SSL / RaPP / Multitask_SSL /Proposed
ABIDE_Stanford_T1w	59.25 / 100.0 / 62.96 / 67.01 / 45.06 / 44.78	71.34 / 39.02 / 84.43 / 67.07 / 83.78 / 90.42
ABIDE_Caltech_T1w	83.26 / 100.0 / 58.94 / 99.68 / 63.88 / 11.71	68.41 / 17.21 / 87.22 / 59.38 / 79.56 / 96.77
HCP_T2w	47.55 / 94.98 / 76.89 / 47.94 / 61.62 / 18.77	72.88 / 62.12 / 57.79 / 70.56 / 73.39 / 95.60

Table 4: Quantitative results on distinguishing test set of HCP_T1w InD dataset from several OOD datasets. $\uparrow$ indicates larger value is better and $\downarrow$ indicates lower value is better. All values are percentages.

3.4.2 Results on segmentation tasks

We present the OOD detection results when InD datasets are ABIDE_Caltech_T1w and HCP_T1w dataset in Tables 3 and 4, respectively. The results demonstrate that the proposed method improves the existing methods in all cases. Since Bishop [2] works on high-dimensional input space, it cannot achieve accurate density estimation and produces poor OOD detection results as expected. Here, the results of the self-supervised methods: SSL, RaPP, and Multitask_SSL, were lower than we expect, and we investigated further to interpret the results better. These methods exhibit diminished performance because the self-supervised networks generalize surprisingly well to OOD images. For example, the network trained on HCP_T1w images for the SSL rotation task predicts the rotation angles with $\approx 75\%$ accuracy for both InD and OOD datasets. This holds for the case when we use ABIDE_Caltech_T1w as InD. Our observation is similar to the autoencoder network trained for RaPP. In Figure 3, we show input images and their reconstructions when the ABIDE_Caltech_T1w is InD. The visual results show that the autoencoder reconstructs images from different datasets equally well.

Figure 3: Input images (first row) and reconstructions (second row) of the autoencoder trained on ABIDE_Caltech_T1w dataset for RaPP. From left to right, the images belong to ABIDE_Caltech_T1w, ABIDE_Stanford_T1w, HCP_T1w, and HCP_T2w datasets, respectively.

	Layer-wise		Channel-wise
	Gaussian (Mahalanobis)	KDE	Gaussian	KDE (Proposed)
SVHN	15.53	24.09	12.00	17.46
TIN	24.33	34.08	13.90	7.64
LSUN	28.68	28.47	5.10	3.73
iSUN	29.46	33.49	7.50	6.07

Table 5: Comparison between different combinations of density estimation methods (Gaussian and KDE) with feature spaces (layer-wise and channel-wise) in terms of FPR at 95% TPR in CIFAR-100 dataset.

3.5 Channel-wise vs layer-wise and KDE vs parametric estimation

In the proposed method, we perform channel-wise KDE. Compared to the closest work Mahalanobis [23], this introduces two changes in the density estimation, one in feature selection (layer-wise vs channel features) and the other in estimation methodology (KDE vs Gaussian). In this section, we quantify the contribution of each change. To this end, we perform OOD detection with all possible combinations. The results in Table 5 demonstrate that performing channel-wise density estimation leads to a large improvement on OOD detection accuracy compared to layer-wise density estimation. We argue that this improvement is due to achieving more accurate density estimation in 1D space with the channel-wise features. Dependencies between channels are taken into account in the logistic regression model. We also observe that performing KDE on the channel-wise features yields further improvements over using Gaussian in most cases. This is expected since KDE is more flexible and can lead to more accurate density estimations. We present further experiments to compare channel-wise vs layer-wise KDE in the supplementary material.

3.6 Computation time

We compare the proposed method with the others in terms of computation time. We measure the computation time needed to perform detection on all OOD datasets in the CIFAR-10 experiments for Baseline, ODIN, MCD, EBM, and Mahalanobis. Computation time of G-ODIN is not presented since the implementation is not publicly available. We show the mean and standard deviation of the computation times in the plot in Fig. 4. The results demonstrate that the Baseline, MCD, and EBM are the fastest methods as expected since they do only a single forward pass in the networks. The proposed method comes after them despite the computational cost of KDE. ODIN and Mahalanobis are the computationally less efficient than the other due to the input pre-processing step, which is computationally costly. Mahalanobis is the slowest method among all since it applies input pre-processing before computing the scores at each layer.

Figure 4: Mean and standard deviation of the computation times of all OOD datasets in the CIFAR-10 experiments.

4 Conclusion

In this paper, we presented a task-agnostic OOD detection method that estimates feature densities for each channel of a DNN using KDE. Features of a test image are evaluated at the corresponding KDEs to obtain a confidence score per channel, which is expected to be higher for InD images than OOD ones. These scores are combined into a final score using logistic regression classifier, that is pre-trained using InD training images and their adversarially perturbed versions. Being task-agnostic, the proposed method can be applied to both classification and non-classification DNNs.

We performed an extensive evaluations on both classification and segmentation networks and compare them with the state-of-the-art methods. The results demonstrate that the proposed method that uses channel-wise KDE improves state-of-the-art in majority of the cases. We also performed an experiment to compare the channel-wise density estimation with layer-wise estimation and performing parametric density estimation (Gaussian) with the nonparametric one (KDE). The results show that channel-wise density estimation plays the major role in improving the results since density estimation is easier lower dimensional space. Using KDE for density estimation instead of assuming that the pdf is Gaussian leads to further improvement thanks to the flexibility of the nonparametric method.

Appendix A Results with Detection Error metric

The experimental results in the main paper are presented with respect to FPR at 95% TPR and AUROC metrics. In addition to these results, we present results with respect to Detection Error metric for all methods in Figures 6, 7, 8, and 9 for CIFAR-10, CIFAR-100, ABIDE_Caltech_T1w, and HCP_T1w dataset, respectively.

OOD	Detection Error $\downarrow$
	Baseline / ODIN / Mahalanobis / MCD / EBM / Proposed
SVHN	15.38 / 11.32 / 5.90 / 32.80 / 12.45 / 5.74
TIN	16.68 / 8.12 / 9.44 / 22.72 / 27.76 / 6.70
LSUN	16.65 / 7.69 / 9.02 / 19.73 / 22.10 / 4.40
iSUN	16.51 / 8.68 / 10.23 / 21.36 / 22.99 / 6.15
Gaussian	5.72 / 3.84 / 0.0 / 4.60 / 2.50 / 0.0
Uniform	7.12 / 4.58 / 0.0 / 9.08 / 2.50 / 0.0

Table 6: Detection Error metric results on distinguishing test set of CIFAR-10 InD dataset from several OOD datasets. All values are percentages.

OOD	Detection Error $\downarrow$
	Baseline / ODIN / Mahalanobis / MCD / EBM / Proposed
SVHN	30.36 / 14.88 / 7.72 / 36.16 / 19.64 / 11.23
TIN	31.98 / 19.37 / 11.71 / 30.94 / 35.85 /6.32
LSUN	34.85 / 21.04 / 12.39 / 31.70 / 33.62 /4.36
iSUN	34.13 / 21.60 / 13.37 / 34.66 / 34.77 /5.53
Gaussian	31.71 / 22.20 / 0.0 / 7.89 / 0.0 /0.0
Uniform	18.52 / 11.74 / 0.0 / 10.49 / 0.0 /0.0

Table 7: Detection Error metric results on distinguishing test set of CIFAR-100 InD dataset from several OOD datasets. All values are percentages.

OOD	Detection Error $\downarrow$
	Baseline / Bishop / SSL / RaPP / Multitask_SSL / Proposed
ABIDE_Stanford_T1w	41.95 / 40.60 / 27.30 / 37.28 / 35.74 / 24.62
HCP_T1w	46.53 / 42.36 / 34.25 / 46.10 / 40.79 / 23.96
HCP_T2w	42.68 / 23.38 / 43.14 / 31.42 / 37.69 / 22.63

Table 8: Detection Error metric results on distinguishing test set of ABIDE_Caltech_T1w InD dataset from several OOD datasets. All values are percentages.

OOD	Detection Error $\downarrow$
	Baseline / Bishop / SSL / RaPP / Multitask_SSL /Proposed
ABIDE_Stanford_T1w	32.12 / 52.50 / 33.98 / 36.00 / 24.53 / 24.89
ABIDE_Caltech_T1w	44.43 / 52.50 / 31.97 / 52.34 / 34.44 / 8.35
HCP_T2w	26.27 / 49.99 / 40.94 / 26.47 / 33.31 / 11.88

Table 9: Detection Error metric results on distinguishing test set of HCP_T1w InD dataset from several OOD datasets. All values are percentages.

Appendix B Results with different number of samples $N$ in KDE

$N$ is a hyperparameter of the proposed method that determines number of samples to be used for unbiased estimation of the target density using KDE. In this section, we analyze the behavior of the proposed approach as a function of different $N$ values. We present the OOD detection performance using different metrics for both CIFAR-10 and CIFAR-100 experiments in Tables 10 and 11, respectively. There results demonstrate that the proposed method is not very sensitive to the number of samples, $N$, used in KDE.

OOD	FPR at 95% TPR $\downarrow$	AUROC $\uparrow$	Detection Error $\downarrow$
	$N=$ 1000/2000/5000/7000
SVHN	23.16 / 19.33 / 6.49 / 17.22	94.19 / 94.57 / 98.48 / 97.89	14.08 / 12.16 / 5.74 / 11.11
TIN	7.02 / 8.45 / 8.41 / 8.02	98.29 / 98.26 / 98.31 / 98.73	6.01 / 6.72 / 6.70 / 6.51
LSUN	1.92 / 2.40 / 3.80 / 1.84	99.36 / 99.29 / 99.01 / 99.46	3.46 / 3.70 / 4.40 / 3.42
iSUN	5.63 / 6.69 / 7.31 / 5.91	98.62 / 98.63 / 98.55 / 98.85	5.31 / 5.84 / 6.15 / 5.45
Gaussian	0.0 / 0.0 / 0.0 / 0.0	100.0 / 100.0 / 100.0 / 100.0	0.0 / 0.0 / 0.0 / 0.0
Uniform	0.0 / 0.0 / 0.0 / 0.0	100.0 / 100.0 / 100.0 / 100.0	0.0 / 0.0 / 0.0 / 0.0

Table 10: Performance of the proposed method on CIFAR-10 dataset as a function of number of samples ($N$) used in KDE.

OOD	FPR at 95% TPR $\downarrow$	AUROC $\uparrow$	Detection Error $\downarrow$
	$N=$ 1000/2000/5000/7000
SVHN	18.65 / 17.19 / 17.46 / 17.69	95.61 / 96.24 / 95.44 / 95.99	11.82 / 11.09 / 11.23 / 11.34
TIN	14.98 / 11.98 / 7.64 / 10.96	96.81 / 97.72 / 98.38 / 98.01	9.99 / 8.49 / 6.32 / 7.98
LSUN	10.69 / 6.65 / 3.73 / 5.17	97.60 / 98.65 / 99.13 / 98.98	7.84 / 5.82 / 4.36 / 5.08
iSUN	12.41 / 9.01 / 6.07 / 7.21	97.33 / 98.24 / 98.75 / 98.62	8.70 / 7.00 / 5.53 / 6.10
Gaussian	0.0 / 0.0 / 0.0 / 0.0	100.0 / 100.0 / 100.0 / 100.0	0.0 / 0.0 / 0.0 / 0.0
Uniform	0.0 / 0.0 / 0.0 / 0.0	100.0 / 100.0 / 100.0 / 100.0	0.0 / 0.0 / 0.0 / 0.0

Table 11: Performance of the proposed method on CIFAR-100 dataset as a function of number of samples ($N$) used in KDE.

Appendix C Details of the adversarial perturbation method - FGSM

As we discussed in Sec. 2.2. of the main paper, we produce OOD samples to train logistic regression using an adversarial attack method called Fast Gradient Sign Method (FGSM) [10]. In this section, we present more details of FGSM.

The FGSM method uses gradients of the neural network when creating an adversarial example. To this end, it computes the gradient of the loss function with respect to the input image and adds the gradient to the original input image to obtain the adversarial image. Perturbing the input image with the gradient leads to maximizing the loss function. Adversarial perturbation is applied to an input image $x$ using the following equation

$$x_{adv}=x+ϵ\times sign(\nabla (L(x,y)))$$

where $y$ is the label of $x$, $L(x,y)$ is the loss function, and $ϵ$ is a parameter to control the strength of the perturbation. We present some example images from CIFAR-10 dataset along with their adversarial version in Fig. 5.

Figure 5: Applying FGSM to some example images from CIFAR-10 dataset. Note that each row corresponds to a different image. Columns from left to right indicate original image, gradient of the loss w.r.t. input image, and adversarial image, respectively.

Appendix D Hyperparameter search for $ϵ$ in FGSM

In our experiments, we use adversarial examples generated by FGSM when applying ODIN, Mahalanobis, MCD, EBM, and the proposed method since they require some OOD samples either for hyperparameter selection or training. As mentioned in the main paper, we selected the best $ϵ$ value from a set of values $\{0.01,0.1,1.0,2.0,5.0\}$ in our experiments based on the performance on a validation set which is a portion of the adversarial examples. In this section, we present the results obtained with each $ϵ$ for Mahalanobis (Table 12), EBM (Table 13), and the proposed method (Table 14) to show the sensitivity to this parameter.

Mahalanobis
$ϵ=$	0.01	0.1	1.0	2.0	5.0
	CIFAR-10 / CIFAR-100
SVHN	44.25 / 32.39	26.46 / 19.72	8.37 / 15.53	4.33 / 14.84	4.70 / 37.36
TIN	12.00 / 24.73	10.07 / 22.66	18.89 / 24.33	16.09 / 26.55	11.57 / 25.90
LSUN	10.72 / 25.42	8.70 / 27.43	19.61 / 28.68	15.98 / 32.30	11.05 / 25.85
iSUN	14.33 / 28.48	11.87 / 28.55	22.46 / 29.46	18.22 / 32.16	13.27 / 29.88
Gaussian	0.0 / 0.0	0.0 / 0.0	0.0 / 0.0	0.0 / 0.0	0.0 / 0.0
Uniform	0.0 / 0.0	0.0 / 0.0	0.0 / 0.0	0.0 / 0.0	0.0 / 0.0

Table 12: Results of Mahalanobis in terms of FPR at 95% TPR on CIFAR-10 and CIFAR-100 for different $ϵ$ values.

EBM
$ϵ=$	0.01	0.1	1.0	2.0	5.0
	CIFAR-10 / CIFAR-100
SVHN	0.03 / 1.39	4.21 / 18.35	6.86 / 45.49	77.59 / 89.0	57.83 / 92.94
TIN	76.13 / 87.07	61.68 / 80.92	35.88 / 70.04	30.40 / 68.86	37.41 / 70.91
LSUN	67.18 / 89.12	44.95 / 76.59	21.62 / 67.99	21.10 / 64.93	28.63 / 67.77
iSUN	66.54 / 87.04	46.88 / 76.76	22.52 / 70.11	22.0 / 65.93	28.43 / 69.89
Gaussian	99.75 / 99.44	91.19 / 93.42	0.13 / 0.0	0.0 / 0.0	0.0 / 0.0
Uniform	80.13 / 100.0	53.87 / 94.59	0.0 / 0.0	0.0 / 0.0	0.0 / 0.0

Table 13: Results of EBM in terms of FPR at 95% TPR on CIFAR-10 and CIFAR-100 for different $ϵ$ values.

Proposed
$ϵ=$	0.01	0.1	1.0	2.0	5.0
	CIFAR-10 / CIFAR-100
SVHN	17.54 / 27.71	12.03 / 28.24	6.49 / 17.46	7.18 / 18.79	4.08 / 43.36
TIN	32.46 / 52.03	17.27 / 25.17	8.41 / 7.64	10.33 / 13.73	19.03 / 31.05
LSUN	21.44 / 49.67	11.82 / 17.55	3.80 / 3.73	4.62 / 8.60	5.30 / 16.29
iSUN	26.83 / 52.90	14.92 / 17.80	7.31 / 6.07	8.29 / 12.25	13.72 / 31.11
Gaussian	3.04 / 0.0	0.0 / 0.0	0.0 / 0.0	0.0 / 0.0	0.0 / 0.0
Uniform	1.17 / 0.0	0.0 / 0.0	0.0 / 0.0	0.0 / 0.0	0.0 / 0.0

Table 14: Results of the proposed method in terms of FPR at 95% TPR on CIFAR-10 and CIFAR-100 for different $ϵ$ values.

Appendix E AUROC trends for CIFAR-100 datasets

In the main paper, we perform analysis on CIFAR-10 dataset to better understand how the regression model trained on adversarial images generalizes to real OOD images. In this section, we perform similar analysis on CIFAR-100 to demonstrate that the analysis on CIFAR-10 holds for the other datasets. We performed OOD detection with each channel’s confidence score separately and plot the AUROC values for SVHN, TIN, LSUN, iSUN datasets, and the adversarial InD images when the InD is CIFAR-100 in Fig. 6. We observe that AUROC trends across channels for all OOD datasets are similar to the adversarial images. This demonstrate that channel-wise scores of adversarial images generalize quite well to the ones obtained with the OOD images.

Figure 6: Trends of channel-wise OOD detection performance evaluated with AUROC for different OOD datasets when the InD dataset is CIFAR-100. The plots show that the AUROC trend of adversarial images is similar to most of the OOD datasets, indicating that the logistic regression classifier trained using adversarial images can generalize to most of the OOD datasets.

Appendix F Comparison between channel-wise vs Layer-wise KDE

In this section, we present additional results to show the improvement that is achieved by using channel-wise density estimation instead of layer-wise on CIFAR-10 (Table 15) and CIFAR-100 (Table 16) datasets. Although, layer-wise KDE achieves slightly better results on CIFAR-10 dataset, channel-wise KDE performs significantly better on CIFAR-100.

OOD	FPR at 95% TPR $\downarrow$	AUROC $\uparrow$	Detection Error $\downarrow$
	Layer-wise KDE / Proposed (Channel-wise KDE)
SVHN	2.44 / 6.49	99.36 / 98.48	3.72 / 5.74
TIN	5.11 / 8.41	98.99 / 98.31	5.05 / 6.70
LSUN	2.54 / 3.80	99.44 / 99.01	3.77 / 4.40
iSUN	3.83 / 7.31	99.26 / 98.55	4.41 / 6.15
Gaussian	0.0 / 0.0	100.0 / 100.0	0.0 / 0.0
Uniform	0.0 / 0.0	100.0 / 100.0	0.0 / 0.0

Table 15: Comparison between channel-wise vs layer-wise KDE on CIFAR-10 dataset.

OOD	FPR at 95% TPR $\downarrow$	AUROC $\uparrow$	Detection Error $\downarrow$
	Layer-wise KDE / Proposed (Channel-wise KDE)
SVHN	24.09 / 17.46	94.66 / 95.44	14.54 / 11.23
TIN	34.08 / 7.64	93.89 / 98.38	19.54 / 6.32
LSUN	28.47 / 3.73	94.55 / 99.13	16.73 / 4.36
iSUN	33.49 / 6.07	93.90 / 98.75	19.24 / 5.53
Gaussian	0.0 / 0.0	100.0 / 100.0	0.0 / 0.0
Uniform	0.0 / 0.0	100.0 / 100.0	0.0 / 0.0

Table 16: Comparison between channel-wise vs layer-wise KDE on CIFAR-100 dataset.