Out-of-Distribution Detection in Dermatology using Input Perturbation and Subset Scanning

Abstract

Recent advances in deep learning have led to breakthroughs in the development of automated skin disease classification. As we observe an increasing interest in these models in the dermatology space, it is crucial to address aspects such as the robustness towards input data distribution shifts. Current skin disease models could make incorrect inferences for test samples from different hardware devices and clinical settings or unknown disease samples, which are out-of-distribution (OOD) from the training samples.To this end, we propose a simple yet effective approach that detect these OOD samples prior to making any decision. The detection is performed via scanning in the latent space representation (e.g., activations of the inner layers of any pre-trained skin disease classifier). The input samples could also perturbed to maximise divergence of OOD samples. We validate our ODD detection approach in two use cases: 1) identify samples collected from different protocols, and 2) detect samples from unknown disease classes. Additionally, we evaluate the performance of the proposed approach and compare it with other state-of-the-art methods. Furthermore, data-driven dermatology applications may deepen the disparity in clinical care across racial and ethnic groups since most datasets are reported to suffer from bias in skin tone distribution. Therefore, we also evaluate the fairness of these OOD detection methods across different skin tones. Our experiments resulted in competitive performance across multiple datasets in detecting OOD samples, which could be used (in the future) to design more effective transfer learning techniques prior to inferring on these samples.

Keywords Skin disease classification $\cdot$ Out-of-distribution sample detection $\cdot$ Algorithmic Fairness

1 Introduction

Skin disease remains a global health challenge, with skin cancer being the most common cancer worldwide (Codella et al., 2017). Following the recent success of deep learning (DL) in various computer vision problems (partly due to its automated feature encoding capability), convolutional neural networks (CNNs) (Huang et al., 2016) have been employed for skin disease classification tasks. As we observe increasing interest in DL in applying dermatology (Esteva et al., 2017; Gomolin et al., 2020), it is imperative to address transparency, robustness, and fairness of these solutions (Adamson and Smith, 2018; Qayyum et al., 2020). While many existing deep learning techniques (Mahbod et al., 2020; Gessert et al., 2018; Ahmed et al., 2019) achieve high performance on publicly available datasets (Codella et al., 2017; Tschandl et al., 2018; Combalia et al., 2019; Sun et al., 2016), they utilize ensembles of multiple models aimed at maximising performance with limited consideration to shifts in the input data (Ahmed et al., 2019; Gessert et al., 2019; Zhang et al., 2019), which might result in incorrectly classifying new samples as one of the training classes (with high confidence) though these samples might be from previously unknown or new classes.

Thus, it is necessary to detect out-of-distribution (OOD) samples prior to making decisions in order to achieve principled transfer of knowledge from in-distribution training samples to OOD test samples, thereby extending the usability of the models to previously unseen scenarios. Furthermore, OOD detectors and other DL solutions need to guarantee equivalent detection capability across sub-populations. Particularly in dermatology, bias in representations of skin tones in academic materials (Mcf, ) and clinical care (rab, ) is becoming a primary concern. For instance, New York Times reports major disparities in dermatology when treating skin of color (rab, ) as common conditions often manifest differently on dark skin, and physicians are trained mostly to diagnose them on light skin. STAT (Mcf, ) also reported that lack of darker skin tones in dermatology academic materials adversely affects the quality of care for patients of color. Alarmingly, the growing practice of using artificial intelligence to aid the diagnosis of skin diseases will further deepen the divide in patient care because of the machine learning algorithms, which are trained with such imbalanced datasets (Codella et al., 2019, 2017; Tschandl et al., 2018; Combalia et al., 2019; Sun et al., 2016) (with overwhelming majority of samples with light skin tones). This is supported by the work of Kinyanjui et al. (Kinyanjui et al., 2019), which use Individual Typology Angle (ITA) to approximate skin tones in various publicly available skin disease datasets (Codella et al., 2017; Tschandl et al., 2018; Combalia et al., 2019; Sun et al., 2016) and show that these datasets heavily under-represent darker skin tones. As a result, we also validated the performance of

To this end, we propose a simple yet effective approach that scans over the activations of the inner layers of any pre-trained skin disease classifier to detect OOD samples. We additionally perturb the input data beforehand with our proposed ODIN_low, a modification of ODIN (Liang et al., 2017), which improve OOD detection performance in earlier layers of the network. In our framework, we define two different OOD use cases: protocol variations (e.g., different hardware devices, lighting settings and not compliant with clinical protocol); and unknown disease types (e.g., samples from new disease type that was not observed during training). Without requiring any prior knowledge of the OOD samples, the proposed approach out-performed existing OOD detectors, softmax score (Hendrycks and Gimpel, 2016) and ODIN (Liang et al., 2017), for OOD samples with different validation protocols, and competitive performance is achieved in detecting samples with unknown disease types. We further explore how our proposed and existing OOD detectors perform across skin tones to evaluate fairness. We show that the current OOD detectors show higher performance in detecting darker skin tones as OOD samples than those of lighter skin tones, which is likely impacted by the imbalanced training skin datasets that heavily lack samples of dark skin tones.

Generally, our main contributions are highlighted as follows: 1) We propose a weakly-supervised approach based on subset scanning over the activations of the inner layers of a pre-trained skin disease classifier to detect OOD samples across two use cases: detection of OOD samples from different collection protocol and those from unknown disease classes; 2)We propose perturb input images with ODIN_low noise, for improved OOD detection performance;3) We evaluate our methods against existing OOD detectors: Softmax Score (Hendrycks and Gimpel, 2016) and ODIN (Liang et al., 2017); Furthermore, we evaluate the fairness of the proposed approach and existing methods in their detection performance across skin tones.

	Ensemble	Test Data Augmentation	OOD Detection Post-Training	New Protocol Detection	New Disease Detection	Algorithmic Fairness
(Ahmed et al., 2019)	✓	✓	✗	✗	✓	✗
(Zhang et al., 2019)	✓	✗	✗	✗	✓	✗
(Gessert et al., 2019)	✓	✓	✗	✗	✓	✗
(Bagchi et al., 2020)	✗	✗	✗	✗	✓	✗
(Pacheco et al., 2019)	✓	✗	✓	✗	✓	✗
(Combalia et al., In Press)	✗	✓	✓	✗	✓	✗
(Pacheco et al., 2020)	✗	✗	✓	✓	✓	✗
Ours	✗	✗	✓	✓	✓	✓

Table 1: Summary of the state-of-the-art OOD sample detection in skin disease classification task, and the differentiation of our proposed approach.

2 Related Work

Our review of existing OOD detection methods is grouped into pre-training (Ahmed et al., 2019; Bagchi et al., 2020; Gessert et al., 2019; Zhang et al., 2019) and post-training (Combalia et al., In Press; Pacheco et al., 2019, 2020), based on where the detection step is applied.

Pre-training OOD detection approaches have prior knowledge of the OOD samples and incorporate it during their training phases. Many of these approaches utilize ensembles of existing CNNs (and their variants) to detect OOD samples (Ahmed et al., 2019; Gessert et al., 2019; Zhang et al., 2019). Ahmed et al. (Ahmed et al., 2019) applied one-class learning using deep neural network features where one-class samples were iteratively discarded as OOD samples in a one-vs-all cross-validation strategy, and the OOD samples were detected by taking the prediction average of all the models. Gessert et al. (Gessert et al., 2019) utilized an additional dataset of skin lesions as OOD samples to train their ensemble of CNNs to detect OODs. Zhang et al. (Zhang et al., 2019) employed an ensemble DenseNet-based CNNs consisting of both multi-class and binary classifiers to detect OOD samples. Bagchi et al. (Bagchi et al., 2020) proposed Class Specific - Known vs. Simulated Unknown to detect OOD samples.

Post-training OOD detection approaches do not require any prior knowledge of the OOD samples during training (Combalia et al., In Press; Pacheco et al., 2019, 2020). Pacheco et al. (Pacheco et al., 2019) detected OOD samples using Shannon entropy (Shannon, 1948) and cosine similarity metrics on their CNN’s probability outputs. Instead, Combalia et al. (Combalia et al., In Press) detected OOD samples using Monte-Carlo Dropout (Gal and Ghahramani, 2016) and test data augmentation to estimate uncertainty such as entropy and variance in their network predictions. Pacheco et al. (Pacheco et al., 2020) extended Gram-OOD (Sastry and Oore, 2019) with layer-specific normalization of Gram Matrix values to detect OOD samples.

Table 1 summarizes notable OOD detection studies in dermatology. The majority of these studies employ pre-training approaches using ensembles of CNNs, which result in model complexity and impracticality due to their need of prior knowledge of OOD samples. Test data augmentation is also less plausible to domain experts as it might partially re-synthesize the samples. In this work, we propose a simple, post-training OOD detector that can be applied to any single pre-trained network without any test data augmentation nor prior knowledge of the OOD samples.

Figure 1: Block diagram of the proposed approach. $C$: a trained model for skin disease classification over mentioned datasets (${𝒟}_1$, ${𝒟}_2$); $T$: a skin tone extractor.

3 Proposed Framework

We propose a weakly-supervised OOD detection method to identify skin images collected in different validation protocols and derived from unknown skin disease types, based on subset scanning (Cintas et al., 2020) and ODIN (Liang et al., 2017). Subset scanning treats the OOD detection problem as a search for the most anomalous subset of observations in the activation space of any pre-trained classifier. This exponentially large search space is efficiently explored by exploiting mathematical properties of our measure of anomalousness (Neill, 2012). Our solution can be applied to any off-the-shelf skin disease classifier. Additionally, we evaluate algorithmic fairness of the proposed and existing OOD detectors across skin tones. The overview of the proposed approach is shown in Fig. 1. Given a set of skin datasets $D$ and a pre-trained skin disease classifier $C$ as an input; first, we stratify each dataset through a skin tone distribution extractor $T$ for evaluation purposes. Then, we apply subset scanning across each layer of the classifier $C$ and compute the subset score for the unknown disease use case. To detect protocol variations, we first perturb the input data for the best performing results. In the following sections, we describe the details of the proposed approach.

3.1 Subset scanning for out-of-distribution sample detection

Given a pre-trained network $C$ for skin disease classification, we apply subset scanning (Cintas et al., 2020) on the activations in the intermediate layers of the network $C$ to detect a subset ($S$) of OOD samples (see Algorithm 1). Subset scanning searches for the most anomalous subset $S^{\ast }=\arg {\max }_{S}F(S)$ in each layer, where the anomalousness is quantified by a scoring function $F(\cdot )$, such as a log-likelihood ratio statistic. When searching for this subset, an exhaustive search across all possible subsets is computationally infeasible as the number of subsets ($2^N$) increases exponentially with the number of nodes ($N$) in a layer. Instead, we utilize a scoring function that satisfies the Linear Time Subset Scanning (LTSS) (Neill, 2012) property, which enables efficient maximization over all subsets of data. This LTSS property guarantees that the highest-scoring subset of nodes in a layer are identified within $N$ searches instead of $2^N$ searches. Following the literature on pattern detection (McFowland et al., 2013), we utilize non-parametric scan statistics (NPSS) (McFowland et al., 2013) as our scoring function as it satisfies LTSS property and makes minimal assumptions on the underlying distribution of node activations.

We apply subset scanning on set of layers $C_Y$ of our pre-trained network $C$. For each layer $C_y\in C_Y$, we form a distribution of expected activations at each node using the known in-distribution (ID) samples $X_z$, which were used during training and can also be referred as background images. Comparing this expected distribution to the node activations of each test or evaluation sample $X_i$, we can obtain p-values $p_{ij}$ for each $i^{th}$ test sample and $j^{th}$ node of layer $C_y$. We can then quantify the anomalousness of the p-values by finding the subset of nodes that maximize divergence of the test sample activations from the expected. This yields $|C_Y|$ anomalous scores $S_{(C_y)}^{\ast }$ for each test sample. We expect OOD samples to yield higher anomalous scores $S$ than ID samples, and we detect OOD samples with simple thresholding. Note that the OOD detection is performed in an unsupervised fashion without any prior knowledge of the OOD samples.

1

input : Background Image: $X_z\in D^{H_0}$, Evaluation Image: $X_i$, training dataset: $D_{train}$, ${\alpha }_{\text{max}}$.

output : $AUROC$, $F_1$, $AURO{C}^t$, and $F_1^t$ for $X_i$

2 $C\leftarrow$ TrainSkinDiseaseClassifier ($D_{train}$);

3 $C_Y\leftarrow$ Set of layers in $C$;

4 $X_i^t\leftarrow$ PredictITASkinTone ($X_i$);

5

6${\widehat{X}}_z\leftarrow$ AddODINNoise ($X_z$);

7 ${\widehat{X}}_i\leftarrow$ AddODINNoise ($X_i$) ;

8 for $C_y$ in $C_Y$ do

9 for $j\leftarrow 0$ to $|C_y|$ do

10 $A_{zj}^{H_0}\leftarrow$ ExtractActivation ($C_y$, ${\widehat{X}}_z$);

11

12 $A_{ij}\leftarrow$ ExtractActivation ($C_y$,${\widehat{X}}_i$);

13

14 $p_{ij}=\frac{{\sum }_{X_z\in D^{H_0}}I(A_{zj}>=A_{ij})+1}{M+1}$;

15 ;

16 $p_{ij}^s\leftarrow$ SortAscending ($p_{ij}^{\ast }$);

17 for $k\leftarrow 1$ to $|C_y|$ do

18 $S_{(k)}=\{p_y\subseteq p_{ij}^s\forall y\in \{1,\mathrm{\dots },k\}\}$;

19 ${\alpha }_k=max(S_{(k)})$;

20 $F(S_{(k)})\leftarrow$ NPSS (${\alpha }_k$, k, k);

21

22 $k_{(C_y)}^{\ast }\leftarrow \arg \max F(S_{(k)})$;

23 ${\alpha }_{(C_y)}^{\ast }={\alpha }_{k_{(C_y)}^{\ast }}$;

24 $S_{(C_y)}^{\ast }=S_{(k_{(C_y)}^{\ast })}$;

25

26$AUROC$, $F_1$ = ComputeDetectionPerformance (${\sum }_{C_y}{S}_{(C_y)}^{\ast }$);

27 $AURO{C}^t$, $F_1^t$ = StratifyPerSkinTone($X_i^t$, $AUROC$, $F_1$);

return $AUROC$, $F_1$, $AURO{C}^t$, and $F_1^t$

Algorithm 1 Pseudo-code for the proposed new protocol (OOD) detection.

3.2 ODIN and ODIN_low Perturbations

We have also evaluated the impact of adding small perturbations, prior to subset scanning, to each test sample following ODIN (Liang et al., 2017) for enhanced OOD. ODIN involves two steps, input pre-processing and temperature scaling. In the first step, $X_i$ is perturbed by adding a small perturbation computed by back-propagating the gradient of the training loss with respect to $X_i$ and weighted by parameter $ϵ$. This pre-processed $X_i$ is then fed into the neural network and temperature scaling with parameter $\tau$ is applied in the final softmax layer $C_s$. The two hyperparemters, $ϵ$ and $\tau$, are chosen so that the OOD detection performance of softmax score (Hendrycks and Gimpel, 2016), the maximum value of the softmax layer output, is optimized. We further modified ODIN and propose ODIN_low with parameters ${\tau }_{low}$ and ${ϵ}_{low}$ that leads to the lowest softmax score performance. As subset scanning is applied not only on the softmax layer but also on the the inner layers of the network, we show that ODIN_low helps improve OOD detection in the earlier layers of the network.

3.3 Algorithmic Fairness of OOD detectors across skin tone

We further evaluate algorithmic fairness of our proposed OOD dectector across skin tones, estimated by adopting an existing framework (Kinyanjui et al., 2019). To this end, the non-diseased regions of a given skin image are segmented using Mask R-CNN (He et al., 2017), and individual typology angle (ITA) values are computed as $ITA=\arctan \left(\frac{L_{\mu }-50}{b_{\mu }}\right)\times \frac{{180}^{\circ }}{\pi }$, where $L_{\mu }$ and $b_{\mu }$ are the average of luminance and yellow values of non-diseased pixels in CIELab-space. ITA values are used to stratify the samples into three Fitzpatrick skin tone categories, Light, Intermediate, and Dark, as shown in Table 2.

ITA Range	Skin Tone Category
$ITA>{41}^{\circ }$	Light
	Intermediate
$ITA\le {28}^{\circ }$	Dark

Table 2: Summary of Fitzpatrick skin tone categorization of computed $ITA$ values.

4 Datasets

We validate the proposed frame work using two datasets: ISIC 2019 (Codella et al., 2017; Tschandl et al., 2018; Combalia et al., 2019) for samples of unknown diseases; and SD-198 (Sun et al., 2016) for samples from unknown collection protocols. We stratify the samples from both datasets based on skin-tones to observe the impact of various OOD methods across the population spectrum (see Figure 2).

4.1 ISIC 2019

ISIC 2019 (Codella et al., 2017; Tschandl et al., 2018; Combalia et al., 2019) dataset is an extension of ISIC 2018 and merges HAM10000 (Tschandl et al., 2018), BCN20000 (Combalia et al., 2019), and MSK (Codella et al., 2017) datasets. It consists of $25,331$ dermoscopic images among eight diagnostic categories: Melanoma, Melanocytic nevus, Basal cell carcinoma, Actinic keratosis, Benign keratosis, Dermatofibroma, Vascular lesion, and Squamous cell carcinoma. As its test set is not available publicly, we set aside Dermatofibroma (DF) and Vascular lesion (VASC) samples during training, and utilize them during the test time as OOD samples of unknown diseases. These two classes are chosen as they contain the least number of samples in the dataset. First row of Figure 2 show example images of this dataset for each of the three skin tone categories we consider in this work.

4.2 SD-198

SD-198 (Sun et al., 2016) dataset contains $198$ different diseases from different types of eczema, acne and various cancerous conditions, totalling $6,584$ images. The images are collected via various devices, mostly digital cameras and mobile phones with higher levels of noise and varying illumination. We use this dataset for OOD samples that are collected from unknown protocols. We show some example images of the dataset in the second row of Figure 2 that are stratified into three skin-tone categories, Light, Intermediate, and Dark.

Light	Intermediate	Dark

Figure 2: Example images from ISIC 2019 (Codella et al., 2017) (top) and SD-198 (Sun et al., 2016) (bottom) stratified into three skin tone categories: Light, Intermediate, and Dark.

5 Experimental setup

5.1 Skin disease model setup

We adopt DenseNet-121 (Huang et al., 2016) pre-trained on ImageNet (Deng et al., 2009) for the skin disease classification task and fine-tune it on ISIC 2019 (Codella et al., 2017). To accommodate for the change in number of classes for the skin disease classification task, we resize the last four fully connected layers of DenseNet to $512$, $256$, $128$, and $7$ nodes followed by a SoftMax with $7$ nodes for the seven skin disease classes. We use Adam (Kingma and Ba, 2015) optimization with a learning rate of $1e^{-4}$ and a batch size of $40$. To address the class imbalance problem, we employ weighted cross-entropy loss. The implementation is done with the Python 3.6 (Harris et al., 2020) and TensorFlow 1.14 (Abadi et al., 2016). To validate detection of unknown disease samples, we use DF and VASC classes from ISIC-2019, consisting of $253$ and $225$ samples, respectively. Similarly, for samples with different collection protocols, we extract $10$ sets of $260$ samples from SD-198 and report their aggregate performance.

5.2 Subset scanning setup

We apply subset scanning across eight layers $C_Y$ consisting of six convolutional layers $(C_{con{v}_1},\mathrm{\dots },C_{con{v}_6})$, global pooling layer $(C_{gp})$, and softmax layer $(C_s)$. For ODIN (Liang et al., 2017), we use temperature scaling parameter $\tau =10$ and perturbation magnitude $ϵ=0$ (optimized on ISIC-2019) for SD-198 samples and $\tau =5$ and $ϵ=0.0002$ (optimized on SD-198) for ISIC-2019 samples. For ODIN_low, we use ${\tau }_{low}=2$ and ${ϵ}_{low}=0.2$, which leads to AUROC equal to 0.5 for Softmax Score for both OOD use cases. We employ Area Under Receiver Operating Characteristic Curve (AUROC) and maximum $F_1$-score ($F_1$) as our metrics to evaluate the OOD detection performance.

6 Results

In this section, we show the result of proposed OOD detector with subset scanning and ODIN as detailed in Section 3. We first compare our result of OOD detection to Softmax Score (Hendrycks and Gimpel, 2016) and ODIN (Liang et al., 2017) in Tables 3 for OOD samples with different collection protocol and in 4 for OOD samples with unknown disease types. We further stratify OOD samples based on skin tone for these approaches and report their performance in Table 5. We show in Figure 3 the detection performance of our proposed method on individual layers across our network and further stratify these performances across skin tone in Figure 4.

Methods	AUROC	$F_1$
Softmax Score (Hendrycks and Gimpel, 2016)	$74.4\pm 1.7$	$71.0\pm 1.1$
ODIN (Liang et al., 2017)	$74.5\pm 1.6$	$70.8\pm 1.1$
SS ($C_s$)	$68.2\pm 1.4$	$71.3\pm 0.5$
SS ($C_{gp}$)	$62.7\pm 1.2$	$72.5\pm 0.6$
SS ($C_{con{v}_1}$)	$41.6\pm 1.8$	$68.1\pm 0.2$
SS ($C_s$)+ODIN	$51.2\pm 1.9$	$67.9\pm 0.3$
SS ($C_{con{v}_1}$)+ODINlow	$85.4\pm 0.6$	$81.9\pm 0.6$
SS (Sum All Layers)+ODINlow	$\mathbf{91.0}\pm \mathbf{0.8}$	$\mathbf{86.9}\pm \mathbf{1.1}$

Table 3: Detection performance for OOD samples of unknown collection protocols validated with SD-198 (Sun et al., 2016). Bold values are the best performers in each column.

6.1 OOD samples from a different protocol or equipment

We first show the result of detecting OOD samples that are collected with different protocols or equipment. Table 3 summarizes the results of the proposed approach - subset scanning (SS) with and without noise, and compared with the existing baselines (Hendrycks and Gimpel, 2016; Liang et al., 2017). In the top panel, we see that ODIN (Liang et al., 2017) increases the AUROC performance of Softmax Score by around 0.1 on average. For samples with ODIN noise, we show the performance of subset scanning on the softmax layer $C_s$, as ODIN is optimized on Softmax Score, and for samples with ODIN_low noise, we show the result of subset scanning on the first convolutional layer ($C_{con{v}_1}$). We achieve the best performance with AUROC of $91.0\pm 0.8$ and maximum $F_1$-score of $86.9\pm 1.1$ using the sum of subset scores $S_{(C_y)}^{\ast }$ across all eight layers with ODIN_low (bottom row in Table 3).

Methods	AUROC		$F_1$
	DF	VASC	DF	VASC
Softmax Score (Hendrycks and Gimpel, 2016)	80.9	73.2	76.5	70.5
ODIN (Liang et al., 2017)	72.3	65.3	70.3	67.4
SS ($C_s$)	80.8	70.8	75.7	72.3
SS ($C_{gp}$)	37.4	57.9	65.9	69.2
SS ($C_{con{v}_1}$)	50.9	62.5	65.8	68.7
SS ($C_s$)+ODIN	71.8	63.3	70.4	67.4
SS ($C_{con{v}_1}$)+ODINlow	47.6	39.8	65.9	67.1
SS (Sum All Layers)+ODINlow	47.6	40.4	65.9	67.2

Table 4: Performances of detecting OOD samples of unknown disease types, DF and VASC. Bold values are the best performers in each column.

6.2 OOD samples of unknown diseases

Table 4 shows the performance of detecting OOD samples of unknown diseases (DF and VASC) that are unseen during training. While Softmax Score (Hendrycks and Gimpel, 2016) yields the best performance, subset scanning on the softmax layer $C_s$ shows comparable performance. We see worse performances with ODIN as these OOD samples are from the same dataset as ID samples and adding noise likely blurs the unique features present in each skin disease class.

Methods	Skin Tone	Unknown diseases				Collection protocol
		DF		VASC		SD-198
		R	AUROC	R	AUROC	R	AUROC
Softmax Score (Hendrycks and Gimpel, 2016)	Light	171	81.0	185	72.1	986	75.8
	Intermediate	52	80.7	58	75.8	1278	73.7
	Dark	10	74.9	9	77.0	326	73.2
ODIN (Liang et al., 2017)	Light	171	71.6	185	64.0	986	76.2
	Intermediate	52	69.9	58	64.9	1278	73.8
	Dark	10	86.3	9	89.4	326	72.1
SS ($C_s$)	Light	171	78.6	185	70.7	986	68.3
	Intermediate	52	87.0	58	71.3	1278	68.0
	Dark	10	87.6	9	69.5	326	68.6
SS ($C_s$)+ODIN	Light	171	69.7	185	62.7	986	52.1
	Intermediate	52	73.8	58	63.1	1278	50.6
	Dark	10	88.2	9	74.5	326	50.9
SS ($C_{con{v}_1}$) + ODINlow	Light	171	45.1	185	38.8	986	83.1
	Intermediate	52	49.9	58	37.8	1278	86.7
	Dark	10	63.6	9	68.4	326	87.2
SS (Sum All Layers) + ODINlow	Light	171	45.1	185	38.4	986	89.3
	Intermediate	52	51.8	58	40.0	1278	92.0
	Dark	10	56.2	9	78.7	326	92.3

Table 5: Performance of methods in Tables 3 and 4 stratified into three different skin tone categories. $R$ represents the number of OOD samples in each category.

6.3 Performance stratified by skin-tone

We further stratify the OOD samples into three skin tone categories and show the results in Table 5. In each set of columns, we include the number of test samples $R$ for each skin tone category and its corresponding AUROC performance. Samples of Dark skin tones constitute only around $3.9\%$ of DF and VASC samples and around $13\%$ of SD-198 samples. Majority of the listed methods (13 out of 18), show higher detection performance of Dark OOD samples. This could be partially because network is trained on the ISIC 2019 dataset that heavily lacks samples of dark skin tones, and thus easily detects OOD samples of dark skin tone to be out of distribution. Overall, it requires further investigation to clearly understand whether such performance reveals the lack of Dark samples in these datasets or variant manifestations of skin diseases in Dark skin.

SS	SS+ODIN	SS+ODINlow

Figure 3: AUROC performance of subset scanning (SS) across various layer of DenseNet-121 that we consider. First column shows the results without any ODIN, the other two columns show the result with ODIN and ODIN_low, respectively for OOD samples of DF (yellow), VASC (green), and SD-198 (red).

6.4 OOD detection across individual layers

Figure 3 shows the OOD detection performance in terms of AUROC of our proposed work on the eight layers of our pre-trained CNN in $C_Y$ that we consider. The first column shows the result of subset scanning without any added noise, and the other columns show the result of applying ODIN (Liang et al., 2017) and ODIN_low perturbations, respectively, to our test images before applying subset scanning. In each sub-plot, we show results of both use cases, i.e., detection of samples of unknown diseases (DF (yellow), VASC (green)) and samples from different protocols (SD-198 (red)). Overall, DF and VASC samples from ISIC 2019 dataset have similar performance across the eight layers we consider while samples from SD-198 dataset leads to varying performances depending on the layer and ODIN parameters. This is partly because DF and VASC samples are from the same distribution as the training set as they are both from the same ISIC 2019 dataset, while SD-198 has different distribution than the training set of ISIC 2019 with different collection protocol. Comparing the last two plots, we see that standard ODIN leads to better performance near the end of the network while ODIN_low leads to better performance in earlier layers of the network. This is as expected as ODIN parameters ($\tau$ and $ϵ$) are optimized on the Softmax Scores while ODIN_low parameters, ${\tau }_{low}$ and ${ϵ}_{low}$, are not.

SS	SS+ODIN	SS+ODINlow

Figure 4: Change in performance ($\mathrm{\Delta }$ AUROC) of OOD detection in Figure 3 for SD-198 (top), DF (middle) and VASC (bottom) stratified into three different skin-tone categories, Light (blue), Intermediate (magenta), and Dark (cyan). First column shows the results without any ODIN, the other two columns show the result with ODIN and ODIN_low, respectively.

We further stratify the performance of individual layers based on skin tone represented in the samples and show the change in AUROC with the stratification in Figure 4. While the samples of Light (blue) and Intermediate (magenta) skin tones show consistent performances throughout the layers, we see varying performances for samples of Dark (cyan) skin tones. This instability of performance for samples of Dark skin tones may be partially because network is trained on the ISIC 2019 dataset that heavily lacks samples of Dark skin tones.

7 Conclusion

We propose a weakly-supervised method to detect out-of-distribution (OOD) skin images (collected in different protocols or from unknown disease types) using input perturbation and scanning of the activations in the intermediate layers of pre-trained on-the-shelf classifier. The scanning of activations is optimised as a search problem to identify nodes in a layer that results in maximum divergence of the activations from subset of test samples compared to the expected activations derived from the training (in-distribution) samples. We exploited Linear Time Subset Scanning (LTSS) (Neill, 2012) property of subset scanning to achieve efficient search that scales linearly with the number of nodes in the a layer. Our proposed method improves on the state-of-the-art detection for OOD samples that are collected from a different protocol or equipment than those in-distribution samples used to train the classifier, and it achieves competitive performance with the state-of-the-art in detecting samples of unknown diseases. We further stratify these OOD samples based on three skin tone categories, Light, Intermediate, and Dark. From our results we observe imbalanced detection performance across skin tones, where the Dark samples are detected as OOD with higher performance. Thus, future work aims to understand the reasons for such detection disparity across skin tones, e.g., lack of training representation or different manifestation of skin diseases.