Medical SAM Adapter++: Adaptive Normalization with Domain Interpretation for Better Adaptation

Vision Transformer (ViT) models have spearheaded the computer vision community for various image-related tasks, specifically showing significant results in interactive segmentation tasks using Segment Anything Model (SAM). Adapters can be incorporated into SAM for adaptation to the different domains, whilst significantly reducing training costs compared to the fine-tuning of the entire model. This was especially true for datasets from the medical domain, where it presented results superior to state-of-the-art models. However, the simple incorporation of adapters fails to leverage the statistical differences between the probability distribution of data from various domains. This paper addresses this issue by introducing Adaptive Layer Normalization into SAM, which has more theoretical background as a solution to domain differences than sequential adapters. Furthermore, this paper also proposes a novel method, domain interpreter, for determining the weights to Adaptive Layer Normalization. Focusing on the Osteoarthritis (OAI) Dataset, the proposed method shows improved results in IoU and DICE metric scores, showing that the distinct domain distributions are the cause of inaccuracies of baseline models, and that the models can quickly be adapted to different domains by adjusting to such data distributions.