Mayank022/sam-vit-base-kvasir-polyp-segmentation

SAM Fine-tuned on Kvasir-SEG for Polyp Segmentation

Fine-tuned facebook/sam-vit-base on Kvasir-SEG for gastrointestinal polyp segmentation.

Model Details

Base Model	facebook/sam-vit-base (93.7M params)
Task	Binary polyp segmentation from endoscopy images
Strategy	Fine-tune mask decoder, freeze vision encoder + prompt encoder
Prompts	Bounding box (with random ±20px perturbation)
Dataset	Kvasir-SEG: 880 train / 120 val images
Loss	DiceCELoss (MONAI)
Optimizer	AdamW (lr=1e-05, wd=0.01)
Scheduler	Cosine Annealing
Epochs	30
Best Val Dice	0.8355

Usage

from transformers import SamModel, SamProcessor
from PIL import Image
import torch, numpy as np

model = SamModel.from_pretrained("Mayank022/sam-vit-base-kvasir-polyp-segmentation")
processor = SamProcessor.from_pretrained("Mayank022/sam-vit-base-kvasir-polyp-segmentation")

image = Image.open("polyp.jpg").convert("RGB")
input_boxes = [[[100, 100, 400, 400]]]  # bounding box prompt

inputs = processor(image, input_boxes=input_boxes, return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs, multimask_output=False)

mask = (torch.sigmoid(outputs.pred_masks.squeeze()) > 0.5).cpu().numpy().astype(np.uint8)

References

• SAM: Kirillov et al., "Segment Anything" (2023)
• Kvasir-SEG: Jha et al. (2020)
• MedSAM: Ma et al. (2024)