Convpaint - Semantic segmentation using pretrained AI models

Convpaint - Semantic segmentation using pretrained AI models | Roman Schwob | November 18th, Halfway to I2K 2025 Authors: Lucien Hinderling, Institute of Cell Biology, University of Bern; Roman Schwob, Data Science Lab, University of Bern; Guillaume Witz, Data Science Lab, University of Bern Workshop Description: We develop Convpaint, a universal computational framework for interactive pixel classification. Convpaint uses pretrained convolutional neural networks (CNNs), vision transformers (ViTs) or classical filter banks for feature extraction in combination with fast-to-train machine learning (ML) classifiers to enable easy segmentation across a wide variety of tasks. By integrating ViT-based features, Convpaint extends traditional pixel classification to image domains that require rich semantic understanding. Convpaint's modular design allows users to rapidly switch between feature extractors, balancing speed, spatial accuracy, and semantic depth based on the specific dataset. 1. Install some type of Conda environment manager (I recommend miniconda, but any alternative will work: https://www.anaconda.com/docs/getting-started/miniconda/main) 2. Install the Napari image viewer, ideally in a fresh Conda environment: https://napari.org/stable/tutorials/fundamentals/installation.html 3. Make yourself a bit familiar with handling Napari; if you know what "image" and "labels" layers are that's already great: https://napari.org/stable/tutorials/fundamentals/viewer.html 4. Install Convpaint, either with `pip install napari-convpaint` into the environment with Napari, OR directly in a Napari viewer through the Plugins menu: https://guiwitz.github.io/napari-convpaint/book/Installation.html 5. Do a quick test that you can open the Convpaint Plugin Target Audience: Beginner users, intermediate users, advanced users Keywords: Segmentation, Pixel Classification, Artificial Intelligence, Machine Learning, CNNs, Vision Transformers, Napari, plug-in, API, multi-dimensional data