Cell Behavior Video Classification Challenge

Quick links

• Dataset (phase1 and phase 2)
• Code (GitHub repository)

Leaderboard

• View Challenge leaderboard
• Submit your results *

🔍 Introduction and Goal

The Cell Behavior Video Classification Challenge (CBVCC) is designed to develop or adapt computer vision methods for classifying videos capturing cell behavior through Intravital Microscopy (IVM). IVM is a powerful imaging technique that allows for non-invasive visualization of biological processes in living animals. Platforms such as two-photon microscopes exploit multiple low-energy photons to deliver high-resolution, three-dimensional videos depicting tissues and cells deep within the body. IVM has been used to visualize a wide range of biological processes, including immune responses, cancer development, and neurovascular function.

The primary goal of the CBVCC challenge is to create models that can accurately classify videos based on the movement patterns of cells. Specifically, the models should be capable of:

• Identifying videos where cells exhibit sudden changes in their direction of movement.
• Distinguishing these from videos where cells show consistent, linear movement, stationary cells, or videos containing only background.

The CBVCC challenge aims to provide a platform for researchers to develop innovative methods for classifying IVM videos, potentially leading to new insights into biological processes.

👩‍🔬👨‍🔬 The CBVCC Challenge

The CBVCC challenge is open to researchers worldwide. Participants are tasked with developing computational models to classify IVM videos. The models will be evaluated based on their accuracy in classifying videos.

The challenge is divided into two phases:

• Phase 1: Participants will be provided with a training dataset and a test dataset of IVM images and labels to develop and evaluate their models.
• Phase 2: Participants will submit their results obtained on a distinct test dataset of IVM images. Models will be ranked based on their performance on this test dataset.

The final results will be determined based on the performance of the Phase 2 submissions.

🗓 The Challenge Timeline

• September 15th, 2024 – November 15th, 2024: Registration period. Participants can register at this link.
• November 15th, 2024: Training dataset released; validation leaderboard becomes available.
• December 13th, 2024: Unannotated test dataset released.
• December 20th, 2024: Challenge closes. Participants must submit predictions on the test set, an abstract describing their methodology, and code or a Docker container to replicate results.
• January 7th, 2025: Online workshop - Results announced and published online.

Description: The CBVCC dataset comprises 2D video patches extracted from IVM videos capturing the behavior of regulatory T cells in the abdominal flank skin of mice undergoing a contact sensitivity response to the sensitizing hapten, oxazolone. Videos are acquired either 24 or 48 hours after the initial skin challenge, with each sequence lasting 30 minutes and images captured at one-minute intervals, resulting in 31 acquisitions per video.
Dataset license: CC-BY
How to cite: Cabini, Raffaella Fiamma, et al. "Cell Behavior Video Classification Challenge, a benchmark for computer vision methods in time-lapse microscopy." arXiv preprint arXiv:2601.10250 (2026).

The dataset includes:

• Class 1: Video patches where cells suddenly change their direction of movement.
• Class 0: Video patches without sudden changes in cell direction, including linear movement, stationary cells, or background-only videos.

The dataset consists of 300 2D video patches (180 class 0 and 120 class 1) extracted from 48 different videos. Each patch is a 2D projection along the z-axis of a 3D video sequence, adjusted to a common contrast range to enhance visibility. All videos have been preprocessed to ensure a uniform pixel size of 0.8 µm and are saved as RGB .avi files.

Dataset splits:

• Training set: 210 labeled video patches (70%) – Download
• Phase 1 test set: 30 labeled video patches (10%) – Download
• Phase 2 test set: 60 unlabeled video patches (20%) – Download

Each subset includes video patches extracted from different and independent IVM videos. In class 1, the change of direction occurs at the midpoint and center of the video patches.

Additional files (not provided during the challenge):

• Ground truth labels for test set (phase 2) – Download
• Manual tracking - Download

📊 Evaluation

Models will be evaluated using the following metrics:

• Area Under the ROC Curve (AUC)
• Precision
• Recall
• Balanced Accuracy

The final score is calculated as:

Score = 0.4 × AUC + 0.2 × (Precision + Recall + Balanced Accuracy)
Missing entries are automatically attributed with a wrong class.

The challenge is now closed.
To submit the results of your method, prepare a CSV file indicating the video id and the cass-1 probability [from 0 to 1].
(e.g.
02_12.avi,0.48
02_4.avi,0.65
02_3.avi,0.84
...
)
An example can be found here: https://www.dp-lab.info/cbvcc/sample_file.csv

Link to submit your results on the validation set (phase 1): https://www.dp-lab.info/cbvcc/data/submitpost1.php
Link to submit your results on the test set (phase 2): https://www.dp-lab.info/cbvcc/data/submitpost2.php

Rules for participation

• Only automated or semiautomated methods are allowed, manual annotation of the test datasets is strictly prohibited.
• Participants are allowed a single submission on the test-set and agree to submit to phase 2 their only best-performing model on the validation-set (phase 1)
• The use of external datasets or pretrained models is allowed, provided they did not involve the same classification task as the CBVCC challenge.
• Any external resources has to be disclosed to the organizers.
• Members of the organizing committee are eligible to participate, subject to the same regulations as all other teams.
• Participants will be included in the manuscript reporting the challenge results. Up to 5 authors per team are allowed.
• No embargo on result publication is applied

Leaderboard - Validation set (phase 1)

Leaderboard - Test set (phase 2)