OmniLabel Challenge 2024

In conjunction with our Workshop at ECCV 2024, we are hosting a challenge with the OmniLabel dataset.

OmniLabel benchmark: Novel dataset with complex, free-form text descriptions of objects. Checkout https://www.omnilabel.org and https://arxiv.org/abs/2304.11463 for details

Train on public datasets and evaluate on our benchmark! We define two tracks to compete, which differ in the allowed training data.

Validation set for the OmniLabel challenge 2024 is available (v0.1.4). Use our evaluation servers (Track A, Track B) to test your model on the validation set and participate in the challenge. Use our toolkit to evaluate a model on the validation set by yourself.

Valudation set for 2024 Omnilabel challenge

Tracks

Track A

Allowed training datasets:
- Object detection: COCO-train17, Objects-365-train
- Referring/Grounding: RefCOCO/+/g, Flickr30k, Visual Genome (like GoldG+ in GLIP, or as defined in MDETR)
- Captions: CC3M, SBU, COCO-captions
- Others: HMDB-51, VAW-attributes
Allowed pre-trained models: Publicly available models only (e.g., CLIP)

Track B

Allowed training datasets: Any
Allowed pre-trained models: Any

Download the data
Check out the task definitions
Compete on the evaluation servers
- Track A: https://codalab.lisn.upsaclay.fr/competitions/19200
- Track B: https://codalab.lisn.upsaclay.fr/competitions/19201