Download annotations

We kindly ask you to join our Google Group to obtain links for downloading the annotations.

Joining is free, without delay (no approval), and we won't spam you with emails. We will only inform you about updates to the dataset.

Download images

OmniLabel uses images from COCO (2017 version), Objects365, and OpenImages v5. Please follow their instructions to prepare the images. You will only need the images of the validation (COCO & Objects365) and test (OpenImages) splits. If you have already downloaded these datasets, you only need to download our OmniLabel annotations (see above).

Dataset structure

Our annotation files store image paths relative to a base image directory, similar to the COCO dataset. The content of this base directory must follow this structure:


    coco/              # points to val2017 directory of COCO dataset

    object365/         # points to val directory of Objects365 dataset

    openimagesv5/      # points to test directory of Open-Images-V5 dataset

Make sure the subdirectory names are correct, because these are part of the annotation files!

If you already have the dataset downloaded somewhere, the above structure can be easily created with symbolic links:


mkdir -p $BASE

cd $BASE

ln -s path/to/coco17/val2017 coco

ln -s path/to/objects365/val object365

ln -s path/to/openimagesv5/test openimagesv5

NB: The structure may change once the test set is finalized ...

Annotation format

Our annotations are stored as json files and have the following format:


    images: [


            id              ... unique image ID

            file_name       ... path to image, relative to a given base directory (see above)




    descriptions: [


            id              ... unique description ID

            text            ... the text of the object description

            image_ids       ... list of image IDs for which this description is part of the label space

            anno_info       ... some metadata about the description




    annotations: [        # Only for val sets and result files. Not given in test set annotations!


            id              ... unique annotation ID

            image_id        ... the image id this annotation belongs to

            bbox            ... the bounding box coordinates of the object (x,y,w,h)

            description_ids ... list of description IDs that refer to this object






All annotations are licensed under a Creative Commons Attribution 4.0 License. All images in the OmniLabel benchmark are from COCO, Objects365, and OpenImages v5. Please see their terms of use: COCO, Objects365, OpenImages v5.