Codabench Datasets

[ihsaan-ullah]

Datasets updates

RELEASE 1 (Done, Solved by #1955):

• new page for public datasets
• filters like competitions page for searching
• new properties in datasets page (name, description, uploader, uploaded when, size, downloads, verified

RELEASE 2 (Done, Solved by #2050)

• add croissant metadata for public datasets
• fix license not added to db on dataset creation

RELEASE 3 (To be discussed)

• Field for citation (also in json-ld)
• Field for version (also in json-ld)
• Documentation (Markdown) about data format and reading, source of data, references, etc.
• Type (tabular, images etc)
• Size (number of examples)
• Tags (multiple)
• Mechanism to update verified flag (verified by admins for quality and safety)
• How to handle revisions or versioning of datasets?
  It should not be possible to update verified datasets, or at least updating should break the verified flag.

Suggestions (from Adrien)

• Having an unique DOI for each dataset
• Progress bar during upload

[ihsaan-ullah]

Resources:

• https://mlcommons.org/2024/03/croissant_metadata_announce/
• https://github.com/mlcommons/croissant/issues?q=state%3Aopen%20label%3A%221.1%22
• https://github.com/mlcommons/croissant/blob/main/docs/croissant-spec-draft.md

Example from OpenML

• https://www.openml.org/search?type=data&status=active&id=44314

[Didayolo]

Check Croissant format

• https://github.com/mlcommons/croissant
• https://mlcommons.org/working-groups/data/croissant/
• https://research.google/blog/croissant-a-metadata-format-for-ml-ready-datasets/

[ihsaan-ullah]

The new datasets page will look like this

What will be the dataset type when we give an upload option (input data, public data, reference data)

[ihsaan-ullah]

User sidebar will look like this.
In next update:

• profile and edit profile will merge, profile will be renamed to account that will have profile, edit profile, change password,
• notifications will be merged in settings. Settings will have notifications and delete account
• Organizations will move to account section
• On the top we will have account settings, then resources, then admin procedures, then logout

Main menu

[ihsaan-ullah]

• For now we decided to use only public datasets in the new datasets page

[Didayolo]

Question about this feature: is there a limit on the dataset size to be uploaded? And/or is the quota applied when trying to upload?