competition dump with files

[ihsaan-ullah]

when you create a dump (by default it is created with files) but you can still see the keys in the yaml file.

[Didayolo]

I am a bit confused because I thought we had the dump with files but not the dump with keys. See:

• [Feature] Option to dump with keys #880

I think only one of the two options is working right now. We should test and merge the two issues into one.

[ihsaan-ullah]

I just tested and the key is for a task. So this issue can be closed

[Didayolo]

But even if it is for a task, you should not have keys if you create a dump with files.

[ihsaan-ullah]

I was thinking the keys are there for datasets only. But now there are keys for tasks too. But there are no files for tasks.

[Didayolo]

I confirm that when you dump using files and then re-upload the bundle, the tasks are linked to the previous tasks. No new tasks are created (see screenshot).

I think we would like that:

• For dumps with keys, we link to the previous tasks,
• For dump with files, we create new tasks from scratch using the files.

[ihsaan-ullah]

When you create a dump with files, you can see a key for each task in YAML file. If you remove that key and upload the bundle, do you still see the new competition linked to already existing tasks?

[ihsaan-ullah]

This issue needs some discussion.

1. why we have key in tasks when we create dump.
2. If an already existing task can be linked through a key then why do we have keys for the datasets (when creating dumps with keys)
3. What should be done for both cases: with keys, with files

[Didayolo]

• For the files case, we want the bundle to be self-contained. Therefore, we should not store the tasks keys in this case,
• For the keys case, I think it is OK to keep both tasks keys and programs/data keys, but we should do some testing to so how it behaves.

[ihsaan-ullah]

Tests

Download Dump with files OK

• Task in yaml has keys
• Uploading this bundle -> uses already existing tasks

Download Dump with files OK

• Task in yaml has keys
• Delete the tasks
• Uploading this bundle -> gives error

Download Dump with files OK

• Task in yaml has keys -> remove keys
• Uploading this bundle -> this competition uses new tasks

Download Dump with keys OK

• Task in yaml has keys
• Dataset in yaml has keys
• Uploading this bundle -> uses already existing tasks

Download Dump with keys OK

• Task in yaml has keys -> remove task keys
• Dataset in yaml has keys
• Uploading this bundle -> creates new tasks

Download Dump with keys OK

• Task in yaml has keys
• Dataset in yaml has keys -> remove dataset keys
• Uploading this bundle -> uses already existing tasks

Problem

The only problem I see is that when you create dump with files, for tasks keys should not be there