Same dataset can be uploaded multiple times

[debayan]

I think that the exact same database should not be allowed to be uploaded multiple times. Currently, if I upload the same dataset twice, is returns new IDs for both. I think on the server side, there should be a check (maybe md5sum hash based) to not allow the exact same content to re-uploaded.

[Shamarvey1]

Hi @debayan, I’d like to work on this issue.

From an initial look, it seems duplicate uploads occur because there is no content-based validation before assigning a new dataset ID.

Proposed approach:

• Compute a hash (e.g., MD5 or SHA256) of the uploaded file

• Store the hash along with dataset metadata

• On upload, check if the hash already exists:

  • If yes, return the existing dataset ID instead of creating a duplicate
  • If no, proceed with storing the dataset

I’ll first explore the current upload flow and share a more concrete implementation plan before proceeding.

Please let me know if this approach aligns with the intended behavior.