Submission appear to fail after 24h

[jpbrunetsx]

Hi,
In our competition we are running into an issue with failed submission with what appears to be a timeout.
We did increase the running time to 72h in the task itself.

Competition : 3282 https://www.codabench.org/competitions/3282/
Exemple submission with a problem : 95068

Description:
The submission fails after around 24h with the following error :

`https://miniodis-rproxy.lisn.upsaclay.fr/coda-v2-prod-private/submission_details/2024-10-22-1729621719/e8ba39ca3b12/prediction_ingestion_stdout.txt?AWSAccessKeyId=EASNOMJFX9QFW4QIY4SL&Signature=RenMoB7mHR2cs23sT7lGFLoAGz8%3D&content-type=application%2Fzip&Expires=1729708119

[2024-10-23 19:32:13,673: INFO/ForkPoolWorker-3] *** PUT RESPONSE: ***
[2024-10-23 19:32:13,674: INFO/ForkPoolWorker-3] response: <Response [403]>
[2024-10-23 19:32:13,674: INFO/ForkPoolWorker-3] content: b'<?xml version="1.0" encoding="UTF-8"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>submission_details/2024-10-22-1729621719/e8ba39ca3b12/prediction_ingestion_stdout.txt</Key><BucketName>coda-v2-prod-private</BucketName><Resource>/coda-v2-prod-private/submission_details/2024-10-22-1729621719/e8ba39ca3b12/prediction_ingestion_stdout.txt</Resource><RequestId>18012BD6DA5A37FD</RequestId><HostId>e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855</HostId></Error>'`

A shorter version running sub 24h does not trigger the issue.

[Didayolo]

Hi, thank you for reporting this issue.

@ObadaS Do you think we have some kind of 24h timeout setting in MinIO?

[ObadaS]

Hello,
The MinIO link in the error does seem to have an expiry date at the end.
I think that links created by the code have a set expiry time which seems to be lower than 24 hours. I think you would need to alter something in the code to request a longer expiry.

I will continue searching on my side and will update this issue if I find something else.

[ObadaS]

I seem to have found the code that creates the URLs

codabench/src/utils/data.py

Line 76 in 1bd7c6f

return BundleStorage.bucket.meta.client.generate_presigned_url(

The duration is 24h, so it should be possible to change it there, or make it variable somehow

codabench/src/utils/data.py

Line 46 in 1bd7c6f

def make_url_sassy(path, permission='r', duration=60 * 60 * 24, content_type='application/zip'):

[Didayolo]

Very nice catch. Should we simply put a higher limit? For instance 4 days? Or can this have bad consequences?

[ObadaS]

Could it be possible to link it to the competition ? So that they can decide to make it higher if needed (up to a hard limit of course) without having every link for all competitions be upgraded to 4 days since it's rarely needed

[Didayolo]

Should be fixed by #1659, soon to be deployed