Fixed content-handler response-headers/object-storage collision.

[ggainey]

fixes #4028.

[lubosmj]

This might be more favourable to store in the constants.py file. Is it worth caching the return value of _set_params_from_headers (the parameters variable will be returned instead of overwriting it inside the function)?

[ggainey]

These both sound like fine ideas, will do!

[mdellweg]

When we call this function, we have identified the storage class already, so i think we can spare ourselves one if and dict lookup by providing the response map instead of the storage class name.

def _set_params_from_headers(headers, response_header_map):
    params = {}
    for key, new_key in response_header_map.items():
        if key in headers:
            params[new_key] = headers[key]

or even:

def _set_params_from_headers(headers, response_header_map):
    return {new_key: headers[key] for key, new_key in response_header_map.items() if key in headers}

We'd call it by, e.g. in the s3 branch:

_set_params_from_headers(headers, S3_RESPONSE_HEADER_MAP)

[ggainey]

Passing the mapping-dictionary "assumes" that a simple lookup is all we're ever going to want to do. If we want/need to do anything more complicated in the future, the current approach lets us do so in a single place.

In terms of performance - given the current state of the content-handler, sparing "one dictionary lookup" is not convincing.

[mdellweg]

I was thinking about my performance in understanding the code. But it's not going to block this contribution.

[lubosmj]

I like the change!

Optionally, we could remove the if storage_domain in STORAGE_RESPONSE_MAP condition and convert it to a try/except block for the for loop (https://www.oreilly.com/library/view/python-essentials/9781784390341/ch09s08.html).

[ggainey]

I like the change!

Optionally, we could remove the if storage_domain in STORAGE_RESPONSE_MAP condition and convert it to a try/except block for the for loop (https://www.oreilly.com/library/view/python-essentials/9781784390341/ch09s08.html).

I've heard that argument before, but personally feel exceptions should be for exceptional issues - not a main-flow "is this one of the things we know how to deal with?" path. Speed doesn't impress me either - no matter how fast python is at exceptions, building one, executing the jump, and then processing it, isn't as fast as "single key lookup into a (tiny) dictionary".

"try it and see" makes more sense the more complicated the possibilities are.

(Or it makes sense until you start fielding "OMG FIX THIS ERROR" issues from users seeing database-logging saying "you can't do that" for things you handle via "ask forgiveness" processing...)

[patchback]

Backport to 3.28: 💚 backport PR created

✅ Backport PR branch: patchback/backports/3.28/4dd2edf6edaf2cdc8a0cb2881e00f6c0bc9007b9/pr-4251

Backported as #4265

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

[patchback]

Backport to 3.29: 💚 backport PR created

✅ Backport PR branch: patchback/backports/3.29/4dd2edf6edaf2cdc8a0cb2881e00f6c0bc9007b9/pr-4251

Backported as #4266

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.