APIGatewayEventRequestContext have confusing or wrong behavior on "get" and "raw_event" properties

[ricardoinacio]

I needed access to key operationName from requestContext from APIGatewayProxyEvent. The problem is that APIGatewayEventRequestContext doesn't have this property, forcing me to use either the get accessor or the raw_event property fallback. And the real problem is that when you use any of these to access the original data, but from the APIGatewayEventRequestContext, it does in fact give you the root APIGatewayProxyEvent event data. So I had lots of confusing behavior and lost time with debugging.

Eg.

def handler(event, context):
    print(event['requestContext']['operationName'])  # prints "some-openapi-operation-name"
    
    apigw_event = APIGatewayProxyEvent(event)
    print(apigw_event.request_context.get('operationName'))  # prints "None" from returned None from getter
    print(apigw_event.request_context.raw_event['operationName'])  # raises KeyError

Expected Behavior

I expected both apigw_event.request_context.get('operationName') and print(apigw_event.request_context.raw_event['operationName']) to return "some-openapi-operation-name".

Current Behavior

Both give errors because they access root self._data: DictWrapper instead of self._data['requestContext'].

Environment

• Powertools version used:
• Packaging format (Layers, PyPi):
• AWS Lambda function runtime:
• Debugging logs

How to enable debug mode**

$ python3 --version
Python 3.8.6
$ pip list
Package                Version
---------------------- ---------------
apipkg                 1.5
appdirs                1.4.4
arrow                  0.17.0
astroid                2.4.2
atomicwrites           1.4.0
attrs                  20.3.0
autopep8               1.5.4
aws-lambda-builders    1.3.0
aws-lambda-powertools  1.13.0
aws-sam-cli            1.21.1
aws-sam-translator     1.35.0
aws-xray-sdk           2.7.0
Babel                  2.8.0
beautifulsoup4         4.9.1
binaryornot            0.4.4
black                  20.8b1
blinker                1.4
boto3                  1.14.63
botocore               1.17.63
breezy                 3.1.0
Brlapi                 0.7.0
certifi                2020.4.5.1
chardet                3.0.4
chevron                0.13.1
chrome-gnome-shell     0.0.0
click                  7.1.2
colorama               0.4.3
command-not-found      0.3
configobj              5.0.6
cookiecutter           1.7.2
coverage               5.3
cryptography           3.0
cupshelpers            1.0
dateparser             0.7.6
dbus-python            1.2.16
decorator              4.4.2
defer                  1.0.6
Deprecated             1.2.10
distro                 1.5.0
distro-info            0.23ubuntu1
dnspython              1.16.0
docker                 4.2.2
docopt                 0.6.2
docutils               0.15.2
dulwich                0.19.15
evdev                  1.3.0
execnet                1.7.1
fastimport             0.9.8
fastjsonschema         2.15.0
fett                   0.3.2
Flask                  1.1.2
future                 0.18.2
gpg                    1.14.0-unknown
gyp                    0.1
hidpidaemon            18.4.6
html5lib               1.1
httplib2               0.18.1
idna                   2.10
importlib-metadata     1.6.0
iniconfig              1.1.1
isort                  5.7.0
itsdangerous           1.1.0
jeepney                0.4.3
Jinja2                 2.11.2
jinja2-time            0.2.0
jmespath               0.10.0
jsonschema             3.2.0
kernelstub             3.1.4
keyring                21.3.0
language-selector      0.1
launchpadlib           1.10.13
lazr.restfulclient     0.14.2
lazr.uri               1.0.5
lazy-object-proxy      1.4.3
libvirt-python         6.1.0
louis                  3.14.0
lxml                   4.5.2
macaroonbakery         1.3.1
MarkupSafe             1.1.1
mccabe                 0.6.1
mercurial              5.5.1
more-itertools         4.2.0
mypy-extensions        0.4.3
netifaces              0.10.4
networkx               2.5
numpy                  1.18.4
oauthlib               3.1.0
olefile                0.46
packaging              20.4
parameterized          0.7.4
pathspec               0.8.1
pathtools              0.1.2
patiencediff           0.1.0
pep8                   1.7.1
Pillow                 7.2.0
pip                    21.0.1
pluggy                 0.13.0
pop-transition         1.1.2
poyo                   0.5.0
protobuf               3.12.3
py                     1.9.0
pycairo                1.16.2
pycodestyle            2.6.0
pycups                 2.0.1
pydbus                 0.6.0
PyGithub               1.43.7
PyGObject              3.38.0
PyJWT                  1.7.1
pylint                 2.6.0
pymacaroons            0.13.0
pymongo                3.11.2
PyNaCl                 1.4.0
pynamodb               5.0.3
pyparsing              2.4.7
pyRFC3339              1.1
pyrsistent             0.17.3
pytest                 6.1.1
pytest-cov             2.10.1
pytest-forked          1.3.0
pytest-rerunfailures   9.1.1
pytest-timeout         1.4.2
pytest-xdist           2.1.0
python-apt             2.1.3+ubuntu1.3
python-dateutil        2.8.0
python-debian          0.1.37
python-gitlab          2.4.0
python-jsonrpc-server  0.3.4
python-magic           0.4.16
python-slugify         4.0.1
python-xlib            0.27
pytz                   2020.1
pyxdg                  0.26
PyYAML                 5.3.1
regex                  2020.11.13
repolib                1.5.2
repoman                1.4.0
requests               2.23.0
requests-unixsocket    0.2.0
rstcheck               3.3.1
s3transfer             0.3.4
scour                  0.37
SecretStorage          3.1.2
serverlessrepo         0.1.10
sessioninstaller       0.0.0
setproctitle           1.1.10
setuptools             49.3.1
simplejson             3.17.0
six                    1.15.0
snooty-lextudio        1.8.6.dev0
soupsieve              2.0.1
SQLAlchemy             1.3.19
systemd-python         234
text-unidecode         1.3
toml                   0.10.1
tomlkit                0.7.0
typed-ast              1.4.2
typing-extensions      3.7.4.3
tzlocal                2.1
ubuntu-advantage-tools 24.4
ubuntu-drivers-common  0.0.0
ufw                    0.36
ujson                  1.35
urllib3                1.25.9
wadllib                1.3.4
watchdog               0.10.3
wcwidth                0.1.9
webencodings           0.5.1
websocket-client       0.57.0
Werkzeug               1.0.1
wheel                  0.34.2
wrapt                  1.11.2
xkit                   0.0.0
zipp                   1.0.0

[michaelbrewer]

@ricardoinacio yes this should help fix this:

    @property
    def request_context(self) -> APIGatewayEventRequestContext:
        return APIGatewayEventRequestContext(self._data["requestContext"])

Could you do me a favor and include a sample event with operationName ?

[michaelbrewer]

@ricardoinacio @heitorlessa

part of the confusion of raw_event is that we added this as a property to get back the original wrapper event dict (eg: #340)

However sometimes it helps for the items inside of a data class to be getting a sub node within the main event dict, so then raw_event would not match the original. And when we don't get a sub node, then raw_event and .get( is scoped within the original event.

Either way, i can fix this behavior, but i just want to be sure we have a good standard going ahead (and in this case an example that include operationName)

[michaelbrewer]

@ricardoinacio i added the missing field: #373

[heitorlessa]

@ricardoinacio this will be out in our next release in April, hopefully in two weeks time if we manage to include the new event_handler utility on time. I'll ping back here once it's out.

Thank you again for raising this so we can iron the edges out!

[ricardoinacio]

@michaelbrewer @heitorlessa Thank you, guys! I think the solution is great, and should help.

Also, maybe APIGatewayEventRequestContext should offer a reference to APIGatewayProxyEvent from where we could access raw_event. It seemed unnatural to me to be able to get the root of the event from a child object, but to be honest, I've been using this library (which is great) for a little more than a week, so it could be just me 😆.

Here is the base event I use for local testing.

{
  "body": null,
  "resource": "/resources",
  "path": "/resources",
  "httpMethod": "GET",
  "isBase64Encoded": false,
  "queryStringParameters": {
    "foo": "bar"
  },
  "multiValueQueryStringParameters": {
    "foo": [
      "bar"
    ]
  },
  "pathParameters": null,
  "stageVariables": null,
  "headers": {
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
    "Accept-Encoding": "gzip, deflate, sdch",
    "Accept-Language": "en-US,en;q=0.8",
    "Cache-Control": "max-age=0",
    "CloudFront-Forwarded-Proto": "https",
    "CloudFront-Is-Desktop-Viewer": "true",
    "CloudFront-Is-Mobile-Viewer": "false",
    "CloudFront-Is-SmartTV-Viewer": "false",
    "CloudFront-Is-Tablet-Viewer": "false",
    "CloudFront-Viewer-Country": "US",
    "Host": "1234567890.execute-api.us-east-1.amazonaws.com",
    "Upgrade-Insecure-Requests": "1",
    "User-Agent": "Custom User Agent String",
    "Via": "1.1 08f323deadbeefa7af34d5feb414ce27.cloudfront.net (CloudFront)",
    "X-Amz-Cf-Id": "cDehVQoZnx43VYQb9j2-nvCh-9z396Uhbp027Y2JvkCPNLmGJHqlaA==",
    "X-Forwarded-For": "127.0.0.1, 127.0.0.2",
    "X-Forwarded-Port": "443",
    "X-Forwarded-Proto": "https"
  },
  "multiValueHeaders": {
    "Accept": [
      "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"
    ],
    "Accept-Encoding": [
      "gzip, deflate, sdch"
    ],
    "Accept-Language": [
      "en-US,en;q=0.8"
    ],
    "Cache-Control": [
      "max-age=0"
    ],
    "CloudFront-Forwarded-Proto": [
      "https"
    ],
    "CloudFront-Is-Desktop-Viewer": [
      "true"
    ],
    "CloudFront-Is-Mobile-Viewer": [
      "false"
    ],
    "CloudFront-Is-SmartTV-Viewer": [
      "false"
    ],
    "CloudFront-Is-Tablet-Viewer": [
      "false"
    ],
    "CloudFront-Viewer-Country": [
      "US"
    ],
    "Host": [
      "0123456789.execute-api.us-east-1.amazonaws.com"
    ],
    "Upgrade-Insecure-Requests": [
      "1"
    ],
    "User-Agent": [
      "Custom User Agent String"
    ],
    "Via": [
      "1.1 08f323deadbeefa7af34d5feb414ce27.cloudfront.net (CloudFront)"
    ],
    "X-Amz-Cf-Id": [
      "cDehVQoZnx43VYQb9j2-nvCh-9z396Uhbp027Y2JvkCPNLmGJHqlaA=="
    ],
    "X-Forwarded-For": [
      "127.0.0.1, 127.0.0.2"
    ],
    "X-Forwarded-Port": [
      "443"
    ],
    "X-Forwarded-Proto": [
      "https"
    ]
  },
  "requestContext": {
    "operationName": "list-resources",
    "accountId": "123456789012",
    "resourceId": "123456",
    "stage": "dev",
    "requestId": "c6af9ac6-7b61-11e6-9a41-93e8deadbeef",
    "requestTime": "09/Apr/2015:12:34:56 +0000",
    "requestTimeEpoch": 1428582896000,
    "identity": {
      "cognitoIdentityPoolId": null,
      "accountId": null,
      "cognitoIdentityId": null,
      "caller": null,
      "accessKey": null,
      "sourceIp": "127.0.0.1",
      "cognitoAuthenticationType": null,
      "cognitoAuthenticationProvider": null,
      "userArn": null,
      "userAgent": "Custom User Agent String",
      "user": null
    },
    "path": "/dev/resources",
    "resourcePath": "/resources",
    "httpMethod": "GET",
    "apiId": "1234567890",
    "protocol": "HTTP/1.1"
  }
}

[michaelbrewer]

@ricardoinacio To be honest, the DictWrapper usage was out of convenience internally, so APIGatewayEventRequestContext uses it to reference the root dict keys.

raw_event was added later to allow for you to access the original event root dict.

[ricardoinacio]

Thank you guys, and great job!