feat!: bump to aiobotocore 2.23.0

[brambaud]

Context

The actual version of aiobotocore, 0.12.0, is five years old.

In some cases (like aio-libs/aiobotocore#659), it produces errors like:

cannot pickle coroutine object
coroutine AioBaseClient._make_api_call was never awaited

These errors have been fixed since then.

Furthermore, installation of the dependencies take longer due to long search to find the compatible versions.

Suggestion

Let's update aiobotocore to its last available versions, right now 2.23.0.

There have been some breaking changes in aiobotocore like API breaking: The result of create_client is now a required async context class in 1.0.0

The naïve implementation of re-using the session and using context manager to create a new client each time like

    async with self._session.create_client('s3', ...) as client:
        resp = await client.put_object(Bucket=bucket, Key=key, Body=data)

is degrading performance at the point of having difficulties to support 50 requests per second during 60s (response time increases)

So I chose to use an asynchronous context manager to continue being able to reuse the client.

[Bladrak]

Hello @brambaud
Thanks for contributing! We had a PR to bump the version a few years ago, but didn't merge it due to performance issues (#157).
I see that you had performance in mind when submitting this PR, did you run some tests with it?

[brambaud]

This branch has been rebased too.

I see that you had performance in mind when submitting this PR, did you run some tests with it?

Yes, I ran some load tests because:

• I saw the previous attempt
• when testing https://github.com/thumbor/thumbor-aws I stumble upon the performance degradation
• when I did the upgrade, the breaking change in aibotocore was a hint that re-creating a client each time may be an issue

I don't have my machine with me right now to re-run them and paste the result because I'm on holiday but the load tests were something like:

import http from "k6/http";

export default function () {
 const response = http.get("https://localhost/unsafe/800x600/filters:quality(80)/some/path/img.png");
}

Without the fix, this small tests (50 requests per second during 60s) was broken

{
    "discardResponseBodies": true,
    "insecureSkipTLSVerify": true,
    "scenarios": {
      "test": {
        "executor": "ramping-arrival-rate",
        "startRate": 0,
        "timeUnit": "1s",
          "preAllocatedVUs": 50,
          "stages": [
          { "target": 50, "duration": "60s" }
        ]
      }
    }
  }

With the fix, it ran ok and a bigger tests was ok too:

{
    "discardResponseBodies": true,
    "insecureSkipTLSVerify": true,
    "scenarios": {
      "test": {
        "executor": "ramping-arrival-rate",
        "startRate": 0,
        "timeUnit": "1s",
          "preAllocatedVUs": 500,
          "stages": [
          { "target": 25, "duration": "60s" },
          { "target": 25, "duration": "180s" },
          { "target": 50, "duration": "60s" },
          { "target": 50, "duration": "180s" },
          { "target": 100, "duration": "60s" },
          { "target": 100, "duration": "180s" },
          { "target": 200, "duration": "60s" },
          { "target": 200, "duration": "180s" },
          { "target": 400, "duration": "60s" },
          { "target": 400, "duration": "180s" }
        ]
      }
    }
  }

For the moment, it has not been tested on another machine than mine.

PS: In order to share the knowledge, I need to propose a patch to https://github.com/thumbor/thumbor-aws to see if the performance issues are only due to the same naïve implementation of re-using the session and using context manager to create a new client each time

[Bladrak]

In any case it's long overdue we upgraded the dependencies, so good enough for me! Thanks!