Unclosed client session + Unclosed connector warnings

[quetz]

What happened?

Creating two AsyncWeb3 instances results in warnings on app exit.

Code that produced the error

#!python3

import asyncio
from web3 import AsyncWeb3, HTTPProvider
from web3.providers import AsyncHTTPProvider

async def main():
    rpc = "https://mainnet.optimism.io"
    w3_1 = AsyncWeb3(AsyncHTTPProvider(rpc))
    await w3_1.eth.block_number
    w3_2 = AsyncWeb3(AsyncHTTPProvider(rpc))
    await w3_2.eth.block_number

asyncio.run(main())

Full error output

Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x103fc0ad0>
Unclosed connector
connections: ['[(<aiohttp.client_proto.ResponseHandler object at 0x103f5f9b0>, 8352140.600221)]']
connector: <aiohttp.connector.TCPConnector object at 0x103fc0c20>

Fill this section in if you know how this could or should be fixed

No response

web3 Version

7.4.0

Python Version

3.13.0

Operating System

osx

Output from pip freeze

aiohappyeyeballs==2.4.3
aiohttp==3.10.10
aiosignal==1.3.1
annotated-types==0.7.0
asttokens==2.4.1
attrs==24.2.0
bitarray==3.0.0
certifi==2024.8.30
charset-normalizer==3.4.0
ckzg==2.0.1
cytoolz==1.0.0
decorator==5.1.1
eth-account==0.13.4
eth-hash==0.7.0
eth-keyfile==0.8.1
eth-keys==0.6.0
eth-rlp==2.1.0
eth-typing==5.0.1
eth-utils==5.1.0
eth_abi==5.1.0
executing==2.1.0
frozenlist==1.5.0
hexbytes==1.2.1
idna==3.10
ipython==8.28.0
jedi==0.19.1
matplotlib-inline==0.1.7
multidict==6.1.0
parsimonious==0.10.0
parso==0.8.4
pexpect==4.9.0
prompt_toolkit==3.0.48
propcache==0.2.0
ptyprocess==0.7.0
pure_eval==0.2.3
pycryptodome==3.21.0
pydantic==2.9.2
pydantic_core==2.23.4
Pygments==2.18.0
pyunormalize==16.0.0
regex==2024.9.11
requests==2.32.3
rlp==4.0.1
six==1.16.0
stack-data==0.6.3
toolz==1.0.0
traitlets==5.14.3
types-requests==2.32.0.20241016
typing_extensions==4.12.2
urllib3==2.2.3
wcwidth==0.2.13
web3==7.4.0
websockets==13.1
wheel @ file:///opt/homebrew/Cellar/python%403.13/3.13.0_1/libexec/wheel-0.44.0-py3-none-any.whl#sha256=f49b82715dce6365f75eddcb0a9bb47d0d46feaf14bbc739dfcbd677b7073f5b
yarl==1.17.0

[alexferl]

I also ran into this. Here's the output of @quetz repro code with export PYTHONASYNCIODEBUG=1.

Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x103d917f0>
source_traceback: Object created at (most recent call last):
  File "/Users/alex/work/reference/derp.py", line 17, in <module>
    asyncio.run(main())
  File "/opt/homebrew/Cellar/python@3.13/3.13.0_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/runners.py", line 194, in run
    return runner.run(main)
  File "/opt/homebrew/Cellar/python@3.13/3.13.0_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
  File "/opt/homebrew/Cellar/python@3.13/3.13.0_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py", line 708, in run_until_complete
    self.run_forever()
  File "/opt/homebrew/Cellar/python@3.13/3.13.0_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py", line 679, in run_forever
    self._run_once()
  File "/opt/homebrew/Cellar/python@3.13/3.13.0_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py", line 2019, in _run_once
    handle._run()
  File "/opt/homebrew/Cellar/python@3.13/3.13.0_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/events.py", line 89, in _run
    self._context.run(self._callback, *self._args)
  File "/Users/alex/work/reference/derp.py", line 8, in main
    await w3_1.eth.block_number
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/eth/async_eth.py", line 150, in block_number
    return await self.get_block_number()
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/module.py", line 159, in caller
    result = await async_w3.manager.coro_request(
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/manager.py", line 415, in coro_request
    response = await self._coro_make_request(method, params)
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/manager.py", line 342, in _coro_make_request
    return await request_func(method, params)
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/middleware/base.py", line 96, in middleware
    await make_request(method, params),
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/middleware/base.py", line 96, in middleware
    await make_request(method, params),
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/middleware/base.py", line 96, in middleware
    await make_request(method, params),
  [Previous line repeated 2 more times]
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/_utils/caching/caching_utils.py", line 365, in wrapper
    return await func(provider, method, params)
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/providers/rpc/async_rpc.py", line 161, in make_request
    raw_response = await self._make_request(method, request_data)
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/providers/rpc/async_rpc.py", line 138, in _make_request
    return await self._request_session_manager.async_make_post_request(
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/_utils/http_session_manager.py", line 284, in async_make_post_request
    response = await self.async_get_response_from_post_request(
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/_utils/http_session_manager.py", line 275, in async_get_response_from_post_request
    session = await self.async_cache_and_return_session(
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/_utils/http_session_manager.py", line 177, in async_cache_and_return_session
    session = ClientSession(raise_for_status=True)
Unclosed connector
connections: ['deque([(<aiohttp.client_proto.ResponseHandler object at 0x103e64c50>, 482264.667342041)])']
connector: <aiohttp.connector.TCPConnector object at 0x103d91d30>
source_traceback: Object created at (most recent call last):
  File "/Users/alex/work/reference/derp.py", line 17, in <module>
    asyncio.run(main())
  File "/opt/homebrew/Cellar/python@3.13/3.13.0_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/runners.py", line 194, in run
    return runner.run(main)
  File "/opt/homebrew/Cellar/python@3.13/3.13.0_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
  File "/opt/homebrew/Cellar/python@3.13/3.13.0_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py", line 708, in run_until_complete
    self.run_forever()
  File "/opt/homebrew/Cellar/python@3.13/3.13.0_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py", line 679, in run_forever
    self._run_once()
  File "/opt/homebrew/Cellar/python@3.13/3.13.0_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py", line 2019, in _run_once
    handle._run()
  File "/opt/homebrew/Cellar/python@3.13/3.13.0_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/events.py", line 89, in _run
    self._context.run(self._callback, *self._args)
  File "/Users/alex/work/reference/derp.py", line 8, in main
    await w3_1.eth.block_number
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/eth/async_eth.py", line 150, in block_number
    return await self.get_block_number()
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/module.py", line 159, in caller
    result = await async_w3.manager.coro_request(
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/manager.py", line 415, in coro_request
    response = await self._coro_make_request(method, params)
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/manager.py", line 342, in _coro_make_request
    return await request_func(method, params)
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/middleware/base.py", line 96, in middleware
    await make_request(method, params),
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/middleware/base.py", line 96, in middleware
    await make_request(method, params),
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/middleware/base.py", line 96, in middleware
    await make_request(method, params),
  [Previous line repeated 2 more times]
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/_utils/caching/caching_utils.py", line 365, in wrapper
    return await func(provider, method, params)
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/providers/rpc/async_rpc.py", line 161, in make_request
    raw_response = await self._make_request(method, request_data)
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/providers/rpc/async_rpc.py", line 138, in _make_request
    return await self._request_session_manager.async_make_post_request(
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/_utils/http_session_manager.py", line 284, in async_make_post_request
    response = await self.async_get_response_from_post_request(
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/_utils/http_session_manager.py", line 275, in async_get_response_from_post_request
    session = await self.async_cache_and_return_session(
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/web3/_utils/http_session_manager.py", line 177, in async_cache_and_return_session
    session = ClientSession(raise_for_status=True)
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/aiohttp/client.py", line 359, in __init__
    connector = TCPConnector(loop=loop)
  File "/Users/alex/Library/Caches/pypoetry/virtualenvs/reference-SpjYadir-py3.13/lib/python3.13/site-packages/aiohttp/connector.py", line 851, in __init__
    super().__init__(

[alexferl]

After doing a bit of digging, the problem is because the underlying aiohttp.ClientSession is never closed and this causes issue if you have more than one client instance. Also, as far as I know, there's no way to close the session(s) with a close() method or __aexit__ context or similar.

Fixing it properly would require a bit more time than I have now, so I came up with the following workaround:

import asyncio

from web3 import AsyncBaseProvider, AsyncWeb3
from web3.providers import AsyncHTTPProvider


async def close_sessions(provider: AsyncBaseProvider):
    # Close the underlying aiohttp client sessions
    for _, session in provider._request_session_manager.session_cache.items():
        await session.close()


async def main():
    rpc = "https://mainnet.optimism.io"
    w3_1 = AsyncWeb3(AsyncHTTPProvider(rpc))
    await w3_1.eth.block_number
    await close_sessions(w3_1.provider)

    w3_2 = AsyncWeb3(AsyncHTTPProvider(rpc))
    await w3_2.eth.block_number


asyncio.run(main())

[dbfreem]

hey @kclowes any idea when this will be released? I ran into this today.

[kclowes]

Hey! It'll be out this week, either tomorrow or Wednesday.

[MaxZarev]

I have the same problem. If I use AsyncWeb3(AsyncHTTPProvider(rpc)) in a context manager object, for example:

class Onchain:
    def __init__(self, account: Account, chain: Chain):
        self.account = account
        self.chain = chain
        self.w3: Optional[AsyncWeb3] = None

    @classmethod
    async def create(cls, account: Account, chain: Chain):
        instance = cls(account, chain)
        instance.w3 = AsyncWeb3(
            provider=AsyncHTTPProvider(
                endpoint_uri=instance.chain.rpc
            )
        )

        return instance

class Bot:
    def __init__(self, account: Account, chain: Chain = config.start_chain) -> None:
        self.chain = chain
        self.account = account

    async def __aenter__(self):
        self.onchain = await Onchain.create(self.account, self.chain)
        return self

    async def __aexit__(self, exc_type, exc_val, exc_tb):
        return True

When aexit is used when closing the object bot in context-manager, I get warnings like client_session: <aiohttp.client.ClientSession object at 0x103fc0ad0>.

I can fix it, added this code in __aexit__ :

async def __aexit__(self, exc_type, exc_val, exc_tb):
        for _, session in self.onchain.w3.provider._request_session_manager.session_cache.items():
            await session.close()
        self.onchain.w3 = None
        return True

[kclowes]

@MaxZarev are you using the latest? This fix came out in v7.7.0. If you're still seeing the warning after upgrading, please open a new issue.