[BUG][Python] Generator produces unusable clients

[Robinsstudio]

Bug Report Checklist

• Have you provided a full/minimal spec to reproduce the issue?
  There is no need for it as any spec will do.
• Have you validated the input using an OpenAPI validator (example)?
  Same as above.
• Have you tested with the latest master to confirm the issue still exists?
• Have you searched for related issues/PRs?
• What's the actual output vs expected output?
  The generated Python client is unable to call any endpoint.
• [Optional] Sponsorship to speed up the bug fix or feature request (example)

Description

I'm using the Python generator to generate a Python client for a corporate API. The generated client is now broken and the code throws when trying to call any endpoint.

Here's my investigation: PR #20697 was merged last week to add the ca_cert_data parameter. The ca_cert_data is added to the pool_args that are then fed into urllib3's PoolManager as keyword arguments:

rest.py

self.pool_manager = urllib3.PoolManager(**pool_args)

Then when connecting to a host using the HTTP or HTTPS schemes, those keyword arguments are merged into the urllib3's request_context.

urllib3's poolmanager.py

    def connection_from_host(self, host, port=None, scheme="http", pool_kwargs=None):
        """
        Get a :class:`urllib3.connectionpool.ConnectionPool` based on the host, port, and scheme.

        If ``port`` isn't given, it will be derived from the ``scheme`` using
        ``urllib3.connectionpool.port_by_scheme``. If ``pool_kwargs`` is
        provided, it is merged with the instance's ``connection_pool_kw``
        variable and used to create the new connection pool, if one is
        needed.
        """

        if not host:
            raise LocationValueError("No host specified.")

        request_context = self._merge_pool_kwargs(pool_kwargs) # <- The problem lies here.
        request_context["scheme"] = scheme or "http"
        if not port:
            port = port_by_scheme.get(request_context["scheme"].lower(), 80)
        request_context["port"] = port
        request_context["host"] = host

        return self.connection_from_context(request_context)

Finally, the request_context is used to construct urllib3's PoolKey. But key_ca_cert_data is not a keyword argument of PoolKey's so the code breaks.

urllib3's poolmanager.py (again)

    def connection_from_context(self, request_context):
        """
        Get a :class:`urllib3.connectionpool.ConnectionPool` based on the request context.

        ``request_context`` must at least contain the ``scheme`` key and its
        value must be a key in ``key_fn_by_scheme`` instance variable.
        """
        scheme = request_context["scheme"].lower()
        pool_key_constructor = self.key_fn_by_scheme.get(scheme)
        if not pool_key_constructor:
            raise URLSchemeUnknown(scheme)
        pool_key = pool_key_constructor(request_context) # 💥 It breaks here.

        return self.connection_from_pool_key(pool_key, request_context=request_context)

I must say I'm not very familiar with the code in this repository so I'm not sure how it could be fixed apart from reverting the PR #20697. In the meantime, I think I will use an earlier version.

If needed, I'm willing to investigate further and help provide a fix for this. Thank you! 🙏

[wing328]

thanks for reporting the issue

did you also try openapi generator v7.11.0 (previous stable release) to generate the python client to see if it has the same issue?

[wing328]

cc @micolous who's the author of #20697

[Robinsstudio]

I will do so on Monday and keep you updated!

[Robinsstudio]

I regenerated my Python client using v7.11.0 and can confirm that the only difference is indeed the absence of the ca_cert_data parameter and the issue is no longer there.

[wing328]

thanks for doing a test with v7.11.0 to confirm it works for you.

cc python technical committee to see if they've any clue about this issue

@cbornet (2017/09) @tomplus (2018/10) @krjakbrjak (2023/02) @fa0311 (2023/10) @multani (2023/10)

[multani]

I regenerated my Python client using v7.11.0 and can confirm that the only difference is indeed the absence of the ca_cert_data parameter and the issue is no longer there.

Can you check if the setting the correct urllib3 version helps?

Cf. the comment in #20697 (comment)

[Robinsstudio]

Setting urllib3 >= 2.1.0, < 3.0.0 seems to do the trick! I rebuilt my Python client with v7.12.0 and urllib3 2.3.0 and the ca_cert_data parameter is no longer an issue it seems. Thank you for your help!

Should I send a PR to perform this update?

[multani]

Should I send a PR to perform this update?

If you can, yes, I think it would be very much appreciated :)

[Robinsstudio]

Here it is: #20845!