[BUG] [python-nextgen] (De)serialization of enums fails due to missing conversion methods

[TimoGuenther]

Bug Report Checklist

• Have you provided a full/minimal spec to reproduce the issue?
• Have you validated the input using an OpenAPI validator (example)?
• Have you tested with the latest master to confirm the issue still exists?
• Have you searched for related issues/PRs?
• What's the actual output vs expected output?
• [Optional] Sponsorship to speed up the bug fix or feature request (example)

Description

The python-nextgen generator assumes the existence of (from|to)_(dict|json) methods on generated enums, which causes enum (de)serialization to fail with an exception.

Affected:

• singular allOf: The parent expects (from|to)_dict from the enum.
• anyOf, oneOf, plural allOf: The composite model expects (from|to)_(dict|json) from the wrapped enum(s).

Impact:

• That is how DRF Spectacular generates enums, for example.

openapi-generator version

6.3.0-20230125.125049-89

OpenAPI declaration file content or url

openapi: 3.0.3
info:
  title: ""
  version: ""
paths: {}
components:
  schemas:
    Parent:
      type: object
      properties:
        number:
          allOf:
          - $ref: '#/components/schemas/Number'
    Number:
      enum:
      - one
      - two
      - three
      type: string

Generation Details

java -jar openapi-generator-cli-6.3.0-20230125.125049-89.jar generate -i openapi-schema.yaml -g python-nextgen

Steps to reproduce

Assuming the import problem described in #14523 is fixed:

>>> from openapi_client import Parent, Number
>>> Parent.from_dict(dict(number="one"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "openapi_client/models/parent.py", line 73, in from_dict
    "number": Number.from_dict(obj.get("number")) if obj.get("number") is not None else None
AttributeError: type object 'Number' has no attribute 'from_dict'
>>> Parent(number=Number.ONE).to_dict()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "openapi_client/models/parent.py", line 60, in to_dict
    _dict['number'] = self.number.to_dict()
AttributeError: 'Number' object has no attribute 'to_dict'

Related issues/PRs

• [BUG] [python-nextgen] Deserialization with singular allOf and readOnly fails due to missing import #14523

Suggest a fix

Manually replacing the from_dict call with an enum value lookup (by calling the enum class directly) and removing the block that calls to_dict on the enum from parent.py fixes (de)serialization for singular allOf:

>>> from openapi_client import Parent, Number
>>> Parent.from_dict(dict(number="one"))
Parent(number=<Number.ONE: 'one'>)
>>> Parent(number=Number.ONE).to_dict()
{'number': <Number.ONE: 'one'>}

This must be automated.

The fix for composite models seems more involved (e.g., checking for existence of the required methods on the wrapped value).

Alternatively, the enums could feature trivial implementations of the required methods.

[spacether]

If you are open to using the python generator, it has verified CI tests of enums at:

1. https://github.com/OpenAPITools/openapi-generator/blob/master/samples/openapi3/client/3_0_3_unit_test/python/test/test_paths/test_request_body_post_enum_with0_does_not_match_false_request_body/test_post.py
2. https://github.com/OpenAPITools/openapi-generator/blob/master/samples/openapi3/client/3_0_3_unit_test/python/test/test_paths/test_request_body_post_enum_with_escaped_characters_request_body/test_post.py
3. https://github.com/OpenAPITools/openapi-generator/blob/master/samples/openapi3/client/3_0_3_unit_test/python/test/test_paths/test_request_body_post_simple_enum_validation_request_body/test_post.py
4. https://github.com/OpenAPITools/openapi-generator/blob/master/samples/openapi3/client/3_0_3_unit_test/python/test/test_models/test_enum_with0_does_not_match_false.py#L25
5. https://github.com/OpenAPITools/openapi-generator/blob/master/samples/openapi3/client/3_0_3_unit_test/python/test/test_models/test_enum_with1_does_not_match_true.py
6. https://github.com/OpenAPITools/openapi-generator/blob/master/samples/openapi3/client/3_0_3_unit_test/python/test/test_models/test_enum_with_escaped_characters.py
7. https://github.com/OpenAPITools/openapi-generator/blob/master/samples/openapi3/client/3_0_3_unit_test/python/test/test_models/test_enum_with_false_does_not_match0.py
8. https://github.com/OpenAPITools/openapi-generator/blob/master/samples/openapi3/client/3_0_3_unit_test/python/test/test_models/test_enum_with_true_does_not_match1.py
9. https://github.com/OpenAPITools/openapi-generator/blob/master/samples/openapi3/client/3_0_3_unit_test/python/test/test_models/test_enums_in_properties.py
10. https://github.com/OpenAPITools/openapi-generator/blob/master/samples/openapi3/client/petstore/python/tests_manual/test_boolean_enum.py#L28
11. https://github.com/OpenAPITools/openapi-generator/blob/master/samples/openapi3/client/petstore/python/tests_manual/test_integer_enum_one_value.py#L52
12. https://github.com/OpenAPITools/openapi-generator/blob/master/samples/openapi3/client/petstore/python/tests_manual/test_string_enum.py

[wing328]

I think this has been resolved in the latest master as we now have better support to handle the following:

        number:
          allOf:
          - $ref: '#/components/schemas/Number'

please give the latest master a try and happy to reopen this if needed.

[TimoGuenther]

I have tried with the latest snapshot (6.5.0-20230322.025654-60), the latest Docker image and even a manual build based on master. Unfortunately, in all cases, I still get the same error described in the first post.

[wing328]

@TimoGuenther I've file #15035 to improve the allOf handling.

Previously it only handled allOf with a single ref and top level attributes (e.g. default, readOnly). The PR make it works without the top level attributes as well.

Tested with your spec above to confirmed the fix.

[TimoGuenther]

I have tried the latest snapshot (6.5.0-20230329.141901-70) and can confirm that it fixes both this issue and the related #14523. Thank you for your effort! :)

[matt7aylor]

Does this fix the other affected cases from the issue description e.g. anyOf. I am hitting what looks like the same issue has no attribute 'from_json' in latest snapshot with a schema using anyOf.
e.g. as above but

        number:
          anyOf:
          - $ref: '#/components/schemas/Number'

[TimoGuenther]

Good catch, @matt7aylor. In the latest snapshot (6.5.0-20230401.025502-73), the issue still persists for anyOf, oneOf, plural allOf.

@wing328, please consider this output for (singular) anyOf:

>>> from openapi_client import Parent, Number
>>> Parent.from_dict(dict(number="one"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "openapi_client/models/parent.py", line 71, in from_dict
    "number": ParentNumber.from_dict(obj.get("number")) if obj.get("number") is not None else None
  File "openapi_client/models/parent_number.py", line 60, in from_dict
    return cls.from_json(json.dumps(obj))
  File "openapi_client/models/parent_number.py", line 69, in from_json
    instance.actual_instance = Number.from_json(json_str)
AttributeError: type object 'Number' has no attribute 'from_json'
>>> Parent(number=Number.ONE)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pydantic/main.py", line 342, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for Parent
number
  value is not a valid dict (type=type_error.dict)
>>> Parent(number="one")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pydantic/main.py", line 342, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for Parent
number
  value is not a valid dict (type=type_error.dict)
>>> Parent.from_dict({})
Parent(number=None)
>>> Parent()
Parent(number=None)
>>> Parent(number=None)
Parent(number=None)
>>> Parent.from_dict(dict(number=None))
Parent(number=None)
>>> Parent().to_dict()
{}

• from_dict still raises AttributeError.
• The constructor now wrongfully raises ValidationError in expectance of a dict instead of an enum value.
• number wrongfully acts as if it were nullable even though it is only not required. Or maybe that is intended because even a singular allOf acts that way. But I would argue that passing None for the value (via either dict or constructor) should be disallowed for a property that is nullable but not required; after all, according to the definition of required, required influences a property's existence (not its value).