Inconsistent content-type header & JSON structure for error/problem responses in 3.2.x

[j-schumann]

API Platform version(s) affected: 3.2.4

Description
We build a Hydra-based API that returns mostly json-ld.

Behaviour in 3.1
(Almost) every error response the API returned contained Hydra properties ('hydra:title', 'hydra:description') and used the content-type 'application/ld+json', regardless of the error source, wether it was an (expected) routing error, authentication error or validation problem.

Behaviour in 3.2
By setting rfc_7807_compliant_errors: true we expect the returned content-type and also the returned JSON structure to change.
But in a consistent way. Currently results are very inconsistent:

E.g. validation errors use 'application/problem+json; charset=utf-8' and return Hydra properties.

But calling an endpoint with an unsupported method, that result in a 405 with No route found for "DELETE http://localhost/documents": Method Not Allowed (Allow: GET, POST), the content-type is 'application/ld+json' with a JSON problem structure but without Hydra properties.
Same for 404s with No route found ..., while other 404s caused by invalid entity IDs etc have Hydra properties and the 'application/problem+json; charset=utf-8' content-type.

JSON content changes
Sometime (often together with the content-type) the value of the 'type' property changes.

E.g. 404 from route not found (no problem header, no hydra props) contains:
{"type":"https:\/\/tools.ietf.org\/html\/rfc2616#section-10","title":"An error occurred","status":404,"detail":"No route found for \u0022GET http:\/\/localhost\/providers\/abc\/challenges\u0022", [...]}
with 'type' referring to the RFC.

While other 404s, e.g. from entity not found contain:
{"@id":"\/errors\/404","@type":"hydra:Error","title":"An error occurred","detail":"Not Found","status":404,"type":"\/errors\/404","hydra:title":"An error occurred","hydra:description":"Not Found"}
with 'type' referring to /errors/404

This also happens for other status codes like 400, 403 etc.

How to reproduce
I have no reproducer but am sure that it could easily be reproduced with the demo app, by calling some nonexisting endpoints in the 3.1. and the 3.2 version and comparing the results.

with the online demo

Non-existent route (books with missing s):

curl -X 'GET'   'https://demo.api-platform.com/book/xx' -H 'accept: application/ld+json' -v
> content-type: application/ld+json
> {"@context":"\/contexts\/Error","@type":"hydra:Error","hydra:title":"An error occurred","hydra:description":"No route found for \u0022GET http:\/\/demo.api-platform.com\/book\/xx\u0022"}

-> no problem header, no JSON problem structure (no 'type', no 'detail')

existing endpoint but entity not found:

curl -X 'GET'   'https://demo.api-platform.com/book/xx' -H 'accept: application/ld+json' -v
> content-type: application/problem+json; charset=utf-8
>{"@id":"\/errors\/404","@type":"hydra:Error","title":"An error occurred","detail":"Invalid uri variables.","status":404,"type":"\/errors\/404","hydra:title":"An error occurred","hydra:description":"Invalid uri variables."}

-> with problem header, with problem + hydra structure

Possible Solution
I would expect an API to show consistent behaviour over all error responses (as it was with ApiPlatform <= 3.1), probably 3.2. needs to be extended to also enhance the Symfony base errors.

Additional Context

    defaults:
        extra_properties:
          rfc_7807_compliant_errors: true
          skip_deprecated_exception_normalizers: true
    event_listeners_backward_compatibility_layer: false
    keep_legacy_inflector: false

[soyuka]

Using rfc_7807_compliant_errors: true every errors should return a problem+json content-type. Hydra properties may be available if you asked for Accept: application/ld+json but when a Content Negotation error occurs we don't know that and we return problem+json.

[j-schumann]

Using rfc_7807_compliant_errors: true every errors should return a problem+json content-type. Hydra properties may be available if you asked for Accept: application/ld+json but when a Content Negotation error occurs we don't know that and we return problem+json.

Thanks @soyuka for confirming how it is intended, that is exactly what I thought.
This issue is about the errors that not comply to this.

All requests I do have this Accept header, also the examples I made to the online demo, although I do not know if the demo actually uses rfc_7807_compliant_errors: true
But from looking at https://github.com/api-platform/demo/blob/main/api/config/packages/api_platform.yaml#L29 it seems like it does.

[soyuka]

#5963

[j-schumann]

Hello @soyuka,
I explicitly created two issues for the distinction: This issue here for the 3.2.x behaviour / backwards incompatiblity.
And #5963 was created as feature request, to discuss how error handling could be improved in the future. Sadly most discussion there was about the current bug(s). Do you want to keep it all in one issue?

[soyuka]

indeed, I correctly closed this as fixed by ba8a7e6 but if it's not let me know.

[walva]

I'm sharing my experience to help showcase the issue.
I encountered this issue within my behat tests (using soyuka/contexts btw, thanks).

I have steps like:

And the JSON node "violations[0].message" should be equal to "Cette valeur ne doit pas être vide."
And the property "currentPassword" should have violation "e2eca605-a077-4352-873b-e401daebe69c"

public function thePropertyShouldNotBeValid($property, $code, $payloadProperty = null, $value = null)
    {
        $violations = $this->getViolationsForPath($property);
        if (empty($violations)) throw new \Exception("The property $property doesn't have any violation");
        foreach ($violations as $violation) {
            $violation = (array) $violation;
            if ($violation['code'] == $code) {
                if ($payloadProperty === null) return;

                $violationPayload = $violation['payload'] ?? null;
                $actualValue = ($violationPayload) ? $violationPayload->$payloadProperty : null;
                if ($actualValue === null) {
                    throw new \Exception("The property containing violation $code does not have payload property $payloadProperty");
                }

                if ($actualValue == $value) return;
                throw new \Exception("The property containing violation $code has $payloadProperty value $actualValue, expected $value");
            }
        }
        throw new \Exception("The property doesn't have the violation $code");
    }

I noticed that those scenarios were borken if using

When I add "Accept" header equal to "application/json"

but not with

When I add "Accept" header equal to "application/ld+json"

Partial solution:

It solves my test but the result is not 100% the same as before.

api_platform.yaml

api_platform:
  formats:
    jsonld: [ 'application/ld+json' ]
    json: [ 'application/json' ]
    html: [ 'text/html' ]
  docs_formats:
    jsonld: ['application/ld+json']
    html: ['text/html']
  exception_to_status:
    Symfony\Component\HttpFoundation\BadRequestException: 400
  defaults:
    extra_properties:
      rfc_7807_compliant_errors: true
      skip_deprecated_exception_normalizers: false

Gives

Feature: The error message is translated

  Scenario: Validate the API return translated violations                                              # features/translations.feature:3
    When I add "accept-language" header equal to "fr"                                                  # Behatch\Context\RestContext::iAddHeaderEqualTo()
    When I add "Content-Type" header equal to "application/ld+json"                                    # Behatch\Context\RestContext::iAddHeaderEqualTo()
    When I add "Accept" header equal to "application/ld+json"                                          # Behatch\Context\RestContext::iAddHeaderEqualTo()
    And I send a POST request to "/admin/login" with body:                                             # Behatch\Context\RestContext::iSendARequestToWithBody()
      """
      {
      }
      """
    Then print last JSON response                                                                      # Behatch\Context\JsonContext::printLastJsonResponse()
      │ {
      │     "@id": "/validation_errors/0=c1051bb4-d103-4f74-8988-acbcafc7fdc3;1=c1051bb4-d103-4f74-8988-acbcafc7fdc3",
      │     "@type": "ConstraintViolationList",
      │     "status": 422,
      │     "violations": [
      │         {
      │             "propertyPath": "username",
      │             "message": "Cette valeur ne doit pas être vide.",
      │             "code": "c1051bb4-d103-4f74-8988-acbcafc7fdc3"
      │         },
      │         {
      │             "propertyPath": "password",
      │             "message": "Cette valeur ne doit pas être vide.",
      │             "code": "c1051bb4-d103-4f74-8988-acbcafc7fdc3"
      │         }
      │     ],
      │     "detail": "username: Cette valeur ne doit pas être vide.\npassword: Cette valeur ne doit pas être vide.",
      │     "hydra:title": "An error occurred",
      │     "hydra:description": "username: Cette valeur ne doit pas être vide.\npassword: Cette valeur ne doit pas être vide.",
      │     "type": "/validation_errors/0=c1051bb4-d103-4f74-8988-acbcafc7fdc3;1=c1051bb4-d103-4f74-8988-acbcafc7fdc3",
      │     "title": "An error occurred"
      │ }
    Then the response status code should be 422                                                        # Behat\MinkExtension\Context\MinkContext::assertResponseStatus()
    And the JSON node "violations[0].message" should be equal to "Cette valeur ne doit pas être vide." # Behatch\Context\JsonContext::theJsonNodeShouldBeEqualTo()

  Scenario: Validate the API return translated violations                                              # features/translations.feature:16
    When I add "accept-language" header equal to "fr"                                                  # Behatch\Context\RestContext::iAddHeaderEqualTo()
    When I add "Content-Type" header equal to "application/ld+json"                                    # Behatch\Context\RestContext::iAddHeaderEqualTo()
    When I add "Accept" header equal to "application/json"                                             # Behatch\Context\RestContext::iAddHeaderEqualTo()
    And I send a POST request to "/admin/login" with body:                                             # Behatch\Context\RestContext::iSendARequestToWithBody()
      """
      {
      }
      """
    Then print last JSON response                                                                      # Behatch\Context\JsonContext::printLastJsonResponse()
      │ {
      │     "status": 422,
      │     "violations": [
      │         {
      │             "propertyPath": "username",
      │             "message": "Cette valeur ne doit pas être vide.",
      │             "code": "c1051bb4-d103-4f74-8988-acbcafc7fdc3"
      │         },
      │         {
      │             "propertyPath": "password",
      │             "message": "Cette valeur ne doit pas être vide.",
      │             "code": "c1051bb4-d103-4f74-8988-acbcafc7fdc3"
      │         }
      │     ],
      │     "detail": "username: Cette valeur ne doit pas être vide.\npassword: Cette valeur ne doit pas être vide.",
      │     "type": "/validation_errors/0=c1051bb4-d103-4f74-8988-acbcafc7fdc3;1=c1051bb4-d103-4f74-8988-acbcafc7fdc3",
      │     "title": "An error occurred"
      │ }
    Then the response status code should be 422                                                        # Behat\MinkExtension\Context\MinkContext::assertResponseStatus()
    And the JSON node "violations[0].message" should be equal to "Cette valeur ne doit pas être vide."