Forwardings not tested properly in testsuites

[DanaSunal]

While running testsuite locally ( newman run ApplicationPattern+testcases.json -d ExecutionAndTraceLog+data.json ) I observed that almost all forwardings tested through ExecutionAndTraceLog/v1/list-records-of-flow are failing due to the forwarding record not being present in Elasticsearch.

Observation:

Test: ServiceLayer / BasicPart / /v1/register-yourself / Acceptance / Update LTP notified? / registration-operation update - 6 Requests in the folder

Iteration: 1 - sampling /v1/register-yourself

REQUEST BODY

  "registry-office-application": "RegistryOffice",
  "registry-office-application-release-number": "2.0.1",
  "registration-operation": "lsYrxOpmWuSV",
  "registry-office-protocol": "HTTP",
  "registry-office-address": {
    "ip-address": {
      "ipv-4-address": "1.1.3.8"
    }
  },
  "registry-office-port": 3008
}

Iteration: 1 - Expected ExecutionAndTraceLog/v1/list-records-of-flow

REQUEST BODY

  "x-correlator": "9db2cf5a-ff9c-a4d4-675d-e21e2c432e4c",
  "latest-match": 0,
  "number-of-records": 100
}

RESPONSE BODY

[
  {
    "x-correlator": "9db2cf5a-ff9c-a4d4-675d-e21e2c432e4c",
    "trace-indicator": "1",
    "user": "iswaryaa.subashchandran",
    "originator": "Postman",
    "application-name": "ExecutionAndTraceLog",
    "release-number": "2.0.1",
    "operation-name": "/v1/register-yourself",
    "response-code": 204
  }
]

Assert failure: Update in LTP is successfully notified to ALT -> expected false to be true

Test failed correctly, because the forwarding is not in response body of list-records-of-flow.

However, after the testsuite was done, I did manually (through swagger API) call list-records-of-flow with the same request body and the result was different:

[
  {
    "x-correlator": "9db2cf5a-ff9c-a4d4-675d-e21e2c432e4c",
    "trace-indicator": "1",
    "user": "iswaryaa.subashchandran",
    "originator": "Postman",
    "application-name": "ExecutionAndTraceLog",
    "release-number": "2.0.1",
    "operation-name": "/v1/register-yourself",
    "response-code": 204
  },
  {
    "x-correlator": "9db2cf5a-ff9c-a4d4-675d-e21e2c432e4c",
    "trace-indicator": "1.1.3",
    "user": "iswaryaa.subashchandran",
    "originator": "ExecutionAndTraceLog",
    "application-name": "RegistryOffice",
    "release-number": "2.0.1",
    "operation-name": "lsYrxOpmWuSV",
    "response-code": 500
  },
  {
    "x-correlator": "9db2cf5a-ff9c-a4d4-675d-e21e2c432e4c",
    "trace-indicator": "1.1.2.12",
    "user": "iswaryaa.subashchandran",
    "originator": "ExecutionAndTraceLog",
    "application-name": "ApplicationLayerTopology",
    "release-number": "2.0.1",
    "operation-name": "/v1/update-ltp",
    "response-code": 204
  }
]

The response body clearly shows record of '/v1/register-yourself' being called through Postman and two callback, to RegistryOffice and to ApplicationLayerTopology. The call to ALT has even response code of 204.

The problem here is, that we do not await the results of callbacks in the code, which means, that the code (and therefore the test) continues and at the time EATL records are checked, the callbacks are not done yet.

1. Postman -> request -> tested app
2. tested app -> request (callback) -> RegistryOffice
3. tested app -> request (callback) -> ALT
4. ALT -> callback execution -> response sent to tested app
5. RegistryOffice -> callback execution -> response sent to tested app
6. tested app -> response -> Postman
7. Postman -> EATL list-records check
8. RegistryOffice -> request logged in EATL
9. ALT -> request logged in EATL

I did a simple test, for one method I put 'await' in the code where callbacks are being sent and for this method, all the callbacks could be found in EATL at the time the test was checking the records.

Proposal: await the results of callbacks in the code to avoid such race conditions.

[IswaryaaS]

await in callback of server side code would result in increased wait time of user. So , as a workaround, the following would be made.

• In testcases, a retry mechanism would be added : 3 times : with delay of 3 seconds for checking if the records are logged into EATL/OL as per Include retry mechanism for forwarding : enhancement #719
• In server side, we can also analyse why the delay is happening for recording services.