Prerequisites
- ES up and running, with RO control-construct present in alt-2-0-1 index
- RO and ALT apps running (for the sake of more readable logs, I disabled recording of service requests, so no EATL needed)
Test scenario
- run a script, which calls register and subsequent deregister of VeryDummyApplication 7 times. Each iteration adds 2 letters to the name VeryDummyApplication
Iteration 1: curl register-application 'VeryDummyApplicationAa', curl deregister-application 'VeryDummyApplicationAa'
Iteration 2: curl register-application 'VeryDummyApplicationBb', curl deregister-application 'VeryDummyApplicationBb'
etc.
Iteration 7: curl register-application 'VeryDummyApplicationGg', curl deregister-application 'VeryDummyApplicationGg'
Curl command by definition waits for the response of the REST call, so, deregister-application will be called ONLY AFTER register-application returns some http status.
register-application will create 5 LTPs (http-c-vdaa, tcp-c-vdaa and 3 op-c-vdaa)
Expectation
After running the script, the RO control-construct in ES (ALT's ApplicationData) will NOT contain any LTPs with 'vda' in their UUID. (CC before running the script, should be the same as CC after running it)
Reality
RO control-construct in ES (ALT's ApplicationData) contains a lot of LTPs with 'vda' in their UUID. Some can be even dangling, such as that the http-c LTP does not exist, but there are op-c referencing it.
Problem description
The problem lies in the order of execution (and termination) of callbacks. Although the callbacks start executing in correct order (note lines 'called with') the order in which they finish execution is completely random. This is not something that can be fixed in ALT, because the requests come in separately.
ALT output with some helper logs (note highlighted ro-2-0-1-http-c-vdaa-5-4-8-000) :
Update-ltp called with ro-2-0-1-http-c-vdaa-5-4-8-000
Update-ltp called with ro-2-0-1-tcp-c-vdaa-5-4-8-000
Update-ltp called with ro-2-0-1-op-c-im-vdaa-5-4-8-000
Update-ltp called with ro-2-0-1-op-c-im-vdaa-5-4-8-001
Update-ltp called with ro-2-0-1-op-c-im-vdaa-5-4-8-002
Delete-ltp is called with ro-2-0-1-http-c-vdaa-5-4-8-000
Delete-ltp is called with ro-2-0-1-tcp-c-vdaa-5-4-8-000
Delete-ltp is called with ro-2-0-1-op-c-im-vdaa-5-4-8-000
Delete-ltp is called with ro-2-0-1-op-c-im-vdaa-5-4-8-001
Delete-ltp is called with ro-2-0-1-op-c-im-vdaa-5-4-8-002
Update-ltp called with ro-2-0-1-http-c-vdab-5-4-8-000
Update-ltp called with ro-2-0-1-tcp-c-vdab-5-4-8-000
Update-ltp called with ro-2-0-1-op-c-im-vdab-5-4-8-000
Update-ltp called with ro-2-0-1-op-c-im-vdab-5-4-8-001
Update-ltp called with ro-2-0-1-op-c-im-vdab-5-4-8-002
POST /v1/disregard-application 204 403.635 ms - -
Delete-ltp is called with ro-2-0-1-http-c-vdab-5-4-8-000
Delete-ltp is called with ro-2-0-1-tcp-c-vdab-5-4-8-000
Delete-ltp is called with ro-2-0-1-op-c-im-vdab-5-4-8-000
Delete-ltp is called with ro-2-0-1-op-c-im-vdab-5-4-8-001
Delete-ltp is called with ro-2-0-1-op-c-im-vdab-5-4-8-002
POST /v1/disregard-application 204 81.350 ms - -
Update-ltp called with ro-2-0-1-http-c-vdac-5-4-8-000
Update-ltp called with ro-2-0-1-tcp-c-vdac-5-4-8-000
Update-ltp called with ro-2-0-1-op-c-im-vdac-5-4-8-000
Update-ltp called with ro-2-0-1-op-c-im-vdac-5-4-8-001
Update-ltp called with ro-2-0-1-op-c-im-vdac-5-4-8-002
POST /v1/disregard-application 204 7.808 ms - -
Delete-ltp is called with ro-2-0-1-http-c-vdac-5-4-8-000
Delete-ltp is called with ro-2-0-1-tcp-c-vdac-5-4-8-000
Delete-ltp is called with ro-2-0-1-op-c-im-vdac-5-4-8-000
Delete-ltp is called with ro-2-0-1-op-c-im-vdac-5-4-8-001
Delete-ltp is called with ro-2-0-1-op-c-im-vdac-5-4-8-002
Delete-ltp done with ro-2-0-1-op-c-im-vdab-5-4-8-001
Delete-ltp done with ro-2-0-1-op-c-im-vdac-5-4-8-000
Delete-ltp done with ro-2-0-1-op-c-im-vdac-5-4-8-001
Delete-ltp done with ro-2-0-1-op-c-im-vdac-5-4-8-002
Delete-ltp done with ro-2-0-1-op-c-im-vdaa-5-4-8-002
Delete-ltp done with ro-2-0-1-http-c-vdaa-5-4-8-000
Delete-ltp done with ro-2-0-1-op-c-im-vdaa-5-4-8-001
Delete-ltp done with ro-2-0-1-tcp-c-vdab-5-4-8-000
Delete-ltp done with ro-2-0-1-http-c-vdac-5-4-8-000
Delete-ltp done with ro-2-0-1-tcp-c-vdac-5-4-8-000
Delete-ltp done with ro-2-0-1-tcp-c-vdaa-5-4-8-000
Delete-ltp done with ro-2-0-1-op-c-im-vdab-5-4-8-002
Delete-ltp done with ro-2-0-1-http-c-vdab-5-4-8-000
Delete-ltp done with ro-2-0-1-op-c-im-vdab-5-4-8-000
Delete-ltp done with ro-2-0-1-op-c-im-vdaa-5-4-8-000
Update-ltp called with ro-2-0-1-http-c-vdad-5-4-8-000
Update-ltp called with ro-2-0-1-tcp-c-vdad-5-4-8-000
Update-ltp called with ro-2-0-1-op-c-im-vdad-5-4-8-000
POST /v1/delete-ltp-and-dependents 204 811.569 ms - -
Update-ltp called with ro-2-0-1-op-c-im-vdad-5-4-8-001
Update-ltp called with ro-2-0-1-op-c-im-vdad-5-4-8-002
POST /v1/delete-ltp-and-dependents 204 412.305 ms - -
Delete-ltp is called with ro-2-0-1-http-c-vdad-5-4-8-000
POST /v1/delete-ltp-and-dependents 204 406.825 ms - -
Delete-ltp is called with ro-2-0-1-tcp-c-vdad-5-4-8-000
POST /v1/delete-ltp-and-dependents 204 407.240 ms - -
POST /v1/delete-ltp-and-dependents 204 1109.806 ms - -
POST /v1/delete-ltp-and-dependents 204 1194.357 ms - -
POST /v1/delete-ltp-and-dependents 204 1115.039 ms - -
POST /v1/delete-ltp-and-dependents 204 853.595 ms - -
POST /v1/delete-ltp-and-dependents 204 450.425 ms - -
Delete-ltp is called with ro-2-0-1-op-c-im-vdad-5-4-8-000
POST /v1/delete-ltp-and-dependents 204 442.599 ms - -
POST /v1/delete-ltp-and-dependents 204 1129.655 ms - -
POST /v1/delete-ltp-and-dependents 204 849.213 ms - -
POST /v1/delete-ltp-and-dependents 204 872.621 ms - -
POST /v1/delete-ltp-and-dependents 204 866.252 ms - -
POST /v1/delete-ltp-and-dependents 204 1137.133 ms - -
Delete-ltp is called with ro-2-0-1-op-c-im-vdad-5-4-8-001
Delete-ltp is called with ro-2-0-1-op-c-im-vdad-5-4-8-002
POST /v1/update-fc 204 1543.543 ms - -
POST /v1/update-fc 204 1272.180 ms - -
POST /v1/update-fc 204 829.803 ms - -
Update-ltp called with ro-2-0-1-http-c-vdae-5-4-8-000
Update-ltp called with ro-2-0-1-tcp-c-vdae-5-4-8-000
Update-ltp called with ro-2-0-1-op-c-im-vdae-5-4-8-000
Update-ltp called with ro-2-0-1-op-c-im-vdae-5-4-8-001
Update-ltp called with ro-2-0-1-op-c-im-vdae-5-4-8-002
Delete-ltp done with ro-2-0-1-tcp-c-vdad-5-4-8-000
Delete-ltp done with ro-2-0-1-http-c-vdad-5-4-8-000
Delete-ltp done with ro-2-0-1-op-c-im-vdad-5-4-8-001
Delete-ltp done with ro-2-0-1-op-c-im-vdad-5-4-8-000
Delete-ltp done with ro-2-0-1-op-c-im-vdad-5-4-8-002
POST /v1/disregard-application 204 381.772 ms - -
POST /v1/delete-ltp-and-dependents 204 373.129 ms - -
POST /v1/delete-ltp-and-dependents 204 386.785 ms - -
Update-ltp done with ro-2-0-1-tcp-c-vdac-5-4-8-000
POST /v1/delete-ltp-and-dependents 204 367.782 ms - -
POST /v1/delete-ltp-and-dependents 204 385.633 ms - -
POST /v1/update-fc 204 424.056 ms - -
Update-ltp done with ro-2-0-1-op-c-im-vdab-5-4-8-000
POST /v1/delete-ltp-and-dependents 204 376.800 ms - -
POST /v1/update-ltp 204 1216.455 ms - -
Update-ltp done with ro-2-0-1-op-c-im-vdac-5-4-8-000
Delete-ltp is called with ro-2-0-1-http-c-vdae-5-4-8-000
Delete-ltp is called with ro-2-0-1-tcp-c-vdae-5-4-8-000
Delete-ltp is called with ro-2-0-1-op-c-im-vdae-5-4-8-000
Delete-ltp is called with ro-2-0-1-op-c-im-vdae-5-4-8-001
Delete-ltp is called with ro-2-0-1-op-c-im-vdae-5-4-8-002
POST /v1/update-ltp 204 1702.578 ms - -
Update-ltp done with ro-2-0-1-op-c-im-vdac-5-4-8-001
POST /v1/update-ltp 204 1254.662 ms - -
POST /v1/update-ltp 204 1280.326 ms - -
Update-ltp done with ro-2-0-1-http-c-vdac-5-4-8-000
Delete-ltp done with ro-2-0-1-tcp-c-vdae-5-4-8-000
Delete-ltp done with ro-2-0-1-http-c-vdae-5-4-8-000
Delete-ltp done with ro-2-0-1-op-c-im-vdae-5-4-8-000
Delete-ltp done with ro-2-0-1-op-c-im-vdae-5-4-8-001
Delete-ltp done with ro-2-0-1-op-c-im-vdae-5-4-8-002
Update-ltp called with ro-2-0-1-http-c-vdaf-5-4-8-000
Update-ltp called with ro-2-0-1-tcp-c-vdaf-5-4-8-000
Update-ltp called with ro-2-0-1-op-c-im-vdaf-5-4-8-000
POST /v1/update-ltp 204 1344.014 ms - -
POST /v1/disregard-application 204 116.295 ms - -
Update-ltp done with ro-2-0-1-op-c-im-vdab-5-4-8-001
POST /v1/delete-ltp-and-dependents 204 116.047 ms - -
Update-ltp called with ro-2-0-1-op-c-im-vdaf-5-4-8-001
POST /v1/delete-ltp-and-dependents 204 120.960 ms - -
POST /v1/delete-ltp-and-dependents 204 119.795 ms - -
Update-ltp called with ro-2-0-1-op-c-im-vdaf-5-4-8-002
POST /v1/delete-ltp-and-dependents 204 123.080 ms - -
POST /v1/delete-ltp-and-dependents 204 125.413 ms - -
POST /v1/update-ltp 204 1827.022 ms - -
Update-ltp done with ro-2-0-1-op-c-im-vdaa-5-4-8-002
POST /v1/update-ltp 204 2129.509 ms - -
Update-ltp done with ro-2-0-1-op-c-im-vdaa-5-4-8-001
POST /v1/update-ltp 204 2177.660 ms - -
Update-ltp done with ro-2-0-1-op-c-im-vdab-5-4-8-002
POST /v1/update-ltp 204 1921.927 ms - -
POST /v1/disregard-application 204 7.024 ms - -
Delete-ltp is called with ro-2-0-1-http-c-vdaf-5-4-8-000
Update-ltp done with ro-2-0-1-tcp-c-vdab-5-4-8-000
Delete-ltp is called with ro-2-0-1-tcp-c-vdaf-5-4-8-000
POST /v1/update-ltp 204 1985.054 ms - -
Delete-ltp is called with ro-2-0-1-op-c-im-vdaf-5-4-8-000
Delete-ltp is called with ro-2-0-1-op-c-im-vdaf-5-4-8-001
Delete-ltp done with ro-2-0-1-http-c-vdaf-5-4-8-000
Delete-ltp is called with ro-2-0-1-op-c-im-vdaf-5-4-8-002
Update-ltp done with ro-2-0-1-tcp-c-vdaa-5-4-8-000
Delete-ltp done with ro-2-0-1-tcp-c-vdaf-5-4-8-000
POST /v1/delete-ltp-and-dependents 204 48.495 ms - -
POST /v1/update-ltp 204 2295.619 ms - -
Delete-ltp done with ro-2-0-1-op-c-im-vdaf-5-4-8-000
POST /v1/delete-ltp-and-dependents 204 43.377 ms - -
Delete-ltp done with ro-2-0-1-op-c-im-vdaf-5-4-8-001
POST /v1/delete-ltp-and-dependents 204 49.844 ms - -
Delete-ltp done with ro-2-0-1-op-c-im-vdaf-5-4-8-002
POST /v1/delete-ltp-and-dependents 204 48.321 ms - -
POST /v1/delete-ltp-and-dependents 204 42.345 ms - -
Update-ltp done with ro-2-0-1-http-c-vdaa-5-4-8-000
For ro-2-0-1-http-c-vdaa-5-4-8-000 the deletion was fnished before the creation of the LTP resulting in this LTP being present in ES.
Although this was demonstrated in ALT using RO, I am sure it is the same case with any 2 complementary operations doing callbacks regardless of the application.
Note that at any point of the running of the script no error was produced either from RO or ALT side.
Prerequisites
Test scenario
Iteration 1: curl register-application 'VeryDummyApplicationAa', curl deregister-application 'VeryDummyApplicationAa'
Iteration 2: curl register-application 'VeryDummyApplicationBb', curl deregister-application 'VeryDummyApplicationBb'
etc.
Iteration 7: curl register-application 'VeryDummyApplicationGg', curl deregister-application 'VeryDummyApplicationGg'
Curl command by definition waits for the response of the REST call, so, deregister-application will be called ONLY AFTER register-application returns some http status.
register-application will create 5 LTPs (http-c-vdaa, tcp-c-vdaa and 3 op-c-vdaa)
Expectation
After running the script, the RO control-construct in ES (ALT's ApplicationData) will NOT contain any LTPs with 'vda' in their UUID. (CC before running the script, should be the same as CC after running it)
Reality
RO control-construct in ES (ALT's ApplicationData) contains a lot of LTPs with 'vda' in their UUID. Some can be even dangling, such as that the http-c LTP does not exist, but there are op-c referencing it.
Problem description
The problem lies in the order of execution (and termination) of callbacks. Although the callbacks start executing in correct order (note lines 'called with') the order in which they finish execution is completely random. This is not something that can be fixed in ALT, because the requests come in separately.
ALT output with some helper logs (note highlighted ro-2-0-1-http-c-vdaa-5-4-8-000) :
Update-ltp called with ro-2-0-1-http-c-vdaa-5-4-8-000
Update-ltp called with ro-2-0-1-tcp-c-vdaa-5-4-8-000
Update-ltp called with ro-2-0-1-op-c-im-vdaa-5-4-8-000
Update-ltp called with ro-2-0-1-op-c-im-vdaa-5-4-8-001
Update-ltp called with ro-2-0-1-op-c-im-vdaa-5-4-8-002
Delete-ltp is called with ro-2-0-1-http-c-vdaa-5-4-8-000
Delete-ltp is called with ro-2-0-1-tcp-c-vdaa-5-4-8-000
Delete-ltp is called with ro-2-0-1-op-c-im-vdaa-5-4-8-000
Delete-ltp is called with ro-2-0-1-op-c-im-vdaa-5-4-8-001
Delete-ltp is called with ro-2-0-1-op-c-im-vdaa-5-4-8-002
Update-ltp called with ro-2-0-1-http-c-vdab-5-4-8-000
Update-ltp called with ro-2-0-1-tcp-c-vdab-5-4-8-000
Update-ltp called with ro-2-0-1-op-c-im-vdab-5-4-8-000
Update-ltp called with ro-2-0-1-op-c-im-vdab-5-4-8-001
Update-ltp called with ro-2-0-1-op-c-im-vdab-5-4-8-002
POST /v1/disregard-application 204 403.635 ms - -
Delete-ltp is called with ro-2-0-1-http-c-vdab-5-4-8-000
Delete-ltp is called with ro-2-0-1-tcp-c-vdab-5-4-8-000
Delete-ltp is called with ro-2-0-1-op-c-im-vdab-5-4-8-000
Delete-ltp is called with ro-2-0-1-op-c-im-vdab-5-4-8-001
Delete-ltp is called with ro-2-0-1-op-c-im-vdab-5-4-8-002
POST /v1/disregard-application 204 81.350 ms - -
Update-ltp called with ro-2-0-1-http-c-vdac-5-4-8-000
Update-ltp called with ro-2-0-1-tcp-c-vdac-5-4-8-000
Update-ltp called with ro-2-0-1-op-c-im-vdac-5-4-8-000
Update-ltp called with ro-2-0-1-op-c-im-vdac-5-4-8-001
Update-ltp called with ro-2-0-1-op-c-im-vdac-5-4-8-002
POST /v1/disregard-application 204 7.808 ms - -
Delete-ltp is called with ro-2-0-1-http-c-vdac-5-4-8-000
Delete-ltp is called with ro-2-0-1-tcp-c-vdac-5-4-8-000
Delete-ltp is called with ro-2-0-1-op-c-im-vdac-5-4-8-000
Delete-ltp is called with ro-2-0-1-op-c-im-vdac-5-4-8-001
Delete-ltp is called with ro-2-0-1-op-c-im-vdac-5-4-8-002
Delete-ltp done with ro-2-0-1-op-c-im-vdab-5-4-8-001
Delete-ltp done with ro-2-0-1-op-c-im-vdac-5-4-8-000
Delete-ltp done with ro-2-0-1-op-c-im-vdac-5-4-8-001
Delete-ltp done with ro-2-0-1-op-c-im-vdac-5-4-8-002
Delete-ltp done with ro-2-0-1-op-c-im-vdaa-5-4-8-002
Delete-ltp done with ro-2-0-1-http-c-vdaa-5-4-8-000
Delete-ltp done with ro-2-0-1-op-c-im-vdaa-5-4-8-001
Delete-ltp done with ro-2-0-1-tcp-c-vdab-5-4-8-000
Delete-ltp done with ro-2-0-1-http-c-vdac-5-4-8-000
Delete-ltp done with ro-2-0-1-tcp-c-vdac-5-4-8-000
Delete-ltp done with ro-2-0-1-tcp-c-vdaa-5-4-8-000
Delete-ltp done with ro-2-0-1-op-c-im-vdab-5-4-8-002
Delete-ltp done with ro-2-0-1-http-c-vdab-5-4-8-000
Delete-ltp done with ro-2-0-1-op-c-im-vdab-5-4-8-000
Delete-ltp done with ro-2-0-1-op-c-im-vdaa-5-4-8-000
Update-ltp called with ro-2-0-1-http-c-vdad-5-4-8-000
Update-ltp called with ro-2-0-1-tcp-c-vdad-5-4-8-000
Update-ltp called with ro-2-0-1-op-c-im-vdad-5-4-8-000
POST /v1/delete-ltp-and-dependents 204 811.569 ms - -
Update-ltp called with ro-2-0-1-op-c-im-vdad-5-4-8-001
Update-ltp called with ro-2-0-1-op-c-im-vdad-5-4-8-002
POST /v1/delete-ltp-and-dependents 204 412.305 ms - -
Delete-ltp is called with ro-2-0-1-http-c-vdad-5-4-8-000
POST /v1/delete-ltp-and-dependents 204 406.825 ms - -
Delete-ltp is called with ro-2-0-1-tcp-c-vdad-5-4-8-000
POST /v1/delete-ltp-and-dependents 204 407.240 ms - -
POST /v1/delete-ltp-and-dependents 204 1109.806 ms - -
POST /v1/delete-ltp-and-dependents 204 1194.357 ms - -
POST /v1/delete-ltp-and-dependents 204 1115.039 ms - -
POST /v1/delete-ltp-and-dependents 204 853.595 ms - -
POST /v1/delete-ltp-and-dependents 204 450.425 ms - -
Delete-ltp is called with ro-2-0-1-op-c-im-vdad-5-4-8-000
POST /v1/delete-ltp-and-dependents 204 442.599 ms - -
POST /v1/delete-ltp-and-dependents 204 1129.655 ms - -
POST /v1/delete-ltp-and-dependents 204 849.213 ms - -
POST /v1/delete-ltp-and-dependents 204 872.621 ms - -
POST /v1/delete-ltp-and-dependents 204 866.252 ms - -
POST /v1/delete-ltp-and-dependents 204 1137.133 ms - -
Delete-ltp is called with ro-2-0-1-op-c-im-vdad-5-4-8-001
Delete-ltp is called with ro-2-0-1-op-c-im-vdad-5-4-8-002
POST /v1/update-fc 204 1543.543 ms - -
POST /v1/update-fc 204 1272.180 ms - -
POST /v1/update-fc 204 829.803 ms - -
Update-ltp called with ro-2-0-1-http-c-vdae-5-4-8-000
Update-ltp called with ro-2-0-1-tcp-c-vdae-5-4-8-000
Update-ltp called with ro-2-0-1-op-c-im-vdae-5-4-8-000
Update-ltp called with ro-2-0-1-op-c-im-vdae-5-4-8-001
Update-ltp called with ro-2-0-1-op-c-im-vdae-5-4-8-002
Delete-ltp done with ro-2-0-1-tcp-c-vdad-5-4-8-000
Delete-ltp done with ro-2-0-1-http-c-vdad-5-4-8-000
Delete-ltp done with ro-2-0-1-op-c-im-vdad-5-4-8-001
Delete-ltp done with ro-2-0-1-op-c-im-vdad-5-4-8-000
Delete-ltp done with ro-2-0-1-op-c-im-vdad-5-4-8-002
POST /v1/disregard-application 204 381.772 ms - -
POST /v1/delete-ltp-and-dependents 204 373.129 ms - -
POST /v1/delete-ltp-and-dependents 204 386.785 ms - -
Update-ltp done with ro-2-0-1-tcp-c-vdac-5-4-8-000
POST /v1/delete-ltp-and-dependents 204 367.782 ms - -
POST /v1/delete-ltp-and-dependents 204 385.633 ms - -
POST /v1/update-fc 204 424.056 ms - -
Update-ltp done with ro-2-0-1-op-c-im-vdab-5-4-8-000
POST /v1/delete-ltp-and-dependents 204 376.800 ms - -
POST /v1/update-ltp 204 1216.455 ms - -
Update-ltp done with ro-2-0-1-op-c-im-vdac-5-4-8-000
Delete-ltp is called with ro-2-0-1-http-c-vdae-5-4-8-000
Delete-ltp is called with ro-2-0-1-tcp-c-vdae-5-4-8-000
Delete-ltp is called with ro-2-0-1-op-c-im-vdae-5-4-8-000
Delete-ltp is called with ro-2-0-1-op-c-im-vdae-5-4-8-001
Delete-ltp is called with ro-2-0-1-op-c-im-vdae-5-4-8-002
POST /v1/update-ltp 204 1702.578 ms - -
Update-ltp done with ro-2-0-1-op-c-im-vdac-5-4-8-001
POST /v1/update-ltp 204 1254.662 ms - -
POST /v1/update-ltp 204 1280.326 ms - -
Update-ltp done with ro-2-0-1-http-c-vdac-5-4-8-000
Delete-ltp done with ro-2-0-1-tcp-c-vdae-5-4-8-000
Delete-ltp done with ro-2-0-1-http-c-vdae-5-4-8-000
Delete-ltp done with ro-2-0-1-op-c-im-vdae-5-4-8-000
Delete-ltp done with ro-2-0-1-op-c-im-vdae-5-4-8-001
Delete-ltp done with ro-2-0-1-op-c-im-vdae-5-4-8-002
Update-ltp called with ro-2-0-1-http-c-vdaf-5-4-8-000
Update-ltp called with ro-2-0-1-tcp-c-vdaf-5-4-8-000
Update-ltp called with ro-2-0-1-op-c-im-vdaf-5-4-8-000
POST /v1/update-ltp 204 1344.014 ms - -
POST /v1/disregard-application 204 116.295 ms - -
Update-ltp done with ro-2-0-1-op-c-im-vdab-5-4-8-001
POST /v1/delete-ltp-and-dependents 204 116.047 ms - -
Update-ltp called with ro-2-0-1-op-c-im-vdaf-5-4-8-001
POST /v1/delete-ltp-and-dependents 204 120.960 ms - -
POST /v1/delete-ltp-and-dependents 204 119.795 ms - -
Update-ltp called with ro-2-0-1-op-c-im-vdaf-5-4-8-002
POST /v1/delete-ltp-and-dependents 204 123.080 ms - -
POST /v1/delete-ltp-and-dependents 204 125.413 ms - -
POST /v1/update-ltp 204 1827.022 ms - -
Update-ltp done with ro-2-0-1-op-c-im-vdaa-5-4-8-002
POST /v1/update-ltp 204 2129.509 ms - -
Update-ltp done with ro-2-0-1-op-c-im-vdaa-5-4-8-001
POST /v1/update-ltp 204 2177.660 ms - -
Update-ltp done with ro-2-0-1-op-c-im-vdab-5-4-8-002
POST /v1/update-ltp 204 1921.927 ms - -
POST /v1/disregard-application 204 7.024 ms - -
Delete-ltp is called with ro-2-0-1-http-c-vdaf-5-4-8-000
Update-ltp done with ro-2-0-1-tcp-c-vdab-5-4-8-000
Delete-ltp is called with ro-2-0-1-tcp-c-vdaf-5-4-8-000
POST /v1/update-ltp 204 1985.054 ms - -
Delete-ltp is called with ro-2-0-1-op-c-im-vdaf-5-4-8-000
Delete-ltp is called with ro-2-0-1-op-c-im-vdaf-5-4-8-001
Delete-ltp done with ro-2-0-1-http-c-vdaf-5-4-8-000
Delete-ltp is called with ro-2-0-1-op-c-im-vdaf-5-4-8-002
Update-ltp done with ro-2-0-1-tcp-c-vdaa-5-4-8-000
Delete-ltp done with ro-2-0-1-tcp-c-vdaf-5-4-8-000
POST /v1/delete-ltp-and-dependents 204 48.495 ms - -
POST /v1/update-ltp 204 2295.619 ms - -
Delete-ltp done with ro-2-0-1-op-c-im-vdaf-5-4-8-000
POST /v1/delete-ltp-and-dependents 204 43.377 ms - -
Delete-ltp done with ro-2-0-1-op-c-im-vdaf-5-4-8-001
POST /v1/delete-ltp-and-dependents 204 49.844 ms - -
Delete-ltp done with ro-2-0-1-op-c-im-vdaf-5-4-8-002
POST /v1/delete-ltp-and-dependents 204 48.321 ms - -
POST /v1/delete-ltp-and-dependents 204 42.345 ms - -
Update-ltp done with ro-2-0-1-http-c-vdaa-5-4-8-000
For ro-2-0-1-http-c-vdaa-5-4-8-000 the deletion was fnished before the creation of the LTP resulting in this LTP being present in ES.
Although this was demonstrated in ALT using RO, I am sure it is the same case with any 2 complementary operations doing callbacks regardless of the application.
Note that at any point of the running of the script no error was produced either from RO or ALT side.