Refactor container-id weblog test by zacharycmontoya · Pull Request #4925 · DataDog/system-tests

zacharycmontoya · 2025-07-11T00:42:57Z

Motivation

We're in the process of enhancing the container-id extraction support in the C++ tracer and I wanted to update the testing to support it. The current logic depends on the weblog applications implementing a /read_file endpoint and the test runner trusts this endpoint, only making assertions if that value is present. This means we're showing false positives against the C++ tracer code. I'm proposing changes to make the test runner get the expected value itself, instead of getting both the expected value and the actual value from the weblog application.

Changes

Refactors the container-id weblog test logic to get the expected value by directly running cat /proc/self/cgroup on the weblog container, instead of depending on the weblog HTTP server to self-report the cgroup via the /read_file endpoint.

This PR also removes all /read_file endpoints since this was the only use of the endpoint.

Workflow

⚠️ Create your PR as draft ⚠️
Work on you PR until the CI passes
Mark it as ready for review
- Test logic is modified? -> Get a review from RFC owner.
- Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

If PR title starts with [<language>], double-check that only <language> is impacted by the change
No system-tests internal is modified. Otherwise, I have the approval from R&P team
A docker base image is modified?
- the relevant build-XXX-image label is present
A scenario is added (or removed)?
- Get a review from R&P team

cbeauchesne · 2025-07-18T08:09:36Z

-        reason="Missing endpoint",
-    )
-    @missing_feature(context.library == "ruby" and context.weblog_variant != "rails70", reason="Missing endpoint")
+        self.r = weblog.get("/")


Is there a reason for keeping this request ?

I kept it because I thought it would be necessary in order to test that trace payloads sent to the agent have the required header. What happens if I remove it?

Your test code is checking all traces trigerred by the scenario, not only this one (it's the default scenario , and there are thousand of them).

Well, it does not hurt, but as your test is no linked to only this requests, it's not that necessary. Up to you, both are fine to me.

cbeauchesne

Just a small non-blocking comment/question

…y call 'cat /proc/self/cgroup' on the weblog container instead of asking the weblog application (over an HTTP call) to read a file. This allows us to instead call the '/' endpoint which all weblog applications should be handling.

…essary. They were only used for the container-id testing.

…ainer_tags'

github-actions · 2026-04-09T15:30:33Z

CODEOWNERS have been resolved as:

tests/test_data_integrity.py                                            @DataDog/system-tests-core
utils/build/docker/cpp_nginx/nginx/backend.c                            @DataDog/system-tests-core
utils/build/docker/cpp_nginx/nginx/nginx-waf.conf                       @DataDog/system-tests-core
utils/build/docker/cpp_nginx/nginx/nginx.conf                           @DataDog/system-tests-core
utils/build/docker/golang/app/chi/main.go                               @DataDog/dd-trace-go-guild @DataDog/system-tests-core
utils/build/docker/golang/app/echo/main.go                              @DataDog/dd-trace-go-guild @DataDog/system-tests-core
utils/build/docker/golang/app/gin/main.go                               @DataDog/dd-trace-go-guild @DataDog/system-tests-core
utils/build/docker/golang/app/net-http-orchestrion/main.go              @DataDog/dd-trace-go-guild @DataDog/system-tests-core
utils/build/docker/golang/app/net-http/main.go                          @DataDog/dd-trace-go-guild @DataDog/system-tests-core
utils/build/docker/java/spring-boot/src/main/java/com/datadoghq/system_tests/springboot/App.java  @DataDog/apm-java @DataDog/asm-java @DataDog/system-tests-core
utils/build/docker/java_otel/spring-boot/src/main/java/com/datadoghq/system_tests/springboot/App.java  @DataDog/opentelemetry @DataDog/system-tests-core
utils/build/docker/nodejs/express/app.js                                @DataDog/dd-trace-js @DataDog/system-tests-core
utils/build/docker/nodejs/fastify/app.js                                @DataDog/dd-trace-js @DataDog/system-tests-core
utils/build/docker/php/apache-mod/php.conf                              @DataDog/apm-php @DataDog/system-tests-core
utils/build/docker/php/php-fpm/php-fpm.conf                             @DataDog/apm-php @DataDog/system-tests-core
utils/build/docker/python/django/app/urls.py                            @DataDog/apm-python @DataDog/asm-python @DataDog/system-tests-core
utils/build/docker/python/fastapi/main.py                               @DataDog/apm-python @DataDog/asm-python @DataDog/system-tests-core
utils/build/docker/python/flask/app.py                                  @DataDog/apm-python @DataDog/asm-python @DataDog/system-tests-core
utils/build/docker/ruby/README.md                                       @DataDog/ruby-guild @DataDog/asm-ruby @DataDog/system-tests-core
utils/build/docker/ruby/graphql23/app/controllers/system_test_controller.rb  @DataDog/ruby-guild @DataDog/asm-ruby @DataDog/system-tests-core
utils/build/docker/ruby/graphql23/config/routes.rb                      @DataDog/ruby-guild @DataDog/asm-ruby @DataDog/system-tests-core
utils/build/docker/ruby/rails72/app/controllers/system_test_controller.rb  @DataDog/ruby-guild @DataDog/asm-ruby @DataDog/system-tests-core
utils/build/docker/ruby/rails72/config/routes.rb                        @DataDog/ruby-guild @DataDog/asm-ruby @DataDog/system-tests-core
utils/build/docker/ruby/rails80/app/controllers/system_test_controller.rb  @DataDog/ruby-guild @DataDog/asm-ruby @DataDog/system-tests-core
utils/build/docker/ruby/rails80/config/routes.rb                        @DataDog/ruby-guild @DataDog/asm-ruby @DataDog/system-tests-core
utils/build/docker/dotnet/weblog/Endpoints/ReadFileEndpoint.cs          @DataDog/apm-dotnet @DataDog/asm-dotnet @DataDog/system-tests-core

datadog-prod-us1-3 · 2026-04-09T19:02:20Z

⚠️ Tests

✨ Fix all issues with BitsAI or with Cursor

⚠️ Other Violations

🧪 5 Tests failed

tests.debugger.test_debugger_exception_replay.Test_Debugger_Exception_Replay.test_exception_replay_rockpaperscissors[poc]

from system_tests_suite

(Fix with Cursor)

AssertionError: assert [{'captures':...ubbed>', ...}] == [{'captures':...ubbed>', ...}]
  At index 1 diff: {'captures': {'return': {'staticFields': {'Empty': {'type': 'EmptyResult'}}, 'throwable': {'message': 'Rock exception', 'type': 'weblog.Models.Debugger.ExceptionReplayRock', 'stacktrace': [{'function': 'weblog.ExceptionReplayController.ExceptionReplayRockPaperScissors', 'lineNumber': 0}, {'<runtime>': '<scrubbed>'}]}, 'locals': {'@exception': {'type': 'ExceptionReplayRock', 'fields': {'Message': {'type': 'String', 'value': 'Rock exception'}, 'InnerException': {'type': 'Exception', 'isNull': 'true'}, 'HelpLink': {'type': 'String', 'isNull': 'true'}, 'Source': {'type': 'String', 'value': 'app'}, 'HResult': {'type': 'Int32', 'value': '-2146233088'}, 'StackTrace': {'type': 'String', 'value': '<scrubbed>'}}}}, 'arguments': {'shape': {'type': 'String', 'value': 'rock'}, 'this': {'type': 'ExceptionReplayController'}}}}, 'probe': {'id': '<scrubbed>', 'version': 1, 'location': {'method': 'ExceptionReplayRockPaperScissors', 'type': 'weblog.ExceptionReplayController'}}, 'stack': [{'function': 'weblog.ExceptionReplayController.ExceptionReplayRockPaperScissors', 'fileName': '/app/ExceptionReplayController.cs', 'lineNumber': 83}, {'function': 'Datadog.Trace.ClrProfiler.AutoInstrumentation.AspNetCore.BlockingMiddleware+<Invoke><scrubbed>', 'lineNumber': 0}, {'function': 'Datadog.Trace.ClrProfiler.AutoInstrumentation.AspNetCore.BlockingMiddleware.Invoke', 'lineNumber': 0}, {'<runtime>': '<scrubbed>'}], 'exceptionHash': 'e6882fa3-e986-3f7f-830f-322e1881ade5', 'exceptionId': '<scrubbed>', 'frameIndex': '0', 'id': '<scrubbed>', 'timestamp': '<scrubbed>', 'duration': '<scrubbed>', 'language': 'dotnet'} != {'captures': {'return': {'staticFields': {'Empty': {'type': 'EmptyResult'}}, 'throwable': {'message': 'Scissors exception', 'type': 'weblog.Models.Debugger.ExceptionReplayScissors', 'stacktrace': [{'function': 'weblog.ExceptionReplayController.ExceptionReplayRockPaperScissors', 'lineNumber': 0}, {'<runtime>': '<scrubbed>'}]}, 'locals': {'@exception': {'type': 'ExceptionReplayScissors', 'fields': {'Message': {'type': 'String', 'value': 'Scissors exception'}, 'InnerException': {'type': 'Exception', 'isNull': 'true'}, 'HelpLink': {'type': 'String', 'isNull': 'true'}, 'Source': {'type': 'String', 'value': 'app'}, 'HResult': {'type': 'Int32', 'value': '-2146233088'}, 'StackTrace': {'type': 'String', 'value': '<scrubbed>'}}}}, 'arguments': {'shape': {'type': 'String', 'value': 'scissors'}, 'this': {'type': 'ExceptionReplayController'}}}}, 'probe': {'id': '<scrubbed>', 'version': 1, 'location': {'method': 'ExceptionReplayRockPaperScissors', 'type': 'weblog.ExceptionReplayController'}}, 'stack': [{'function': 'weblog.ExceptionReplayController.ExceptionReplayRockPaperScissors', 'fileName': '/app/ExceptionReplayController.cs', 'lineNumber': 83}, {'function': 'Datadog.Trace.ClrProfiler.AutoInstrumentation.AspNetCore.BlockingMiddleware+<Invoke><scrubbed>', 'lineNumber': 0}, {'function': 'Datadog.Trace.ClrProfiler.AutoInstrumentation.AspNetCore.BlockingMiddleware.Invoke', 'lineNumber': 0}, {'<runtime>': '<scrubbed>'}], 'exceptionHash': '71174315-56b8-3c4b-83db-90764540ceae', 'exceptionId': '<scrubbed>', 'frameIndex': '0', 'id': '<scrubbed>', 'timestamp': '<scrubbed>', 'duration': '<scrubbed>', 'language': 'dotnet'}
  Left contains one more item: {'captures': {'return': {'arguments': {'shape': {'type': 'String', 'value': 'scissors'}, 'this': {'type': 'ExceptionRe...}, 'duration': '<scrubbed>', 'exceptionHash': '71174315-56b8-3c4b-83db-90764540ceae', 'exceptionId': '<scrubbed>', ...}
  Full diff:
    [
     {'captures': {'return': {'arguments': {'shape': {'type': 'String',
                                                      'value': 'paper'},
                                            'this': {'type': 'ExceptionReplayController'}},
                              'locals': {'@exception': {'fields': {'HResult': {'type': 'Int32',
                                                                               'value': '-2146233088'},
...

tests.debugger.test_debugger_inproduct_enablement.Test_Debugger_InProduct_Enablement_Exception_Replay.test_inproduct_enablement_exception_replay[poc]

from system_tests_suite

(Fix with Cursor)

AssertionError: Expected snapshots to be emitting after enabling exception replay
assert False
 +  where False = <tests.debugger.test_debugger_inproduct_enablement.Test_Debugger_InProduct_Enablement_Exception_Replay object at 0x7fb4cd22fe60>.er_explicit_enabled

self = <tests.debugger.test_debugger_inproduct_enablement.Test_Debugger_InProduct_Enablement_Exception_Replay object at 0x7fb4cd22fe60>

    def test_inproduct_enablement_exception_replay(self):
        self.assert_rc_state_not_error()
        self.assert_all_weblog_responses_ok(expected_code=500)
    
...

tests.debugger.test_debugger_probe_budgets.Test_Debugger_Probe_Budgets.test_log_line_budgets[uds] from system_tests_suite

(Fix with Cursor)

ValueError: Snapshot log303e2-1158-42db-854b-b855210df35b was not received.

self = <tests.debugger.test_debugger_probe_budgets.Test_Debugger_Probe_Budgets object at 0x7f116bb788c0>

    def test_log_line_budgets(self):
        self._assert()
>       self._validate_snapshots()

tests/debugger/test_debugger_probe_budgets.py:75: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
...

View all

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: a88d5f4 | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!}

zacharycmontoya mentioned this pull request Jul 11, 2025

feat: support additional cgroup formats for container-id parsing DataDog/dd-trace-cpp#222

Merged

zacharycmontoya marked this pull request as ready for review July 17, 2025 20:39

zacharycmontoya requested review from a team as code owners July 17, 2025 20:39

zacharycmontoya requested review from P403n1x87, jade-guiton-dd, quinna-h, sezen-datadog, smola and ygree and removed request for a team July 17, 2025 20:39

cbeauchesne reviewed Jul 18, 2025

View reviewed changes

cbeauchesne approved these changes Jul 18, 2025

View reviewed changes

zacharycmontoya added 4 commits April 9, 2026 08:26

Remove all /read_file weblog endpoints because they are no longer nec…

06e225d

…essary. They were only used for the container-id testing.

Fix lint issue

3aa0573

Restore test skip for cpp_nginx for test case 'test_trace_header_cont…

28c2335

…ainer_tags'

zacharycmontoya force-pushed the zach.montoya/fix-container-id-testing branch from 0ae53aa to 28c2335 Compare April 9, 2026 15:29

Fix lint/formatting issues after rebase

a88d5f4

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor container-id weblog test#4925

Refactor container-id weblog test#4925
zacharycmontoya wants to merge 5 commits into
mainfrom
zach.montoya/fix-container-id-testing

zacharycmontoya commented Jul 11, 2025

Uh oh!

cbeauchesne Jul 18, 2025

Uh oh!

zacharycmontoya Jul 22, 2025

Uh oh!

cbeauchesne Jul 23, 2025

Uh oh!

cbeauchesne left a comment

Uh oh!

github-actions Bot commented Apr 9, 2026

Uh oh!

datadog-prod-us1-3 Bot commented Apr 9, 2026 •

edited by datadog-datadog-prod-us1 Bot

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

zacharycmontoya commented Jul 11, 2025

Motivation

Changes

Workflow

Reviewer checklist

Uh oh!

cbeauchesne Jul 18, 2025

Choose a reason for hiding this comment

Uh oh!

zacharycmontoya Jul 22, 2025

Choose a reason for hiding this comment

Uh oh!

cbeauchesne Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

cbeauchesne left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Apr 9, 2026

Uh oh!

datadog-prod-us1-3 Bot commented Apr 9, 2026 • edited by datadog-datadog-prod-us1 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Other Violations

ℹ️ Info

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

datadog-prod-us1-3 Bot commented Apr 9, 2026 •

edited by datadog-datadog-prod-us1 Bot

Loading