Add support for bounded trie metric in legacy worker

[rohitsinha54]

• Adds support for bounded trie in Dataflow legacy workers.
• Streaming reports delta counters. This is essential because without it streaming will end up remaining same counters as cumulative through it's entire lifespan. This will be bad even with bounded trie which is self-limiting in size because in backend the level db monitoring writer keeps taking snapshot of counters on publish and writing to level db and for a long running job even with self-limiting data store the file sizes will keep growing.

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

[github-actions]

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

[github-actions]

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @johnjcasey added as fallback since no labels match configuration

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

[rohitsinha54]

R: @robertwb

[github-actions]

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment assign set of reviewers

[rohitsinha54]

@robertwb Please take a look. Thanks.

[rohitsinha54]

https://yaqs.corp.google.com/eng/q/775343714070429696

[rohitsinha54]

The two failure seem unrelated to this change.

[robertwb]

Thanks, this looks good. Just one small question.

[robertwb]

I'm still seeing

2025-02-05T18:44:03.8370351Z > Task :runners:google-cloud-dataflow-java:worker:test
2025-02-05T18:44:03.8371552Z 
2025-02-05T18:44:03.8373642Z org.apache.beam.runners.dataflow.worker.BatchModeExecutionContextTest > extractMetricUpdatesBoundedTrie FAILED

in the failed test run.

[rohitsinha54]

Sorry. Fixed.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 59.09%. Comparing base (fc43c12) to head (e6c55be).
Report is 24 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff            @@
##             master   #33474   +/-   ##
=========================================
  Coverage     59.08%   59.09%           
- Complexity     3238     3240    +2     
=========================================
  Files          1156     1156           
  Lines        176924   176924           
  Branches       3391     3391           
=========================================
+ Hits         104543   104553   +10     
+ Misses        69015    69008    -7     
+ Partials       3366     3363    -3     

Flag	Coverage Δ
java	70.36% <ø> (+0.03%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[rohitsinha54]

Failing test testOnNewWorkerMetadata_redistributesBudget

is not related to change. I also ran locally and it passed.

[rohitsinha54]

@robertwb can you please help merging this? Thanks.

[robertwb]

Yes. I was trying to get the tests to pass, but this does look unrelated (and possibly flaky). I'll go ahead and merge.

[rohitsinha54]

Thank you.

Created a CP to 2.63: #33890