fix: ensure sequence metadata list accesses are threadsafe in task runner

[jtuglu1]

Description

Fixes #19458 and cleans up SeekableStreamIndexTaskRunnerTest.java

Old code had racy logic which was performing size checks on COWsequences in multiple places then attempting to index into the sequences list. This size check could be invalidated by a publishing thread calling .remove() on the between the size() check and the index operation.

Race:

1. [main runner thread]: calls getLastSequenceMetadata() here.
2. [main runner thread]: in the definition, sequences.size() returns 2 here
3. [publisher thread]: publisher thread removes element from the sequences list.
4. [main runner thread]: sequences.get() throws OOB error due to stale read race.

Fix

While technically this issue can be solved by taking snapshots before doing multiple reads on the sequence list, that still doesn't prevent read/write inter-leavings that might cause temporary lapses in in-memory/on-disk state (especially if we crash). So, I guarded the sequence list with a reentrant lock. While we re-acquire this lock per record, running this under load/taking some flamegraphs did not add any noticeable overhead. I believe this is mainly because the lock is not frequently contended in the common case and bottlenecks in the ingestion code lie elsewhere. Switching to a rw lock is another option here but performance was the same between the two options. Will add the performance benchmarks soon.

Release note

Fix fatal race in streaming ingest task during segment publish.

---

This PR has:

• been self-reviewed.
  • using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)
• added documentation for new or modified features or behaviors.
• a release note entry in the PR description.
• added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
• added or updated version, license, or notice information in licenses.yaml
• added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
• added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
• added integration tests.
• been tested in a test Druid cluster.

[FrankChen021]

I have reviewed the code for correctness, edge cases, concurrency, and integration risks; no issues found.

Reviewed 3 of 3 changed files.

---

This is an automated review by Codex GPT-5.5