fix: keep TMK duplicate evidence content-bound

[seonghobae]

Summary

• expose audio and TMK exact-duplicate groups separately in the Rust inventory
• quarantine only current, content-verified TMK duplicates through the Python API/Rust mutation boundary
• keep stale iCloud/File Provider SHA hints unverified and document the evidence rule
• add bounded direct-read policy for small sidecars when provider status is unavailable

Validation

• python3 -m pytest -q — 629 passed, 751 subtests passed
• python3 -m ruff check audio_library.py tests/test_audio_library.py
• cargo fmt --manifest-path rust-core/Cargo.toml -- --check
• cargo test --manifest-path rust-core/Cargo.toml --release — 23 passed

Summary by CodeRabbit

• 개선 사항

  • 전사 결과에서 타임스탬프·화자 제어 토큰만 포함된 불필요한 세그먼트와 대체 텍스트를 자동으로 제거합니다.
  • 장시간 오디오 처리 후 메모리와 GPU 캐시를 정리해 안정성을 높였습니다.
  • 오디오와 TMK 파일의 중복을 별도로 관리하고, 검증된 중복 TMK 파일을 안전하게 격리합니다.
  • 파일 제공자 상태가 불명확해도 작은 파일은 직접 읽고, 큰 오디오는 안정적인 조정 경로로 처리합니다.

• 문서

  • 중복 파일 처리 및 파일 읽기 동작에 대한 설명을 보완했습니다.

[coderabbitai]

Warning

Review limit reached

@seonghobae, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 19 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 83e9962b-3cbf-4ac1-a5b9-1a965b84c5e9

📥 Commits

Reviewing files that changed from the base of the PR and between 9302a74 and 7076211.

📒 Files selected for processing (3)

• audio_library.py
• rust-core/src/lib.rs
• tests/test_audio_library.py

📝 Walkthrough

Walkthrough

File Provider 상태가 불명확할 때 직접 읽기를 8 MiB 이하 파일로 제한합니다. 인벤토리는 TMK 중복 그룹을 오디오 중복 그룹과 분리합니다. 전사 처리는 제어 토큰 전용 출력을 제거하고 청크별 메모리 정리를 수행합니다.

Changes

인벤토리 및 전사 처리

Layer / File(s)	Summary
제한된 직접 읽기와 인벤토리 계약 rust-core/src/lib.rs, docs/architecture/gpu-transcription-rust-backend.md, README.md	File Provider 상태와 파일 크기에 따라 직접 읽기와 조정된 읽기를 선택합니다. InventoryManifest에 기본값을 사용하는 tmk_duplicate_groups 필드를 추가합니다. 관련 동작과 테스트를 문서화합니다.
TMK 중복 검증과 격리 audio_library.py, README.md, tests/test_audio_library.py	검증된 SHA-256 기준으로 오디오와 TMK 중복 그룹을 분리합니다. 선택된 오디오에 연결된 검증된 TMK 중복 파일만 격리 계획에 추가합니다.
전사 출력 정리와 청크 메모리 정리 audio_library.py, tests/test_audio_library.py	MOSS 및 fallback 결과에서 제어 토큰 전용 텍스트를 제외합니다. 각 MLX 청크 처리 후 원시 결과를 해제하고 gc.collect 및 MLX 캐시 정리를 호출합니다. 테스트는 숫자 텍스트, 제어 토큰 출력, 청크별 정리를 검증합니다.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

• ContextualWisdomLab/codec-carver#396: Rust 인벤토리 및 스테이징의 TMK 처리와 직접 읽기 제한을 확장한 변경입니다.
• ContextualWisdomLab/codec-carver#399: GpuTranscriber의 청크별 MLX 및 Python 메모리 정리를 확장하고 회귀 테스트를 추가한 변경입니다.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 65.38% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	제목은 검증된 콘텐츠에 기반한 TMK 중복 증거 처리라는 주요 변경 사항을 정확하고 간결하게 설명합니다.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/gpu-transcription-rust-backend

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@rust-core/src/lib.rs`:
- Line 66: Apply #[cfg(any(target_os = "macos", test))] to
MAX_UNKNOWN_PROVIDER_DIRECT_READ_BYTES so its compilation conditions match
should_try_direct_read and prevent unused-constant warnings on non-macOS,
non-test builds.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: b17b27e9-d72e-46a2-b215-c2027c32f7c0

📥 Commits

Reviewing files that changed from the base of the PR and between 40f36cb and 9302a74.

📒 Files selected for processing (5)

• README.md
• audio_library.py
• docs/architecture/gpu-transcription-rust-backend.md
• rust-core/src/lib.rs
• tests/test_audio_library.py