fix: isolate embedding staging dir to prevent cross-model contamination

[vojtech-cifka]

Why: embedding extraction staging dir was shared across models, risking cross-model contamination of cached embeddings.

What: isolate staging dir per model run.

Summary by CodeRabbit

• Bug Fixes
  • Improved processing reliability by staging split outputs in isolated temporary directories.
  • Automatically creates the configured output location when it does not already exist.
  • Prevents stale tile data from persisting between processing runs.

[coderabbitai]

Warning

Review limit reached

@vojtech-cifka, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 57 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c67db5a3-0176-4a47-b3a7-962065598e43

📥 Commits

Reviewing files that changed from the base of the PR and between fa85342 and 4167bb4.

📒 Files selected for processing (1)

• preprocessing/embeddings.py

📝 Walkthrough

Walkthrough

Changes

Embedding output staging

Layer / File(s)	Summary
Stage per-split embedding outputs preprocessing/embeddings.py	Replaces persistent split-directory writes and tile cleanup with TemporaryDirectory staging under config.output_dir, writing slides.parquet and tiles, then logging the staged directory.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: vejtek, matejpekar

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main change: isolating embedding staging directories to avoid cross-model contamination.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/embedding-dedup-isolation

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (1)

preprocessing/embeddings.py (1)

224-241: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Core staging logic is sound.

The isolated TemporaryDirectory approach correctly prevents cross-run contamination, the artifact layout (split_dir containing slides.parquet and tiles/) matches the downstream contract (runs:/{embedding_run_id}/{split}/tiles), and auto-cleanup on context exit is safe since MLflow log_artifacts copies files.

One minor nit: Path(config.output_dir).mkdir(parents=True, exist_ok=True) at line 228 is inside the per-split loop and runs redundantly on every iteration. Consider moving it before the loop.

♻️ Move mkdir before the loop

    Path(config.output_dir).mkdir(parents=True, exist_ok=True)
+
    for name in config.get("splits", ["train", "test"]):
        # ... existing loop body ...
        # Isolated per-run staging dir: avoids cross-model contamination and
        # stale-file carryover from concurrent or previous runs sharing
        # output_dir. Auto-removed after upload; nothing reads it afterward
        # (training reads the mlflow run artifacts via runs:/ URIs).
-       Path(config.output_dir).mkdir(parents=True, exist_ok=True)
        with tempfile.TemporaryDirectory(dir=config.output_dir) as tmp_root:

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@preprocessing/embeddings.py` around lines 224 - 241, Move the
Path(config.output_dir).mkdir(parents=True, exist_ok=True) initialization out of
the per-split loop and execute it once before iteration begins. Keep the
TemporaryDirectory staging and per-split artifact generation unchanged.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@preprocessing/embeddings.py`:
- Around line 224-241: Move the Path(config.output_dir).mkdir(parents=True,
exist_ok=True) initialization out of the per-split loop and execute it once
before iteration begins. Keep the TemporaryDirectory staging and per-split
artifact generation unchanged.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 9a9714ff-c2c7-4f70-8ecf-edc479960d54

📥 Commits

Reviewing files that changed from the base of the PR and between 0e6bf51 and fa85342.

📒 Files selected for processing (1)

• preprocessing/embeddings.py

[gemini-code-assist]

Code Review

This pull request refactors the preprocessing pipeline in preprocessing/embeddings.py to use a temporary directory (tempfile.TemporaryDirectory) for staging output files instead of writing directly to a shared output directory. This ensures isolated concurrent runs and prevents stale-file carryover. There are no review comments, and I have no additional feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.