Skip to content

Add translation fallback tests#618

Merged
riderx merged 3 commits into
mainfrom
codex/translation-local-tests
May 1, 2026
Merged

Add translation fallback tests#618
riderx merged 3 commits into
mainfrom
codex/translation-local-tests

Conversation

@riderx
Copy link
Copy Markdown
Member

@riderx riderx commented May 1, 2026

Summary

  • add a local Bun test harness for the translation worker with fake AI, fake Cache API, and fake R2
  • reproduce the production failure mode where the model returns invalid batch JSON
  • fall back to single-string translation after batch JSON retries, so malformed batch output can still produce a cached localized page
  • keep queue retry behavior covered when both batch and single-string translation fail

Validation

  • bun run ci:verify:translation
  • cd apps/translation-worker && bunx wrangler deploy --dry-run
  • cd apps/translation-worker && bunx wrangler deploy --dry-run -e development

Summary by CodeRabbit

  • Tests

    • Added comprehensive tests for the translation worker covering fallback, single-item retry logic, and error-retry behaviors.
  • Bug Fixes

    • Improved batch translation robustness with fallback to individual text translation when batch processing fails.
    • Enhanced handling of varied AI response formats to ensure reliable translation results and avoid crashes.
  • Chores

    • CI now runs translation worker tests.
    • Added a test script to the translation worker package.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 1, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 7d81d21d-7a4d-4ded-8247-d6cc475b7092

📥 Commits

Reviewing files that changed from the base of the PR and between 1b06d49 and 78e5717.

📒 Files selected for processing (1)
  • apps/translation-worker/test/index.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • apps/translation-worker/test/index.test.ts

📝 Walkthrough

Walkthrough

Batch translation now falls back to per-string single-text translations when batch AI responses are invalid or exhausted; new parsing/stripping helpers, retry logic, tests and a test script were added, and TRANSLATION_CACHE_VERSION is exposed via __translationWorkerTest.

Changes

Cohort / File(s) Summary
Package scripts & CI
apps/translation-worker/package.json, package.json
Adds test script to translation-worker ("test":"bun test test") and updates CI to run translation-worker tests in addition to checks.
Translation worker core
apps/translation-worker/src/index.ts
Adds fallback from batch to per-item single-text translation with TRANSLATION_MODEL_ATTEMPTS/TRANSLATION_SINGLE_TEXT_ATTEMPTS retries, stricter single-text prompt, parsing helpers to extract/clean translations (strip code fences, unquote, handle varied AI shapes), logging/warnings for rejected responses, validation via assertTranslatedBatch, and exports __translationWorkerTest = { TRANSLATION_CACHE_VERSION }.
Tests
apps/translation-worker/test/index.test.ts
Adds Bun tests that mock cache/R2/AI.run and queue sender; verifies fallback behavior when batch returns non-JSON, validates HTTP response and cache header, and ensures queue retry is enqueued when AI fails completely.

Sequence Diagram

sequenceDiagram
    participant Client as Client
    participant Worker as Translation Worker
    participant AI as AI.run
    participant Validator as Validator
    participant Cache as Translation Cache

    Client->>Worker: Submit translation job (batch)
    Worker->>AI: Request batch translation (array)
    AI-->>Worker: Non-JSON / malformed response
    Worker->>Worker: Log error, begin fallback

    loop per input string
        Worker->>AI: Request single-text translation (stricter prompt)
        alt AI returns usable text
            AI-->>Worker: Raw translation (various shapes)
            Worker->>Validator: plainTranslationFromUnknown (strip/unquote)
            Validator-->>Worker: Parsed translated text
        else AI returns invalid / throws
            AI-->>Worker: Error / malformed
            Worker->>Worker: Retry (up to TRANSLATION_SINGLE_TEXT_ATTEMPTS)
        end
    end

    Worker->>Validator: assertTranslatedBatch
    Validator-->>Worker: Validated results
    Worker->>Cache: Store translations (update cache/version)
    Worker-->>Client: Return 200 + translated HTML (set X-Capgo-Translation-Cache)
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Poem

🐰 I nudge the batch that wouldn't play,
Then hop to each small phrase and say,
I strip the fences, calm the noise,
Retry with patience, coax out voice,
Tests and carrots guide the way.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Add translation fallback tests' accurately reflects the main changes: new tests for translation worker fallback behavior when batch JSON is invalid.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/translation-local-tests

Review rate limit: 2/5 reviews remaining, refill in 26 minutes and 19 seconds.

Comment @coderabbitai help to get the list of available commands and usage tips.

@riderx riderx marked this pull request as ready for review May 1, 2026 20:02
@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@riderx
Copy link
Copy Markdown
Member Author

riderx commented May 1, 2026

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 1, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@apps/translation-worker/test/index.test.ts`:
- Around line 94-103: The test setup replaces globalThis.caches in beforeEach
but never restores it; save the original caches reference at the top of the test
file (e.g., const originalCaches = (globalThis as any).caches) before beforeEach
runs, and in afterEach restore it (set (globalThis as any).caches =
originalCaches) alongside the existing restoration of originalWarn and
originalError; update the beforeEach/afterEach blocks that reference
globalThis.caches, MemoryCache, originalWarn, and originalError to use this
saved originalCaches so other tests aren't affected.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: a672dce8-cb68-4ce9-bab9-ec8cb2ece2a3

📥 Commits

Reviewing files that changed from the base of the PR and between 4232781 and 636f053.

📒 Files selected for processing (4)
  • apps/translation-worker/package.json
  • apps/translation-worker/src/index.ts
  • apps/translation-worker/test/index.test.ts
  • package.json

Comment thread apps/translation-worker/test/index.test.ts
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
apps/translation-worker/test/index.test.ts (1)

131-132: ⚡ Quick win

Avoid pinning the exact batch retry count here.

This assertion couples the test to the current retry policy rather than the fallback behavior. If the worker still falls back correctly but changes the number of batch attempts, this test will fail for an unrelated reason.

[details>

♻️ Suggested tweak
-    expect(calls.filter((input) => Array.isArray(lastUserPayload(input).texts))).toHaveLength(3)
+    expect(calls.some((input) => Array.isArray(lastUserPayload(input).texts))).toBe(true)
     expect(calls.some((input) => typeof lastUserPayload(input).text === 'string')).toBe(true)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@apps/translation-worker/test/index.test.ts` around lines 131 - 132, The test
currently asserts an exact number of batched retry attempts by expecting
calls.filter(...).toHaveLength(3), which couples it to the retry policy; change
this to assert behavior instead of count—e.g., verify at least one payload
contains an array of texts (use a "greater than 0" or "not empty" assertion on
calls.filter((input) => Array.isArray(lastUserPayload(input).texts)) ) and keep
the existing assertion that some payload has a string (calls.some((input) =>
typeof lastUserPayload(input).text === 'string')). This removes dependence on
the exact batch retry count while still validating fallback behavior in the
test.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@apps/translation-worker/test/index.test.ts`:
- Around line 131-132: The test currently asserts an exact number of batched
retry attempts by expecting calls.filter(...).toHaveLength(3), which couples it
to the retry policy; change this to assert behavior instead of count—e.g.,
verify at least one payload contains an array of texts (use a "greater than 0"
or "not empty" assertion on calls.filter((input) =>
Array.isArray(lastUserPayload(input).texts)) ) and keep the existing assertion
that some payload has a string (calls.some((input) => typeof
lastUserPayload(input).text === 'string')). This removes dependence on the exact
batch retry count while still validating fallback behavior in the test.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 0515ef2d-b5de-496b-ad5a-7507a0e371e6

📥 Commits

Reviewing files that changed from the base of the PR and between 636f053 and 1b06d49.

📒 Files selected for processing (1)
  • apps/translation-worker/test/index.test.ts

@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented May 1, 2026

@riderx riderx merged commit c2bb57b into main May 1, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant