-
Notifications
You must be signed in to change notification settings - Fork 0
Fix flaky defineCodeJudge test failing in full suite #437
Copy link
Copy link
Closed
Description
Problem
The test CodeEvaluator > works with defineCodeJudge-based code judge in packages/core/test/evaluation/code-evaluator-file-backed.test.ts passes when run individually but fails when run as part of the full test suite.
This is a test isolation issue — some shared state from other tests affects this test's outcome.
Reproduction
# Passes individually:
bun test packages/core/test/evaluation/code-evaluator-file-backed.test.ts
# Fails in full suite:
bun testExpected Behavior
The test should pass reliably regardless of execution order or which other tests run alongside it.
Likely Cause
Test isolation issue — possibly shared module state, global mocks, or file system side effects from other tests leaking into this one.
Acceptance Signals
- Test passes consistently in full suite (
bun test) across multiple runs - No other tests broken by the fix
- Root cause documented in commit message
Related
Discovered during #431 / PR #436 implementation (855/856 tests passing, this was the 1 pre-existing failure).
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels