feat(ranking): port path penalties + rerankTopK from semble

[amondnet]

Summary

Worker Unit 6: ports src/semble/ranking/penalties.py → src/ranking/penalties.ts.

What's in

• Regexes (verbatim from Python):
  • TEST_FILE_RE — multi-language test-file alternation (Python, Go, Java, PHP, Ruby, JS/TS, Kotlin, Swift, C#, C/C++, Scala, Dart, Lua, plus shared test_helpers? helpers).
  • TEST_DIR_RE, COMPAT_DIR_RE, EXAMPLES_DIR_RE, TYPE_DEFS_RE.
• Constants: STRONG_PENALTY=0.3, MODERATE_PENALTY=0.5, MILD_PENALTY=0.7, FILE_SATURATION_THRESHOLD=1, FILE_SATURATION_DECAY=0.5.
• REEXPORT_FILENAMES: Set(['__init__.py', 'package-info.java']).
• _filePathPenalty(filePath): multiplicative combination over normalised path; basename check uses raw filePath to match Python's POSIX Path.name semantics.
• rerankTopK(scores, topK, { penalisePaths }): single-sort greedy pass with file-saturation decay; early break when selected >= topK && penScore <= minSelected.

Notes

• Chunk type is inlined with a comment pointing to Unit 1 (src/types.ts) — will be replaced once that lands.
• The Python code OR's TEST_FILE_RE and TEST_DIR_RE into a single penalty branch, so tests/test_foo.py gets one STRONG_PENALTY (0.3), not 0.09. The tests reflect this (and the worker-prompt expectation has been double-checked against running Python).
• Saturation decay formula matches Python: FILE_SATURATION_DECAY ** (already_selected - threshold + 1).

Verification

• 21 unit tests pass: bun test src/ranking/penalties.test.ts.
• _filePathPenalty outputs cross-checked against running the original Python source for 10+ inputs (including tests/test_foo.py, examples/foo.test.ts, src/__init__.d.ts, src\\foo.test.ts, etc.).
• TEST_FILE_RE matches verified identical to Python across 34 positive/negative cases.

Out of scope

• e2e (skipped per worker instructions).
• package.json is untouched.
• Wiring Chunk to src/types.ts waits on Unit 1.

---

Summary by cubic

Ported path-based penalties and rerankTopK from Semble to TypeScript to improve ranking by downweighting tests/compat/examples and limiting repeated chunks from the same file. Adds parity tests and a guard for non‑positive topK.

• New Features

  • Multi-language TEST_FILE_RE plus dir regexes for tests, compat/legacy, examples, and .d.ts.
  • Penalty constants and REEXPORT_FILENAMES for re-export barrels.
  • _filePathPenalty(filePath) combines applicable penalties with POSIX-style basename.
  • rerankTopK(scores, topK, { penalisePaths }) applies path penalties and per-file saturation decay with a single sort and early exit.
  • 21 unit tests for penalties, decay, sorting, and topK edge cases.

• Bug Fixes

  • Return [] when topK <= 0 to avoid negative slice behavior; tests added.

^{Written for commit 8dd26b0. Summary will update on new commits.}

[gemini-code-assist]

Code Review

This pull request introduces path penalties and saturation decay logic for reranking search chunks, along with comprehensive unit tests. The feedback highlights a performance bottleneck in the greedy selection loop of rerankTopK that leads to O(N^2) complexity, suggesting a sorted array approach to optimize it. Additionally, it is recommended to handle non-positive topK values defensively to prevent incorrect slicing behavior.

[cubic-dev-ai]

No issues found across 2 files

_{Re-trigger cubic}

[amondnet]

Applied 1, deferred 1.

• Applied (medium): defensive topK <= 0 guard with regression tests covering 0, -1, -5 (commit 8dd26b0).
• Deferred (high): O(N²) greedy-loop rewrite — the suggested top-K sorted array changes two load-bearing semantics vs. semble's rerank_topk (unbounded selected array; early-exit min computed over all selected, not just top-K). Diverging requires either an upstream change or parity-snapshot tests proving the rewrite is equivalent. Filed as follow-up.