docs(references): add cocoindex-code and model2vec reference analyses#36
Conversation
- cocoindex-code: prior-art comparison (dense-only MCP vs csp hybrid); includes published CoIR benchmark row and a caveat footnote on methodology differences - model2vec: analysis of the model2vec / model2vec-rs dependency used by csp's dense retrieval leg (potion-code-16M); includes MTEB and CoIR benchmark figures - index.md: registered both analyses in the Documents table and Sync baselines table Key finding: model2vec's own card shows dense+BM25 hybrid (40.41 CoIR) beats dense-only (37.05), directly validating csp's hybrid design over cocoindex's dense-only approach.
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughAdds two new reference documentation pages: ChangesReference Documentation Additions
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~4 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Up to standards ✅🟢 Issues
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds reference analysis documentation for CocoIndex Code (cocoindex.md) and Model2Vec (model2vec.md), and updates the main reference index. The review feedback correctly identifies broken wiki-style links ([[dense-embedding-is-a-stub]]) in both files that will not render properly in standard GitHub Markdown, and suggests replacing them with code blocks or relative links, along with fixing a minor typo (an extra double quote) in model2vec.md.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
… wiki-links Apply AI code review suggestions (gemini-code-assist): GitHub markdown does not render [[wiki-link]] syntax, and the target (dense-embedding-is-a-stub) is a memory note with no in-repo path. Switch the 3 occurrences to backtick code format.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.please/docs/references/index.md (1)
24-31:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winSoften the baseline wording for reference-only analyses.
These new entries are intentionally source-based (web docs / HF cards) and not commit-pinned, so “each analysis records the exact upstream commit” is now too strict. Please switch this to “revision or source set,” or add an explicit non-code analysis escape hatch.
📝 Suggested wording change
- Each analysis records the exact upstream commit it was written against. + Each analysis records the exact upstream revision or source set it was written against.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.please/docs/references/index.md around lines 24 - 31, The opening statement about analyses recording the exact upstream commit is too strict for entries like cocoindex-code and model2vec which use web docs and HF cards instead of commit-pinned sources. Update the introductory sentence (before the table) to accommodate both commit-pinned and source-based analyses by using broader language like "revision or source set" or by explicitly noting that some analyses are based on non-code sources like documentation and HF cards, ensuring the baseline description accurately reflects the mixed analysis approach shown in the table rows below.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.please/docs/references/cocoindex.md:
- Around line 40-41: The Embeddings row in the CSP column of the comparison
table currently states that TS/Rust "ship a deterministic stub until
integration", which misleadingly implies the Model2Vec integration is still
pending. Update the wording to clarify that the real Model2Vec static embeddings
integration is already implemented and loaded from
`crates/csp/src/indexing/dense.rs` via `model2vec-rs::StaticModel`, with the
deterministic stub serving only as a fallback when the model loading fails.
In @.please/docs/references/model2vec.md:
- Around line 5-8: The current wording in the model2vec documentation suggests
the model2vec-rs integration is still pending ("until integration lands"), but
the actual implementation in crates/csp/src/indexing/dense.rs already wires the
model2vec-rs StaticModel directly. Rephrase the sentence about the deterministic
stub fallback to clarify that the StaticModel is already integrated and the stub
is only used as a fallback when model loading fails, removing any language that
implies the integration is incomplete or pending.
---
Outside diff comments:
In @.please/docs/references/index.md:
- Around line 24-31: The opening statement about analyses recording the exact
upstream commit is too strict for entries like cocoindex-code and model2vec
which use web docs and HF cards instead of commit-pinned sources. Update the
introductory sentence (before the table) to accommodate both commit-pinned and
source-based analyses by using broader language like "revision or source set" or
by explicitly noting that some analyses are based on non-code sources like
documentation and HF cards, ensuring the baseline description accurately
reflects the mixed analysis approach shown in the table rows below.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: d73f0be4-de3f-4ac2-a7db-21d9afd493ea
📒 Files selected for processing (3)
.please/docs/references/cocoindex.md.please/docs/references/index.md.please/docs/references/model2vec.md
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
1 issue found and verified against the latest diff
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
Apply AI review (coderabbitai, cubic-dev-ai): the prior wording ('stub until
integration lands') implied the model2vec-rs wiring was still a TODO. Verified
against crates/csp/src/indexing/dense.rs — it already loads StaticModel via
StaticModel::from_pretrained; the deterministic stub is only a fallback on load
failure (offline/missing weights/bad path) or in tests. Reword cocoindex.md and
model2vec.md to reflect this, and keep the accurate note that the TS port's
dense signal is still a stub until Rust reaches parity.
1 participant
Summary
Adds two reference analyses under
.please/docs/references/and registers them inindex.md. These are reference materials only (not contracts), per the references/index.md preamble — they inform design decisions but do not constrain the implementation.Changes
.please/docs/references/cocoindex.md— Prior-art comparison of cocoindex-code, an independent AST-based code-search MCP server. Documents its dense-only retrieval approach (OpenAI embeddings), architecture, and a benchmark row with caveat footnote on methodology differences vs csp..please/docs/references/model2vec.md— Analysis of model2vec and its Rust port model2vec-rs, the dense-retrieval dependency csp uses (potion-code-16M). Includes published MTEB and CoIR benchmark figures..please/docs/references/index.md— Registered both new analyses in the Documents table and Sync baselines table.Key Finding
model2vec's own published card shows that dense+BM25 hybrid (CoIR 40.41) beats dense-only (37.05). This directly validates csp's hybrid design choice over cocoindex-code's dense-only path.
Test Plan
Summary by cubic
Docs-only: add reference analyses for
cocoindex-codeandmodel2vec, register them in.please/docs/references/index.md, fix[[...]]wiki-links (use backticked code), and clarify that the Rust dense path already usesmodel2vec-rswith a deterministic stub only as a fallback.The
model2veccard showspotion-code-16M+ BM25 hybrid (CoIR 40.41) beats dense-only (37.05), validating csp’s hybrid design overcocoindex-code’s dense-only path.Written for commit e22e340. Summary will update on new commits.
Summary by CodeRabbit
potion-code-16M, including the model lineup and benchmark context.