Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .claude/settings.json
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
{
"enabledPlugins": {
"typescript-lsp@code-intelligence": true,
"rust-analyzer-lsp@code-intelligence": true,
"rust-analyzer-lsp@code-intelligence": true,
"eslint-lsp@code-intelligence": true,
"bun@pleaseai": true,
"claude-md-management@claude-plugins-official": true,
"code-review@pleaseai": true,
"please@passionfactory": true,
"standards@passionfactory": true,
"review@passionfactory": true
"review@passionfactory": true,
"plannotator@passionfactory": true
}
}
11 changes: 0 additions & 11 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,14 +33,3 @@ jobs:

- name: Lint
run: bun run lint

- name: Test with coverage
run: bun test --coverage --coverage-reporter=lcov

- name: Upload coverage to Codecov
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with:
files: ./coverage/lcov.info
fail_ci_if_error: false
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
14 changes: 14 additions & 0 deletions .github/workflows/release-please.yml
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,20 @@ jobs:
with:
tag: ${{ needs.release-please.outputs.tag_name }}

# Build + publish the napi-rs SDK (@pleaseai/csp-sdk) — the separate in-process
# native-addon channel. Reuses release-sdk.yml so its cross-compile matrix
# lives in one place. id-token: write is granted here (the caller) so the
# reusable workflow's publish job can use npm Trusted Publishing (OIDC).
build-and-publish-sdk:
needs: release-please
if: ${{ needs.release-please.outputs.release_created }}
permissions:
contents: read
id-token: write
uses: ./.github/workflows/release-sdk.yml
with:
publish: true

# Generate the per-platform npm packages from the released binaries and
# publish the wrapper + platform packages via npm Trusted Publishing (OIDC).
# No NPM_TOKEN needed — auth is the OIDC id-token, and provenance is generated
Expand Down
181 changes: 181 additions & 0 deletions .github/workflows/release-sdk.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
# napi-rs SDK release pipeline — the @pleaseai/csp-sdk in-process native addon.
#
# Distinct from release-rust.yml (which ships the standalone `csp` binary + its
# npm launcher). This builds `csp-sdk.<platform>.node` for each target from
# crates/csp-node, then publishes the wrapper + per-platform packages to npm via
# Trusted Publishing (OIDC — no NPM_TOKEN; provenance is automatic).
#
# Runs two ways: manually (workflow_dispatch) for ad-hoc rebuilds, and as a
# reusable workflow (workflow_call) invoked by release-please.yml on
# release_created. Each per-platform package (@pleaseai/csp-sdk-*) and the
# wrapper (@pleaseai/csp-sdk) must have a trusted publisher configured on
# npmjs.com pointing at this repo + workflow before the publish job can succeed.

name: Release (SDK)

on:
workflow_dispatch:
inputs:
publish:
description: Publish to npm after building (needs trusted-publisher config). Leave false to only build artifacts.
required: false
type: boolean
default: false
workflow_call:
inputs:
publish:
description: Publish to npm after building.
required: false
type: boolean
default: true

permissions:
contents: read

concurrency:
group: release-sdk-${{ github.ref }}
cancel-in-progress: false

jobs:
build:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
include:
- os: macos-14 # Apple Silicon
target: aarch64-apple-darwin
- os: macos-15-intel # Intel
target: x86_64-apple-darwin
- os: ubuntu-latest
target: x86_64-unknown-linux-gnu
- os: ubuntu-24.04-arm
target: aarch64-unknown-linux-gnu
- os: ubuntu-latest
target: x86_64-unknown-linux-musl
- os: windows-latest
target: x86_64-pc-windows-msvc

steps:
- name: Checkout code
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
with:
# Build-only job — it never pushes, so don't persist the token.
persist-credentials: false

Comment thread
coderabbitai[bot] marked this conversation as resolved.
# rust-toolchain.toml pins the toolchain; add the target triple so the
# cross-target build resolves its std.
- name: Add Rust target
run: rustup target add ${{ matrix.target }}

# @napi-rs/cli is a Node tool (devDependency of crates/csp-node). Bun
# installs it; the runner's preinstalled Node runs the `napi` binary.
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
with:
bun-version: 1.3.14

- name: Install SDK build tooling
run: bun install
working-directory: crates/csp-node

# musl pulls C/C++ deps (esaxx-rs C++ via tokenizers, onig/zstd C) and
# musl-tools ships only musl-gcc, not musl-g++. cargo-zigbuild + zig is a
# full C/C++ cross toolchain; napi's `--zig` flag routes the build through
# it. zig is minisign-verified against the official key (mirrors
# release-rust.yml) before it is extracted and run.
- name: Set up cargo-zigbuild (musl)
if: ${{ endsWith(matrix.target, '-musl') }}
run: |
ZIG_VERSION=0.13.0
ZIG_PUBKEY="RWSGOq2NVecA2UPNdBUZykf1CCb147pkmdtYxgb3Ti+JO/wCYvhbAb/U"
TARBALL="zig-linux-x86_64-${ZIG_VERSION}.tar.xz"
sudo apt-get update && sudo apt-get install -y minisign
curl -fsSLO "https://ziglang.org/download/${ZIG_VERSION}/${TARBALL}"
curl -fsSLO "https://ziglang.org/download/${ZIG_VERSION}/${TARBALL}.minisig"
minisign -Vm "${TARBALL}" -P "${ZIG_PUBKEY}"
tar -xJf "${TARBALL}" -C "$RUNNER_TEMP"
echo "$RUNNER_TEMP/zig-linux-x86_64-${ZIG_VERSION}" >> "$GITHUB_PATH"
cargo install --locked cargo-zigbuild --version '^0.19'

- name: Build native addon
shell: bash
working-directory: crates/csp-node
run: |
if [[ "${{ matrix.target }}" == *-musl ]]; then
bunx napi build --platform --release --target "${{ matrix.target }}" --zig
else
bunx napi build --platform --release --target "${{ matrix.target }}"
fi
ls -lh ./*.node

- name: Upload artifact
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: csp-sdk-${{ matrix.target }}
path: crates/csp-node/*.node
if-no-files-found: error

# Assemble the per-platform packages from the built addons and publish them +
# the wrapper via npm Trusted Publishing. Resilient: publishes whatever
# targets built (always()), and fails loudly if nothing built.
publish:
needs: build
if: ${{ always() && inputs.publish }}
runs-on: ubuntu-latest
permissions:
contents: read
id-token: write
steps:
- name: Checkout code
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
with:
# Publish-only; never pushes to git, so don't persist the token.
persist-credentials: false

- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
with:
bun-version: 1.3.14

# Trusted Publishing requires npm >= 11.5.1; ubuntu-latest ships npm 10.x.
- name: Upgrade npm for Trusted Publishing
run: sudo npm install -g npm@latest

- name: Install SDK build tooling
run: bun install
working-directory: crates/csp-node

- name: Download built addons
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
pattern: csp-sdk-*
merge-multiple: true
path: crates/csp-node/artifacts

# `create-npm-dirs` materializes the npm/<platform>/ package dirs from the
# `napi.triples` config; `artifacts` moves each built .node into its dir.
- name: Assemble platform packages
working-directory: crates/csp-node
run: |
bunx napi create-npm-dirs
bunx napi artifacts --dir artifacts
ls -R npm

# OIDC supplies auth + provenance, so no token / --provenance flag.
# Platform packages first; the wrapper last (its optionalDependencies pin
# them by version). Skip any platform dir that has no addon (partial matrix).
- name: Publish to npm
working-directory: crates/csp-node
run: |
set -e
for dir in npm/*/; do
if compgen -G "$dir/*.node" > /dev/null; then
echo "publishing platform package: $dir"
npm publish "$dir" --access public
else
echo "skipping $dir (no addon built for this target)"
fi
done
echo "publishing wrapper: @pleaseai/csp-sdk"
npm publish --access public
44 changes: 25 additions & 19 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,18 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co

## Project context

`@pleaseai/csp` (binary: `csp`) is a TypeScript/Bun port of [MinishLab/semble](https://github.com/MinishLab/semble), a Python hybrid code-search library for agents. The current repo is an **initial scaffold only** — `src/index.ts` and `src/cli.ts` are placeholders. The README is the canonical spec for the intended public surface (MCP server, CLI, library).
`@pleaseai/csp` (binary: `csp`) is a **Rust** port of [MinishLab/semble](https://github.com/MinishLab/semble), a Python hybrid code-search library for agents. The implementation lives in `crates/csp` (library) + `crates/csp-cli` (`csp` binary). The README is the canonical spec for the public surface (MCP server, CLI, library).

### Rust rewrite (ADR-0003)
The deprecated TypeScript implementation that formerly lived under `src/` has been **removed** — the Rust port is the only implementation. The root `package.json` / `tsconfig.json` / `eslint.config.ts` remain only as repo JS tooling (lint/typecheck of `npm/`) and the release-please version anchor. The **napi-rs native-binding SDK** binds the `crates/` Rust directly (it does not reintroduce a TS port).

A Rust port lives in `crates/csp` (library) + `crates/csp-cli` (`csp` binary). **The Python upstream ([MinishLab/semble](https://github.com/MinishLab/semble)) is the source of truth** — the Rust port targets behavioral equivalence with the upstream Python. The TS `src/` is **deprecated**: slated for deletion and retained only as a historical/reference implementation; it is **no longer** the source of truth or the parity oracle.
**SDK packaging decision: keep the two distribution channels separate.** `npm/` stays the Biome-style CLI/MCP launcher — a thin Node shim that execs the **standalone Rust binary** (this preserves the no-runtime Homebrew story; do NOT convert it to napi). The napi-rs SDK is a distinct concern: `crates/csp-node` holds `#[napi]` bindings over `crates/csp` and is shipped as its **own npm package** (`@pleaseai/csp-sdk`), an in-process native addon — not merged into `npm/`. Both build outputs share the one `crates/csp` core. The SDK is in place: `#[napi]` bindings (`fromPath`/`fromGit`/`loadFromDisk` are async on the libuv worker pool; `search`/`findRelated`/`save`/`stats` sync, with `inner` held behind `Arc` to enable a future async move), the `napi build` toolchain (`.node` + `index.js`; `index.d.ts` is the committed type surface), and the cross-compile + Trusted-Publishing release in `release-sdk.yml`. The remaining step is publish-only — a maintainer must configure the npm trusted publisher for `@pleaseai/csp-sdk` + its platform packages (see `crates/csp-node/README.md`).

### Rust port (ADR-0003)

**The Python upstream ([MinishLab/semble](https://github.com/MinishLab/semble)) is the source of truth** — the Rust port targets behavioral equivalence with the upstream Python.
- Quality gate before every Rust commit: `cargo fmt --all && cargo clippy --all-targets --all-features -- -D warnings && cargo test --workspace`.
- Parity oracle = the **Python upstream** behavior (read the source directly — see the fetch note below). The TS test suite stays usable as language-neutral golden fixtures for already-ported modules, but is not authoritative where it disagrees with upstream. The Rust port has intentionally moved **past the old TS stubs** to match upstream: dense embeddings are real (`model2vec-rs`, not the deterministic stub), the ranking pipeline is wired (query boosts + path penalties + file saturation), and the chunk length is `750`. The TS `src/` still carries the older stubs/values until it is removed.
- CLI/MCP output is a **snake_case** wire dict (`csp::utils::format_results`, mirroring TS `SearchResult.toDict`), distinct from the camelCase `ChunkDict` used for on-disk persistence.
- Parity oracle = the **Python upstream** behavior (read the source directly — see the fetch note below). Dense embeddings are real (`model2vec-rs`), the ranking pipeline is wired (query boosts + path penalties + file saturation), and the chunk length is `750`.
- CLI/MCP output is a **snake_case** wire dict (`csp::utils::format_results`, mirroring upstream `SearchResult.to_dict`), distinct from the camelCase `ChunkDict` used for on-disk persistence.
- rmcp 1.7: the default `#[tool_handler]` rebuilds the router via `Self::tool_router()` and leaves a stored `tool_router` field unread (clippy `dead_code`) — use `#[tool_handler(router = self.tool_router)]`.

When porting modules from semble, fetch the upstream source and read the Python directly:
Expand All @@ -22,32 +26,34 @@ ask src github:MinishLab/semble@main # absolute path to the cached checkout (
curl -fsSL https://raw.githubusercontent.com/MinishLab/semble/main/src/semble/search.py
```

Read the Python source directly — do not infer behavior from the README. Key upstream modules and their target TS counterparts live under `src/semble/` (Python): `types.py`, `tokens.py`, `chunking/`, `index/` (files, file_walker, dense, sparse, create, index), `ranking/` (boosting, penalties, weighting), `search.py`, `mcp.py`, `cli.py`, `cache.py`, `stats.py`, `utils.py`.
Read the Python source directly — do not infer behavior from the README. Key upstream modules (mapped to their `crates/csp` Rust counterparts in `.please/docs/references/semble.md`) live under `src/semble/` (Python): `types.py`, `tokens.py`, `chunking/`, `index/` (files, file_walker, dense, sparse, create, index), `ranking/` (boosting, penalties, weighting), `search.py`, `mcp.py`, `cli.py`, `cache.py`, `stats.py`, `utils.py`.

## Stack

- **Runtime / package manager**: Bun 1.3.10+ (`packageManager` pinned in `package.json`). Node.js 22+ supported.
- **Module system**: ESM only (`"type": "module"`). Use `.ts` imports with `verbatimModuleSyntax`.
- **Build**: `tsdown` — config at `tsdown.config.ts`, two entries (`src/index.ts`, `src/cli.ts`), `unbundle: true`, emits ESM + DTS into `dist/`.
- **Lint**: `@pleaseai/eslint-config` (wraps `@antfu/eslint-config`). Flat config at `eslint.config.ts`. No semicolons, single quotes, 2-space indent. Type-aware rules enabled via `tsconfigPath`.
- **TypeScript**: strict + `noUncheckedIndexedAccess` + `exactOptionalPropertyTypes` + `verbatimModuleSyntax`. Target ES2022, `moduleResolution: bundler`.
- **Tests**: `bun:test` (no jest/vitest). Run with `bun test`.
The implementation is **Rust** (a Cargo workspace). A thin Node/Bun toolchain remains for repo-level JS lint/typecheck and the future napi-rs SDK.

- **Impl**: Rust, edition 2021. Cargo workspace (`crates/csp` lib + `crates/csp-cli` `csp` binary), toolchain pinned by `rust-toolchain.toml`. Single-binary release profile (`lto`, `codegen-units=1`, `strip`).
- **Tests**: `cargo test --workspace` (255+ lib + CLI tests). Network-gated grammar-fetch tests run with `-- --ignored` (see ADR-0004).
- **Distribution**: self-contained Rust binary via Homebrew (`pleaseai/homebrew-tap`) + an npm wrapper under `npm/` that preserves the `bunx @pleaseai/csp` entrypoint.
- **JS tooling** (no TS implementation): Bun 1.3.10+ / Node 22+. `@pleaseai/eslint-config` (wraps `@antfu/eslint-config`) lints `npm/` JS + `eslint.config.ts`; `tsc --noEmit` typechecks. No semicolons, single quotes, 2-space indent.

## Commands

```bash
bun install # install deps
bun run build # tsdown build → dist/
bun run dev # tsdown --watch
# Rust (the implementation)
cargo build --release # → target/release/csp
cargo run -p csp-cli -- search "query" . # run the CLI locally
cargo test --workspace # test runner
cargo fmt --all && cargo clippy --all-targets --all-features -- -D warnings # pre-commit gate

# JS tooling (lint/typecheck of npm/ + configs; no TS sources to build)
bun install # install dev tooling
bun run typecheck # tsc --noEmit
bun run lint # eslint . --cache
bun run lint:fix # eslint . --fix --cache
bun test # bun:test runner
bun test path/to/file.test.ts # single file
bun test --watch # watch mode
```

`bunx @pleaseai/csp` is the published-package entrypoint referenced throughout the README (MCP/CLI setup snippets). Locally, use `bun run --bun src/cli.ts` or build first.
`bunx @pleaseai/csp` is the published-package entrypoint referenced throughout the README (MCP/CLI setup snippets); it resolves to the npm wrapper (`npm/`) that execs the Rust binary.

## Public API surface (target, from README)

Expand Down
Loading
Loading