refactor!: streaming API redesign - #61
Merged
Merged
Conversation
Replaces the v1 parser with four orthogonal primitives, each doing one thing.
The previous all-in-one `parseSession` returned the full transcript inline,
forcing every consumer to load up to 80 MB of turns even for analytics-only
workloads. The new shape lets callers fetch only what they need and stream
the rest.
API changes:
- Add `readSessionIds(dir)` (was internal `listSessionIds`)
- Rename `parseSession` -> `readSession` (now returns aggregate metadata only,
no `turns` / `subagentTurns` in `ParsedSession`)
- Add `readSessionTurns(dir, id)` async generator over the main transcript
- Add `readSubagentTurns(dir, id)` async generator over `<sessionId>/subagents/`,
yielding `{ agentId, turn }` tagged tuples
- Remove `parseAllSessions` (callers compose `readSessionIds` + their own
concurrency strategy)
Internal:
- Add `bun run ci` script chaining check + typecheck + test --coverage + typedoc
- Cap subagent directory recursion at depth 8 (symlink loop guard)
- Document the asymmetry between `userMessageCount` (stricter than CC,
excludes subagent / meta / compactSummary) and `assistantMessageCount`
(includes subagent turns, dedupes on messageId+requestId)
BREAKING CHANGE: `parseSession`, `parseAllSessions`, `ParsedSession.turns`
and `ParsedSession.subagentTurns` are removed. Migrate to `readSession`
(metadata) + `readSessionTurns` / `readSubagentTurns` (streamed turns).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the v1 parser with four orthogonal primitives, each doing one thing. The previous all-in-one
parseSessionreturned the full transcript inline, forcing every consumer to load up to 80 MB of turns even for analytics-only workloads. The new shape lets callers fetch only what they need and stream the rest.API changes
Why
readSessionstays light;readSessionTurnsstreams.parseAllSessionsdidPromise.allover every session with no concurrency cap → OOM on large histories. The new shape forces the caller to pick its strategy.readSubagentTurnsexposes the previously-internal<sessionId>/subagents/walker, including nested layouts (workflows/<runId>/agent-*.jsonl).gray-matter/marked(no orchestration baked into the lib).Validation
~/.claude/projects/(12 projects, 64 sessions, 80 MB peak): 0 errors, 14332 turns streamed in 659 ms with bounded heap (76 → 32 MB)Breaking changes
parseAllSessions,parseSessionare removedParsedSession.turnsandParsedSession.subagentTurnsare removed (migrate toreadSessionTurns/readSubagentTurns)userMessageCount/assistantMessageCountdoc clarified — the lib applies stricter filters than CC's own metric (documented inparsed-session.ts)Released as v2.0.0.
Test plan
~/.claude/projects/directory