Skip to content

Commit aba2abd

Browse files
salmad3claude
andcommitted
feat: init nous
Adaptive documentation infrastructure that infers semantic metadata from prose at build time and emits structured outputs across multiple agent-readability protocols. Parsers for Markdown, KD, KDX, MDX, AsciiDoc, and reStructuredText converge on a shared document model. Built-in documentation agents refine annotations through consumption-driven feedback. Co-Authored-By: Claude <noreply@anthropic.com>
0 parents  commit aba2abd

File tree

214 files changed

+38926
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

214 files changed

+38926
-0
lines changed

.github/workflows/ci.yml

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
branches: [main]
8+
9+
concurrency:
10+
group: ${{ github.workflow }}-${{ github.ref }}
11+
cancel-in-progress: true
12+
13+
jobs:
14+
build-and-test:
15+
runs-on: ubuntu-latest
16+
17+
strategy:
18+
matrix:
19+
node-version: [20, 22]
20+
21+
steps:
22+
- uses: actions/checkout@v4
23+
24+
- uses: pnpm/action-setup@v4
25+
26+
- uses: actions/setup-node@v4
27+
with:
28+
node-version: ${{ matrix.node-version }}
29+
cache: pnpm
30+
31+
- run: pnpm install --frozen-lockfile
32+
33+
- run: pnpm build
34+
35+
- run: pnpm typecheck
36+
37+
# Build docs to produce dist/ artifacts required by integration tests
38+
- run: node packages/cli/dist/bin.js build
39+
40+
- run: pnpm test

.gitignore

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
node_modules/
2+
dist/
3+
.turbo/
4+
*.tsbuildinfo
5+
.DS_Store
6+
coverage/
7+
.env
8+
.env.local
9+
.claude/
10+
packages/mcp-server/src/*.js
11+
packages/mcp-server/src/*.js.map
12+
packages/mcp-server/src/*.d.ts
13+
packages/mcp-server/src/*.d.ts.map

CHANGELOG.md

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# Changelog
2+
3+
All notable changes to Nous are documented here. This project adheres to [Semantic Versioning](https://semver.org/).
4+
5+
> **Alpha Notice**: Nous is in active development. All `0.x` releases may contain breaking changes between minor versions. Pin exact versions in production.
6+
7+
## [0.2.0] — 2026-03-24
8+
9+
Initial public release.
10+
11+
### Pipeline
12+
13+
- Four-phase pipeline: Parse → Infer → Transform → Render/Emit
14+
- Inference engine attaches `type`, `audience`, and `answers` annotations to blocks automatically
15+
- Inferred annotations carry `_inferred: true` and `_tier` fields (`tier1`: author-declared, `tier2`: pipeline-inferred, `tier3`: auto-generated)
16+
- Engine mode (`nous engine`) emits structured outputs without HTML rendering
17+
- Docs mode (`nous build`) produces HTML with the Nous design system
18+
19+
### Parsers
20+
21+
- Markdown/GFM (`@nousdev/parser-md`)
22+
- KD: CommonMark + block annotations (`@nousdev/parser-kd`)
23+
- KDX: KD + JSX components (`@nousdev/parser-kdx`)
24+
- MDX (`@nousdev/parser-mdx`)
25+
- AsciiDoc (`@nousdev/parser-adoc`)
26+
- reStructuredText (`@nousdev/parser-rst`)
27+
28+
### Emission Protocols
29+
30+
- Schema.org JSON-LD (TechArticle, FAQPage, BreadcrumbList)
31+
- Google A2A Agent Cards
32+
- agents.json (Wildcard AI)
33+
- OpenAPI 3.1
34+
- MCP Resources
35+
- AGENTS.md
36+
- RAG Chunks (semantic, with metadata envelope)
37+
- Cross-reference graphs
38+
- Code examples index
39+
- llms.txt and llms-full.txt
40+
41+
### Adaptive Pipeline
42+
43+
- Coverage agent (`nous agents --coverage`): LLM-powered annotation suggestions
44+
- Query analysis (`nous analyze`): consumption-driven adaptation from MCP server logs
45+
- Protocol-agnostic observation types for future transport adapters
46+
47+
### Security
48+
49+
- SafeHtml branded types with DOMPurify sanitization
50+
- Input validation at system boundaries
51+
- Rate limiting on MCP server
52+
- Symlink traversal prevention via `followSymbolicLinks: false` on file discovery
53+
54+
### Design System
55+
56+
- Four theme presets: Radiance, Obsidian, Terminal, Meridian
57+
- Glassmorphic surfaces with depth tokens and atmosphere controls
58+
- Cards, badges, steps, admonitions, tab groups, feature grids
59+
60+
## Packages
61+
62+
| Package | Description |
63+
|---------|-------------|
64+
| `@nousdev/types` | NDM types, configuration, SafeHtml |
65+
| `@nousdev/core` | Pipeline orchestration: parse, infer, transform, render |
66+
| `@nousdev/parser-md` | Markdown/GFM → NDM |
67+
| `@nousdev/parser-kd` | KD → NDM |
68+
| `@nousdev/parser-kdx` | KDX (KD + JSX) → NDM |
69+
| `@nousdev/parser-mdx` | MDX → NDM |
70+
| `@nousdev/parser-adoc` | AsciiDoc → NDM |
71+
| `@nousdev/parser-rst` | reStructuredText → NDM |
72+
| `@nousdev/search` | Full-text search index generation and client-side query engine |
73+
| `@nousdev/plugin-shiki` | Syntax highlighting via Shiki |
74+
| `@nousdev/themes` | Design system: depth tokens, atmosphere, typography |
75+
| `@nousdev/agent-metadata` | Protocol emitters: Schema.org, A2A, agents.json, OpenAPI, RAG, MCP |
76+
| `@nousdev/renderer-html` | NDM → HTML with design system integration |
77+
| `@nousdev/mcp-server` | MCP server: search, filter, query logging |
78+
| `@nousdev/agents` | Documentation agents: coverage analysis, annotation suggestions |
79+
| `@nousdev/cli` | CLI: init, build, dev, serve, engine, serve-mcp, analyze, agents |

CONTRIBUTING.md

Lines changed: 165 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,165 @@
1+
# Contributing to Nous
2+
3+
Thanks for considering a contribution. Whether you're fixing a bug, adding a parser, writing an emitter, or improving documentation, this guide will get you oriented quickly.
4+
5+
Nous is a monorepo managed with [pnpm workspaces](https://pnpm.io/workspaces) and [Turborepo](https://turbo.build/repo). Every package under `packages/` is independently buildable and publishable. Browse the [README](README.md) for a full package index and architecture overview.
6+
7+
## Prerequisites
8+
9+
- **Node.js** >= 20.0.0
10+
- **pnpm** >= 10.18.0 (corepack-managed)
11+
12+
```bash
13+
corepack enable
14+
corepack prepare pnpm@10.18.3 --activate
15+
```
16+
17+
## Setup
18+
19+
```bash
20+
git clone https://github.com/salmad3/nousdev.git
21+
cd nousdev
22+
pnpm install
23+
pnpm build
24+
```
25+
26+
## Development Workflow
27+
28+
### Building
29+
30+
Turborepo handles dependency ordering across packages:
31+
32+
```bash
33+
pnpm build # Build all packages
34+
pnpm dev # Start dev server with watch mode
35+
pnpm typecheck # Type-check all packages
36+
pnpm lint # Lint all packages
37+
pnpm test # Run all test suites
38+
```
39+
40+
To build a single package:
41+
42+
```bash
43+
pnpm --filter @nousdev/agent-metadata build
44+
```
45+
46+
### Pre-Build Verification
47+
48+
The `@nousdev/agent-metadata` package runs protocol version verification before every build via the `prebuild` npm lifecycle hook. This check scans emitter code, documentation, and type comments for version strings that have drifted from the centralized manifest at `packages/agent-metadata/src/protocols.ts`.
49+
50+
If you update a protocol version, edit `protocols.ts` first, then run:
51+
52+
```bash
53+
pnpm --filter @nousdev/agent-metadata verify-protocols
54+
```
55+
56+
The script identifies every stale reference with its file path and line number.
57+
58+
### Running the Documentation Site
59+
60+
```bash
61+
pnpm nous dev
62+
```
63+
64+
Opens at `http://localhost:4321`. The site is built entirely with Nous, with every page authored in KD. Changes to parsers, emitters, or the renderer are immediately visible here.
65+
66+
## Architecture
67+
68+
Nous processes documentation through a four-phase pipeline:
69+
70+
1. **Parsing**: Format-specific parsers (`parser-kd`, `parser-md`, etc.) convert source files into the Nous Document Model (NDM), an immutable semantic AST.
71+
72+
2. **Inference**: The inference engine analyzes parsed NDM nodes and attaches block-level annotations (`type`, `audience`, `answers`) to blocks that lack explicit metadata.
73+
74+
3. **Transformation**: Plugins compose pure functions that receive document nodes and return new nodes without mutation. The `agent-metadata` plugin collects documents during this phase.
75+
76+
4. **Emission**: The HTML renderer produces human-readable output. The agent-metadata plugin emits machine-readable formats (Schema.org JSON-LD, A2A agent cards, MCP resources, RAG chunks, llms.txt, AGENTS.md, and others).
77+
78+
The NDM is the contract between parsers and consumers. Any format that can produce valid NDM nodes integrates with the full pipeline automatically.
79+
80+
## Writing Parsers and Adapters
81+
82+
Adding support for a new source format means writing an NDM adapter. KD and KDX have custom parsers because their block annotation model has no upstream equivalent. For other formats, adapters wrap an established parser and convert its AST into NDM nodes.
83+
84+
Each parser package exports a single `parse` function:
85+
86+
```typescript
87+
import type { DocumentNode } from '@nousdev/types';
88+
89+
export function parse(source: string, filePath: string): DocumentNode;
90+
```
91+
92+
The returned `DocumentNode` must conform to NDM types. Adapters should delegate syntax parsing to the upstream library and focus their logic on the AST-to-NDM conversion: mapping frontmatter, headings, code blocks, and (where available) semantic annotations into their corresponding NDM node types.
93+
94+
## Writing Emitters
95+
96+
Agent metadata emitters follow a consistent signature:
97+
98+
```typescript
99+
import type { ExtractedMetadata, AgentMetadataConfig, EmitterOutput } from '@nousdev/agent-metadata';
100+
101+
export function emitMyFormat(
102+
metadata: ExtractedMetadata,
103+
config: AgentMetadataConfig,
104+
): MyFormatOutput;
105+
```
106+
107+
Register new emitters in `packages/agent-metadata/src/emitters/index.ts` by adding them to the `EMITTER_REGISTRY` and `ALL_EMITTERS` array.
108+
109+
## Commit Conventions
110+
111+
Follow [Conventional Commits](https://www.conventionalcommits.org/):
112+
113+
```
114+
<type>: <description>
115+
116+
<optional body>
117+
```
118+
119+
| Type | Usage |
120+
|------|-------|
121+
| `feat` | New feature or capability |
122+
| `fix` | Bug fix |
123+
| `refactor` | Code restructuring without behavior change |
124+
| `docs` | Documentation changes |
125+
| `test` | Adding or modifying tests |
126+
| `chore` | Build tooling, dependency updates, housekeeping |
127+
| `perf` | Performance improvement |
128+
| `ci` | CI/CD pipeline changes |
129+
130+
Keep the subject line under 72 characters. Use the body for context on *why* the change was made when the diff alone does not make the rationale clear.
131+
132+
## Pull Request Process
133+
134+
1. Create a feature branch from `main`.
135+
2. Make changes, ensuring `pnpm build` and `pnpm test` pass.
136+
3. Write a PR description that includes:
137+
- A summary of what changed and why.
138+
- A test plan describing how to verify the changes.
139+
4. Request review. Address feedback as new commits rather than force-pushing, so reviewers can track incremental changes.
140+
141+
## Code Standards
142+
143+
These conventions keep the codebase consistent as it grows. They are enforced in review.
144+
145+
- **Immutability**: Create new objects rather than mutating existing ones. Transforms return new document nodes; they never modify the input.
146+
- **Small files**: Aim for 200-400 lines per module. Extract utilities when files exceed 800 lines.
147+
- **Explicit error handling**: Handle errors at every level. Provide actionable messages.
148+
- **Input validation**: Validate at system boundaries (user config, file content, external protocol responses).
149+
- **No hardcoded protocol versions**: All version strings in `@nousdev/agent-metadata` must reference `protocols.ts` constants. The pre-build verification script enforces this.
150+
151+
## Adding a New Package
152+
153+
1. Create the directory under `packages/`.
154+
2. Add a `package.json` with the `@nousdev/` scope and appropriate `main`/`types` fields.
155+
3. Add a `tsconfig.json` extending the root config.
156+
4. Register the package in `pnpm-workspace.yaml` if not already covered by the glob pattern.
157+
5. Add build/test scripts and wire them into `turbo.json` if the package has build dependencies on other Nous packages.
158+
159+
## Questions?
160+
161+
Open an issue or start a discussion. We are happy to help you find the right entry point for your contribution.
162+
163+
## License
164+
165+
By contributing to Nous, you agree that your contributions will be licensed under the [MIT License](LICENSE).

LICENSE

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2026 salmad3
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

0 commit comments

Comments
 (0)