diff --git a/README.md b/README.md index f57aaaf..210fea0 100644 --- a/README.md +++ b/README.md @@ -24,10 +24,28 @@ phase and [docs/architecture/](docs/architecture/) for the design. Repository: `github.com/HummingByteDev/vpsa-network-discovery` (private until the [launch checklist](docs/operations/launch-checklist.md) passes). +## Documentation + +**[docs/README.md](docs/README.md) is the documentation home** — a full site that +teaches VAPN from first principles (no networking background assumed) and guides +every audience from beginner to advanced operator: +[Getting Started](docs/getting-started/README.md) · +[Core Concepts](docs/concepts/README.md) · +[Walkthroughs](docs/walkthroughs/end-to-end.md) · +[Architecture](docs/architecture/README.md) · +[Builder](docs/builder/README.md) · +[Workers](docs/worker/README.md) · +[VPS Advisor Integration](docs/integration/README.md) · +[API Reference](docs/api/README.md) · +[Operations](docs/operations/README.md) · +[Development](docs/development/README.md) · +[Reference](docs/reference/README.md) · +[Project Handbook](docs/handbook/README.md). + ## Run a worker (community) ```sh -curl -fsSL https://install.vpsadvisor.com | bash +curl -fsSL https://raw.githubusercontent.com/HummingByteDev/vpsa-network-discovery/main/deploy/worker/install.sh | bash ``` Then `vapn status` · `vapn pause` · `vapn update` · `vapn uninstall` — @@ -48,10 +66,11 @@ rollback, kill switch, audit). ## Website team -[The integration guide](docs/integration/vpsadvisor-integration-guide.md) -is the complete contract: models, endpoints, dashboards, permissions, jobs, -rollout order. The `mockadvisor` stub in this repo implements it, so the -guide is executable. +[The Django integration guide](docs/integration/django-integration.md) +is the complete contract: models, migrations, endpoints, dashboards, +permissions, jobs, rollout order. The `mockadvisor` stub in this repo +implements it, so the guide is executable. Start at the +[integration overview](docs/integration/README.md). ## Layout diff --git a/deploy/worker/install.sh b/deploy/worker/install.sh index e0026dc..26cd93a 100755 --- a/deploy/worker/install.sh +++ b/deploy/worker/install.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash -# VAPN worker bootstrap — the `curl -fsSL https://install.vpsadvisor.com | bash` +# VAPN worker bootstrap — the canonical +# curl -fsSL https://raw.githubusercontent.com/HummingByteDev/vpsa-network-discovery/main/deploy/worker/install.sh | bash # entrypoint. Installs the `vapn` CLI for this machine's architecture, then # hands over to `vapn install` (interactive: coordinator URL, enrollment # token, system checks, start, verify). diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..848641b --- /dev/null +++ b/docs/README.md @@ -0,0 +1,156 @@ +# VAPN Documentation + +**VPS Advisor Probe Network** — the distributed network-observability backend +that measures the public network health of VPS providers and feeds +trust-weighted results back to the [VPS Advisor](https://vpsadvisor.example) +website. + +> **New here?** Read [What is VAPN?](#what-is-vapn) below, then jump to the +> path that matches you in [Where should I start?](#where-should-i-start). + +--- + +## What is VAPN? + +VAPN answers one question at scale: **"Is this hosting provider's network +healthy right now — globally, and in my part of the world?"** + +It does that without ever logging into anyone's server. Instead, a fleet of +small, community-run programs (**workers**) send lightweight network probes to +the *public* addresses that each provider announces to the Internet, from many +vantage points around the world. No single worker's opinion is trusted on its +own. The platform combines many independent measurements into a **consensus** +verdict, weights each worker by how reliable it has proven to be (**trust**), +and publishes only the agreed-upon result. + +```mermaid +flowchart LR + A[VPS Advisor website
which providers exist] -->|catalog + ASNs| B[VAPN platform] + B -->|assignments| C[Community workers
probe from many locations] + C -->|signed measurements| B + B -->|"healthy / degraded / outage
(consensus)"| A +``` + +**This repository is not the VPS Advisor website.** VPS Advisor is an +independent, already-live provider-review platform. VAPN is the *measurement +machine* behind one of its features (Provider Network Health). VPS Advisor +stays the source of truth for which providers exist; VAPN never crawls the +Internet looking for them. See [Project background](#project-background). + +### Why does it exist? + +Traditional uptime monitors watch a *specific server* you own and control. +That tells you nothing about a provider you're *considering*. VAPN measures the +thing a prospective customer actually cares about: **the provider's own public +network** — is it reachable, is it fast, is it stable — measured +independently, from the outside, by a community rather than by the provider +itself. See [Core Concepts → Why community measurement](concepts/measurement-and-consensus.md). + +### What problems does it solve? + +| Problem | How VAPN addresses it | +|---|---| +| "Uptime" pages are self-reported and easy to game | Measurements come from independent community workers, not the provider | +| A single monitor is one biased vantage point | Many workers across regions and networks; results are a **consensus** | +| Any one worker could lie or be compromised | Every measurement is cryptographically signed; workers earn **trust**; liars are outvoted and quarantined | +| Global "up" hides regional problems | Verdicts are computed per region, not just globally | +| Monitoring must not become abuse | Workers only probe addresses derived from providers' own announced routes, rate-limited by policy | + +--- + +## Who is this documentation for? + +VAPN touches several very different audiences. Each has a reading path below. + +| You are… | You want to… | Start here | +|---|---|---| +| **A community member** with a spare Linux box | Run a worker and contribute measurements | [Getting Started → Quick Start](getting-started/quickstart.md) | +| **An operator / DevOps engineer** | Deploy and run the platform itself | [Setup → Deploy the platform](operations/deployment.md) | +| **A VPS Advisor / Django developer** | Integrate the website with VAPN | [VPS Advisor Integration](integration/README.md) | +| **A contributor / developer** | Understand the code and submit changes | [Development](development/README.md) | +| **Someone who wants to understand it deeply** | Learn the whole system from first principles | [The Project Handbook](handbook/README.md) | + +No networking background is assumed anywhere. Every networking term — +**IP address, CIDR, ASN, BGP, RIPE, RIS, MRT, GeoIP** — is explained from +scratch in [Core Concepts](concepts/README.md) as if you are an experienced +software developer who has simply never worked with Internet routing. + +--- + +## Where should I start? + +### 🧭 Contributors (run a worker) +1. [Quick Start](getting-started/quickstart.md) — a worker running in minutes +2. [What a worker does & what it costs you](worker/resources-and-privacy.md) +3. [Worker command reference](worker/command-reference.md) +4. [Worker FAQ & troubleshooting](getting-started/faq.md) + +### 🛠️ Operators (run the platform) +1. [Architecture overview](architecture/01-system-architecture.md) — the big picture +2. [Deployment guide](operations/deployment.md) — a single VM with Docker Compose +3. [The RIPE Routing Builder](builder/README.md) — how the routing data is made +4. [Operations](operations/README.md) — monitoring, backups, upgrades, incident response + +### 🌐 VPS Advisor / Django developers +1. [Integration overview](integration/README.md) +2. [Django integration guide](integration/django-integration.md) — models, endpoints, tasks, admin +3. [API Reference → VPS Advisor endpoints](api/README.md#a-vps-advisor-endpoints) + +### 💻 Developers & the curious +1. [Core Concepts](concepts/README.md) — the networking foundations +2. [End-to-end walkthrough](walkthroughs/end-to-end.md) — follow one measurement from provider to public verdict +3. [Development guide](development/README.md) — build, test, and contribute +4. [The Project Handbook](handbook/README.md) — the whole story, cover to cover + +--- + +## Documentation map + +| Section | What's inside | +|---|---| +| [Getting Started](getting-started/README.md) | Quick start, install, update, uninstall, FAQ, troubleshooting | +| [Core Concepts](concepts/README.md) | The Internet, IP/CIDR, ASN, BGP, RIPE/RIS/MRT, GeoIP, consensus & trust | +| [Architecture](architecture/README.md) | System design, domain model, database, security & trust, lifecycles, deployment, risk, roadmap | +| [Walkthroughs](walkthroughs/README.md) | End-to-end trace plus focused lifecycles (install, publish, auth, measurement, trust, updates) | +| [RIPE Routing Builder](builder/README.md) | How routing intelligence is built from RIPE RIS data, and how to run the builder | +| [Community Workers](worker/README.md) | Everything about running a worker: install, commands, lifecycle, cost, privacy | +| [VPS Advisor Integration](integration/README.md) | The complete contract for the Django website team | +| [API Reference](api/README.md) | Every endpoint: purpose, auth, request, response, errors, examples | +| [Operations](operations/README.md) | Deploy, monitor, back up, upgrade, recover, respond to incidents | +| [Development](development/README.md) | Repo layout, standards, testing, CI, releases, contribution guide | +| [Reference](reference/README.md) | CLI, configuration, environment variables, database schema, glossary | +| [Project Handbook](handbook/README.md) | The entire system as one book, from networking fundamentals up | + +--- + +## Project background + +VPS Advisor is an independent cloud/VPS provider review platform: providers +claim listings, publish products, and receive reviews. One planned feature is +**Provider Network Health** — and VAPN is the backend that powers it. + +Two systems, one clean boundary: + +- **VPS Advisor (the website)** owns identity, the provider catalog, ASN + ownership, worker enrollment, the admin dashboard, and presentation. It + already exists and is in production. +- **VAPN (this repository)** owns routing intelligence, probe scheduling, + measurement execution, consensus, trust, and aggregation. It publishes + results *back* to VPS Advisor. + +The full brief lives in [`CLAUDE.md`](../CLAUDE.md) at the repository root. +The design that flowed from it is in [Architecture](architecture/README.md). + +## Status + +All eleven build phases are complete. Per-phase runnable walkthroughs are in +[`docs/demos/`](demos/); the design record is in +[`docs/architecture/`](architecture/). The supported v1 deployment is a single +Ubuntu VM running Docker Compose behind Caddy. + +## A note on terminology + +Networking is full of acronyms. Whenever this documentation uses one for the +first time in a document, it links to its definition. If you ever hit a term +you don't recognize, the [Glossary](reference/glossary.md) defines every one in +plain language. diff --git a/docs/api/README.md b/docs/api/README.md new file mode 100644 index 0000000..5a9a392 --- /dev/null +++ b/docs/api/README.md @@ -0,0 +1,209 @@ +# API Reference + +Every endpoint in the system, grouped by the three API surfaces, each with +purpose, authentication, request, response, status codes, and examples. This is +the canonical schema reference; the design rationale for *why* there are three +surfaces is in [architecture 04](../architecture/04-api-contracts.md), and the +Django-side implementation guide is [here](../integration/django-integration.md). + +## The three surfaces + +```mermaid +flowchart TB + W[Community workers] -->|"B. Coordinator API
Ed25519-signed"| CO[Coordinator] + CO -->|"A. VPS Advisor API
service token"| VA[VPS Advisor website] + AG[Aggregator] -->|"A. Results push"| VA + BU[Builder] -->|"A. Provider pull"| VA + OPS[Operators / vapnctl] -->|"C. Admin API
admin token"| CO +``` + +| Surface | Who calls it | Who implements it | Auth | +|---|---|---|---| +| **[A. VPS Advisor endpoints](#a-vps-advisor-endpoints)** | the platform (builder/coordinator/aggregator) | the **website team** | service token (bearer/HMAC/mTLS) | +| **[B. Coordinator API](#b-coordinator-api)** | community workers | this platform | Ed25519 request signing | +| **[C. Platform admin API](#c-platform-admin-api)** | operators / `vapnctl` | this platform | admin token, network-restricted | + +## Conventions (all surfaces) + +- **Transport:** JSON over HTTPS; `Content-Type: application/json`. +- **Errors:** RFC 7807 `application/problem+json`: + ```json + { "type": "about:blank", "title": "invalid signature", + "status": 401, "detail": "signature verification failed" } + ``` +- **Pagination:** cursor-based — `?cursor=&limit=`; responses carry + `next_cursor` (follow it while non-null). +- **Idempotency:** mutating platform→website calls carry an `Idempotency-Key`; + worker uploads are idempotent by batch id. +- **Versioning:** base path `/api/v1/`. +- **Timestamps:** RFC 3339 / ISO 8601, UTC. + +--- + +## A. VPS Advisor endpoints + +Implemented by the website team; consumed by the platform (server-to-server) or +humans. Full Django guide: [integration](../integration/django-integration.md). +Auth: platform service credential (`Authorization: Bearer `), scoped to +`/api/v1/monitoring/*`; human pages use website session auth + new permissions. + +### A1. Provider catalog (platform pulls) + +| Method & path | Purpose | +|---|---| +| `GET /api/v1/monitoring/providers` | List monitorable providers. Filters `?enabled=true`, `?updated_since=`. | +| `GET /api/v1/monitoring/providers/{id}` | Single provider; `404` if unknown. | +| `GET /api/v1/monitoring/asns` | Flat ASN→provider map for cheap delta sync. | + +**Request:** `GET /api/v1/monitoring/providers?enabled=true` +**Response 200:** +```json +{ "providers": [ { "provider_id": "7f9c...", "name": "ExampleHost", + "asns": [64500, 64501], "monitoring_enabled": true, "priority": 10, + "updated_at": "2026-07-18T06:00:00Z" } ], "next_cursor": null } +``` +**Status:** `200` ok · `401` bad token. + +### A2. Enrollment (operator UI + platform pull) + +| Method & path | Auth | Purpose | +|---|---|---| +| `POST /api/v1/monitoring/workers` | operator session | Create worker; returns one-time token. | +| `POST /api/v1/monitoring/workers/{id}/regenerate-token` | operator session | Replace unused/expired token. | +| `GET /api/v1/monitoring/workers` | operator session | Operator's own workers + state. | +| `GET /api/v1/monitoring/enrollments/pending` | service token | Pending enrollments + token hashes. | +| `POST /api/v1/monitoring/enrollments/{id}/registered` | service token | Ack registration; `204`, idempotent. | + +**`GET …/enrollments/pending` → 200:** +```json +{ "enrollments": [ { "enrollment_id": "en-123", "worker_id": "9f30...", + "worker_name": "helsinki-1", "operator_id": "user-uuid", + "token_hash": "hex sha256", "expires_at": "2026-07-19T08:00:00Z" } ], + "next_cursor": null } +``` + +### A3. Admin decisions (admin UI + platform pull) + +| Method & path | Purpose | +|---|---| +| `GET /api/v1/monitoring/admin/decisions?since=` | Platform syncs approve/suspend/quarantine/retire decisions (oldest first). | + +**Response 200:** +```json +{ "decisions": [ { "decision_id": "d-1", "worker_id": "9f30...", + "state": "active", "reason": "verified operator", + "decided_at": "2026-07-18T09:00:00Z" } ], "next_cursor": null } +``` + +### A4. Results ingestion (platform pushes — idempotent) + +| Method & path | Purpose | +|---|---| +| `PUT /api/v1/monitoring/results/providers/{id}` | Upsert current status document. | +| `POST /api/v1/monitoring/results/anomalies` | Open/update/resolve anomaly events. | +| `POST /api/v1/monitoring/results/history` | Periodic rollup batches for charts. | +| `POST /api/v1/monitoring/telemetry/fleet` | Fleet summary for the admin dashboard. | + +**`PUT …/results/providers/{id}` body:** +```json +{ "as_of": "2026-07-18T08:05:00Z", + "global": { "verdict": "healthy", "confidence": 0.97, + "metrics": { "rtt_p50_ms": 21.4, "rtt_p95_ms": 38.2, "loss_rate": 0.001, + "worker_count": 14, "dissent_ratio": 0.02 } }, + "regions": [ { "region": "eu-west", "verdict": "healthy", "confidence": 0.99 }, + { "region": "ap-south", "verdict": "insufficient_data", "confidence": 0 } ] } +``` +**Verdicts:** `healthy | degraded | outage | insufficient_data`. +**Status:** `2xx` (fast ack; `202` fine). Non-2xx → platform retries with +backoff. **Render `insufficient_data` as "not enough data," never an outage.** + +--- + +## B. Coordinator API + +The high-volume, machine-facing surface workers talk to. **Auth:** Ed25519 +request signing on every call except `register` — headers `X-Worker-Id`, +`X-Timestamp`, `X-Nonce`, `X-Signature` over +`method | path | timestamp | nonce | sha256(body)`. Full mechanics: +[worker authentication](../walkthroughs/worker-authentication.md). + +| Method & path | Purpose | +|---|---| +| `POST /api/v1/workers/register` | One-time: enrollment token + public key + facts → `worker_id`, state `pending`. | +| `POST /api/v1/workers/heartbeat` | Liveness + version + stats → config, lease renewals, snapshot version, control actions. | +| `GET /api/v1/artifacts/routing/current` | Current snapshot manifest (version, url, sha256, signature, min_worker_version). | +| `POST /api/v1/assignments/lease` | Request work → assignment batch with lease expiry. | +| `POST /api/v1/assignments/release` | Voluntarily return assignments (shutdown/drain). | +| `POST /api/v1/observations` | Upload signed observation batch (idempotent by batch id). | +| `POST /api/v1/workers/keys/rotate` | Submit next public key signed by current; returns overlap window. | +| `GET /api/v1/workers/me` | Own state/trust snapshot (operator transparency). | + +**`POST /api/v1/workers/register` body:** +```json +{ "enrollment_token": "…", "public_key": "base64 ed25519", + "software_version": "1.2.0", "reported_country": "FI" } +``` +**→ 201:** `{ "worker_id": "9f30...", "state": "pending" }` + +**`POST /api/v1/workers/heartbeat` → 200 (abridged):** +```json +{ "state": "active", + "snapshot": { "version": "20260718T0800Z-1" }, + "leases": [ { "assignment_id": 812, "expires_at": "..." } ], + "control": [ ], + "config": { "heartbeat_seconds": 30, "upload_seconds": 60 } } +``` +Control actions include `rotate_key`, `drain`, `suspend`, `upgrade_required`. + +**`POST /api/v1/observations` body (abridged):** +```json +{ "batch_id": "0197a3...", "observations": [ { + "assignment_id": 812, "target": "203.0.113.7", "probe_type": "icmp", + "measured_at": "2026-07-18T08:04:31.201Z", "ok": true, + "rtt_ms": 22.9, "packets_sent": 4, "packets_lost": 0, + "signature": "base64…" } ] } +``` +**→ 200** with per-item accept/reject (207-style). + +**Status codes:** `401` invalid signature / expired timestamp · `403` wrong +state (e.g. suspended) · `409` replayed nonce / duplicate batch · `422` +malformed observation · `429` policy rate-limit. + +--- + +## C. Platform admin API + +Operator/automation surface on the coordinator, authenticated by an **admin +token** (`Authorization: Bearer `) and network-restricted +(bind to an internal interface / allowlist CIDR via `VAPN_ADMIN_ALLOW_CIDR`). +This is what [`vapnctl`](../reference/cli.md#vapnctl-platform-administration) drives — the operational +escape hatch mirroring what the VPS Advisor admin dashboard does via the A3 +sync. + +| Area | Capability | +|---|---| +| Fleet | status overview; worker list; worker detail (state, trust, leases, events) | +| Worker lifecycle | create (returns one-time token); approve / suspend / quarantine / retire (with reason); force key rotation | +| Snapshots | list versions; rollback to a previous published version | +| Scheduler | pause / resume (global assignment kill switch) | +| Audit | query the append-only audit log (`?category=&since=&limit=`) | + +These map 1:1 to `vapnctl` subcommands — see the +[CLI reference](../reference/cli.md#vapnctl-platform-administration). Because it's the direct control +plane, keep it off the public Internet; the website's admin dashboard (surface +A) is the primary human UI and this is the automation/escape-hatch layer. + +## Error catalog (quick reference) + +| Status | Meaning | Surfaces | +|---|---|---| +| `400` | Malformed request | all | +| `401` | Missing/invalid credential or signature | all | +| `403` | Authenticated but not allowed (wrong worker state, missing permission) | B, C, A(human) | +| `404` | Unknown resource | A, C | +| `409` | Conflict — replayed nonce, duplicate batch, idempotency clash | B, A(push) | +| `422` | Semantically invalid (bad observation) | B | +| `429` | Rate-limited by policy | B | +| `5xx` | Server error — callers retry with backoff | all | + +All error bodies are RFC 7807 problem+json. diff --git a/docs/architecture/README.md b/docs/architecture/README.md index cc6919d..75d8afb 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -1,8 +1,16 @@ # Architecture — VPS Advisor Probe Network (VAPN) -This directory contains the Phase 1 architectural analysis required by the project brief -(`CLAUDE.md`). **No implementation exists yet**; per the implementation strategy, coding -begins only after this architecture is approved. +> Part of the [VAPN documentation](../README.md). For a gentler, first-principles +> path into the same material, read [Core Concepts](../concepts/README.md) and the +> [end-to-end walkthrough](../walkthroughs/end-to-end.md) first; this section is +> the authoritative design record. + +This directory contains the architectural analysis required by the project brief +(`CLAUDE.md`). It was written in Phase 1 as the design to be approved before coding; +**all eleven phases are now implemented** (see [`docs/demos/`](../demos/) for runnable +per-phase walkthroughs). The documents remain the canonical description of the design — +where an implementation detail has since been fixed, the [Reference](../reference/README.md) +and [API](../api/README.md) sections carry the current specifics. ## Documents diff --git a/docs/builder/README.md b/docs/builder/README.md new file mode 100644 index 0000000..25c1a23 --- /dev/null +++ b/docs/builder/README.md @@ -0,0 +1,160 @@ +# The RIPE Routing Builder + +The **builder** is the component that turns raw global routing data into a +trustworthy, signed list of addresses for workers to probe. It is the only part +of VAPN that ever reads RIPE data or MaxMind databases. This guide explains it +from first principles — what the inputs are, what it does with them, and why — +then the [installation guide](installation.md) covers running one. + +> **New to routing?** Read [Core Concepts](../concepts/README.md) first — +> especially [RIPE/RIS/MRT](../concepts/ripe-and-ris.md) and +> [prefix ownership](../concepts/prefix-ownership.md). This guide assumes those. + +## What problem the builder solves + +Workers need to know *exactly which addresses belong to each monitored +provider*, and they need to trust that list absolutely — a wrong entry means +probing someone else's network. But computing that list is expensive and +security-sensitive: it means downloading a ~1M-route global routing dump, +parsing a binary format, filtering, deduplicating, validating, and +geolocating. + +The builder does all of that **once, centrally, and auditably**, then publishes +a tiny **signed** artifact. Thousands of workers consume the finished artifact; +none of them ever touches the raw data. This single boundary is the reason +workers can be a few MB of RAM and the routing logic can be improved without +redeploying the fleet. + +```mermaid +flowchart LR + subgraph Inputs + A["Monitored ASNs
(VPS Advisor)"] + B["RIPE RIS bview
(MRT dump)"] + G["MaxMind GeoLite2
(deployer's key)"] + end + A & B & G --> BUILD[Builder] + BUILD --> PG[("PostgreSQL
routing.* (canonical)")] + BUILD --> ART["Signed SQLite artifact
+ manifest"] + ART --> STORE[(Artifact store)] + STORE --> WK[Workers] +``` + +## The inputs, explained + +### 1. The monitored ASN list (from VPS Advisor) + +The builder asks VPS Advisor which providers/ASNs to monitor. This is the +*membership* input — it decides which slice of the global table to keep. VAPN +never scans the Internet; it only ever extracts prefixes for these ASNs. + +### 2. The RIPE RIS `bview` dump (MRT) + +This is the routing input: a full snapshot of the global routing table. + +- **What RIPE is:** a [Regional Internet Registry](../concepts/ripe-and-ris.md#ripe-ncc-one-of-the-internets-registries) + that, among other things, runs a free public recording of global routing. +- **What RIS is:** the [Routing Information Service](../concepts/ripe-and-ris.md#ris-the-routing-information-service) — + collectors that passively record BGP announcements from many real networks. +- **What a `bview`/RIB dump is:** a periodic (every 8 h) full-table snapshot — + "here is every route the collector currently sees." +- **What MRT is:** the binary file format (RFC 6396) the dump is stored in. + Compact, standard, complete — and the reason a dedicated parser + (`internal/routing/mrtreader`) exists. +- **Why MRT and not a live BGP feed:** VAPN wants *membership* ("which prefixes + do these ASNs announce right now"), which a full-table snapshot answers + directly, plus *churn* (from diffing snapshots) — neither needs the raw + second-by-second update stream, and a snapshot is far simpler and safer to + process. + +### 3. MaxMind GeoLite2 (the deployer's own key) + +The [GeoIP](../concepts/geoip.md) input — used to attach country/city/coords to +each prefix so verdicts can be regional. Each deployer supplies their own +MaxMind license key; the databases are never redistributed by the project, and +GeoIP refresh runs on an independent cadence. + +## What the builder does, in order + +This is [the snapshot-publishing walkthrough](../walkthroughs/snapshot-publishing.md) +in condensed reference form. Each numbered step maps to real code in +`internal/builder` and `internal/artifact`. + +1. **Sync monitored ASNs** from VPS Advisor. +2. **Fetch the RIS `bview`** (or use the pre-downloaded `data/ripe/` copy in + dev); reject it if older than `VAPN_RIS_BVIEW_MAX_AGE`. +3. **Parse + origin-filter**: stream the MRT, keep only prefixes originated by a + monitored ASN. The big reduction (≈1M → a few thousand). +4. **Deduplicate**: collapse many per-peer reports of a `(prefix, origin AS)` + into one fact. → [why duplicates exist](../concepts/prefix-ownership.md#complication-1-duplicate-prefixes) +5. **Validate**: + - **Bogon filter** (`internal/routing/bogon`) — drop private/reserved/absurd + prefixes that must never be probed. + - **MOAS flagging** — mark ambiguous multi-origin prefixes for review rather + than guessing ownership. +6. **Enrich with GeoIP** (`internal/routing/geo`) — country, city, coordinates. +7. **Load into `routing.*`** under a new `routing.snapshot` (status `building`). + Prefixes are per-snapshot and immutable. +8. **Derive probe targets** — representative addresses per prefix/region, each + with a recorded rationale, capped at `MAX_TARGETS_PER_PROVIDER`. +9. **Sanity gate** — if the prefix count swung more than + `SANITY_MAX_DELTA_PCT` vs the previous snapshot, hold for admin approval + (route-leak protection). `SANITY_FORCE=true` overrides. +10. **Export + sign** — write the worker-facing subset (`prefixes`, `targets`, + `meta`) to a compact **SQLite** artifact and produce a **manifest** with + counts, sha256, Ed25519 signature, and `min_worker_version`. +11. **Publish atomically** — upload to the artifact store, verify readback, mark + `published`, previous → `superseded`. Any failure → `failed`, previous stays + in force. + +## Snapshot versioning, validation, and rollback + +- **Versioning:** each snapshot has a monotonic version like + `20260718T0800Z-1`. Workers refuse to downgrade to an older version. +- **Validation, two layers:** the *build-time* sanity gate (prefix-count swing) + guards against poisoned input; the *consume-time* signature + sha256 check on + every worker guards against a tampered or corrupt artifact. +- **Rollback:** if a bad snapshot slips through, an admin runs + `vapnctl snapshots rollback ` to re-point `current` at a known-good + snapshot — an audited action. Because publication is atomic, rollback is + instantaneous from the workers' view. + +## How workers consume the artifact + +The published SQLite file contains only the worker-facing subset: + +| Table | Columns (essentials) | Worker uses it to… | +|---|---|---| +| `prefixes` | `prefix, origin_asn, provider_id` | validate that a target is legitimate | +| `targets` | `address, provider_id, assignment hints` | know the probeable addresses | +| `meta` | `version, built_at, min_worker_version` | check compatibility + freshness | + +Workers use it for **local validation and enrichment** — never to *choose* +targets (assignments come from the coordinator's scheduler). Download and +verification are covered in the +[worker authentication walkthrough](../walkthroughs/worker-authentication.md). + +## Failure, recovery, and scalability + +| Concern | Behavior | +|---|---| +| **RIS download fails / stale** | Build aborts; previous published snapshot stays fully in force | +| **Any build step fails** | Snapshot marked `failed`; atomic publish means workers never see a partial build | +| **Poisoned routing data (leak/hijack)** | Sanity gate holds anomalous builds for a human; bogon/MOAS validation drops/flags bad prefixes | +| **Bad snapshot published** | `vapnctl snapshots rollback` re-points to a good version (audited) | +| **Artifact store compromised** | Store is untrusted; signature verification on every worker catches tampering | +| **Fleet growth** | Artifact distribution is CDN-offloadable from day one; the builder's cost is independent of worker count | + +The builder is a **batch job, not a daemon** — it runs, publishes, and exits, so +it has no inbound network surface and scales trivially (run it on a schedule). + +## Future extensions + +- Additional RIS collectors / cross-collector agreement for stronger ownership + signals. +- RPKI validation (is this origin cryptographically authorized for this prefix?) + layered onto the existing MOAS/bogon validation. +- Richer target selection (per-region representative sampling) behind the same + `targets` table shape — no worker or schema change needed. + +To run one, continue to [Builder Installation](installation.md). For the design +record, see [architecture 01 §4.1](../architecture/01-system-architecture.md#41-snapshot-builder-builder). diff --git a/docs/builder/installation.md b/docs/builder/installation.md new file mode 100644 index 0000000..c34bd53 --- /dev/null +++ b/docs/builder/installation.md @@ -0,0 +1,162 @@ +# Builder Installation (Simplified) + +A step-by-step guide to running the routing builder, written for someone +comfortable following commands but **not** assumed to be a Linux or networking +expert. If a term is unfamiliar, the [builder overview](README.md) and +[Glossary](../reference/glossary.md) explain it. + +> **Who needs this?** Only **platform operators** — the people running VAPN +> itself. Community worker operators never run the builder; +> [running a worker](../getting-started/quickstart.md) is a completely separate, +> simpler task. + +In production the builder runs as part of the platform's Docker Compose stack on +a schedule — you usually don't invoke it by hand. This guide covers both the +managed path (recommended) and a manual run for understanding and debugging. + +## What you'll need + +| Requirement | Why | How to get it | +|---|---|---| +| The VAPN platform stack running | The builder writes to the platform's PostgreSQL and artifact store | [Deployment guide](../operations/deployment.md) | +| A **MaxMind license key** | To download GeoLite2 for geolocation | Free account at maxmind.com → license key | +| A **snapshot signing key** | To sign artifacts so workers can trust them | Generate with `keygen` (below) | +| Access to **RIPE RIS** (outbound HTTPS) | To download the routing dump | Just outbound internet; no account needed | +| The **VPS Advisor service token** | To fetch the monitored ASN list | From the website team | + +## Step 1 — Generate a snapshot signing key + +Workers verify every snapshot against a **pinned public key**. You generate an +Ed25519 keypair once; the builder holds the private half, and the public half is +baked into the worker image. + +```sh +./bin/keygen +# prints a base64 private key and its public key +``` + +Keep the **private key** secret (a password manager or secrets store — ideally +injected per-run and kept offline-capable). Publish the **public key** to the +worker image / pinned config. Losing the private key just means generating a new +one and rotating; leaking it would let someone forge snapshots, so treat it like +a signing certificate. + +## Step 2 — Configure the builder + +The builder is configured entirely through `VAPN_`-prefixed environment +variables (it prints its effective configuration at startup, with secrets +redacted). The important ones: + +| Variable | Purpose | Example / default | +|---|---|---| +| `VAPN_DB_DSN` | PostgreSQL connection (builder role) | `postgres://vapn:…@postgres:5432/vapn` | +| `VAPN_ADVISOR_URL` | VPS Advisor base URL | `https://vpsadvisor.example` | +| `VAPN_ADVISOR_TOKEN` | Service credential for the ASN list | *(secret)* | +| `VAPN_RIS_BVIEW_URL` | Where to download the RIS dump | `https://data.ris.ripe.net/rrc00/latest-bview.gz` | +| `VAPN_RIS_BVIEW_PATH` | Local path/cache for the dump | `/work/latest-bview.gz` (dev: `data/ripe/latest-bview.gz`) | +| `VAPN_RIS_BVIEW_MAX_AGE` | Reject dumps older than this | `6h` | +| `VAPN_GEOIP_CITY_MMDB` | Path to GeoLite2-City | `/geoip/GeoLite2-City.mmdb` | +| `VAPN_SNAPSHOT_SIGNING_KEY` | Base64 private signing key (Step 1) | *(secret)* | +| `VAPN_MIN_WORKER_VERSION` | Oldest worker version allowed to use the snapshot | `0.1.0` | +| `VAPN_MAX_TARGETS_PER_PROVIDER` | Cap on probe targets per provider | `100` | +| `VAPN_SANITY_MAX_DELTA_PCT` | Hold for approval if prefix count swings more than this % | `50` | +| `VAPN_SANITY_FORCE` | Bypass the sanity gate (use sparingly) | `false` | +| `VAPN_RETAIN_SNAPSHOTS` | How many old snapshots to keep | `5` | +| `VAPN_ARTIFACT_S3_*` | Artifact store (endpoint, bucket, keys) | see [deployment](../operations/deployment.md) | + +The full list with defaults is in the +[configuration reference](../reference/configuration.md). + +## Step 3 — Run it + +### Managed (recommended): the production stack + +In the production Compose stack the builder is wired as a scheduled job via +systemd timer units shipped in the repo: + +``` +deploy/prod/systemd/vapn-builder.service +deploy/prod/systemd/vapn-builder.timer +``` + +Install and enable the timer (runs the builder daily): + +```sh +sudo cp deploy/prod/systemd/vapn-builder.* /etc/systemd/system/ +sudo systemctl enable --now vapn-builder.timer +systemctl list-timers vapn-builder.timer # confirm next run +``` + +The Compose file already mounts the GeoIP volume (kept fresh by the +`geoipupdate` container) and passes the environment. See the +[deployment guide](../operations/deployment.md) for the whole stack. + +### Manual: one run, for understanding or debugging + +```sh +# from a checkout, with the env vars above exported: +make build +./bin/builder +``` + +It will log each stage — sync, download, parse, filter, validate, enrich, load, +target-derivation, sanity gate, export, publish — and exit. In development the +defaults point at the pre-downloaded `data/ripe/` and `data/geo-data/`, so it +runs fully offline. + +## Step 4 — Verify it worked + +```sh +# Are snapshots being published? +./bin/vapnctl snapshots list +# → shows versions, status (published/superseded/failed), counts, timestamps +``` + +A healthy result: a recent snapshot with status **`published`**, a sensible +prefix count, and a `built_at` within your cadence. You can also check that +workers see it — `vapn status` on a worker shows the snapshot version it holds. + +## Updating the builder + +The builder is just a container image in the platform stack. Update it with the +rest of the platform during a normal [platform upgrade](../operations/upgrades.md) +(pull the new image tag, let the timer run the new version). There's no separate +builder update ceremony — because it's a batch job, the next scheduled run +simply uses the new image. + +## Monitoring + +Watch these (details in [Operations → Monitoring](../operations/monitoring.md)): + +| Signal | Healthy | Alert when | +|---|---|---| +| **Snapshot age** | < 1× cadence | > 2× cadence (builds are failing or the timer isn't firing) | +| **Snapshot status** | `published` | repeated `failed` | +| **Prefix count** | stable-ish day to day | wild swings (may indicate a route leak — the sanity gate should catch these) | +| **Builder run duration** | consistent | growing sharply (data-size or resource issue) | + +## Troubleshooting + +| Symptom | Likely cause | Fix | +|---|---|---| +| `RIS bview too old` | Download failed or cache stale | Check outbound HTTPS to `data.ris.ripe.net`; delete the cached file to force a fresh fetch | +| Build **held for approval** | Prefix count swung past `SANITY_MAX_DELTA_PCT` | Investigate the swing (real routing change vs leak); approve, or set `SANITY_FORCE=true` only if you're sure | +| `GeoIP … not found` | GeoLite2 not present or key missing | Confirm the `geoipupdate` container ran and the mount path matches `VAPN_GEOIP_CITY_MMDB` | +| `configuration errors: VAPN_…` | A required variable is unset | The builder lists exactly which — set them | +| Snapshot `failed` repeatedly | Any pipeline step erroring | Read the builder logs; the failing stage is named. Previous published snapshot stays in force meanwhile | +| Signature/verify errors on workers | Signing key ≠ pinned public key | Ensure the worker image's pinned public key matches `VAPN_SNAPSHOT_SIGNING_KEY` | + +## Recovery + +- **A bad snapshot got published.** Roll back instantly: + `./bin/vapnctl snapshots rollback ` (audited). Workers switch on + their next heartbeat. +- **The builder can't run for a while.** No emergency — workers keep using the + last published snapshot indefinitely. Fix the cause and let the next run + publish. Routing membership changes slowly. +- **Signing key compromised.** Generate a new key, update the worker image's + pinned public key (a worker release), and rotate. The old key's snapshots + remain verifiable until you retire them. + +For the full design and rationale, return to the +[builder overview](README.md). diff --git a/docs/concepts/README.md b/docs/concepts/README.md new file mode 100644 index 0000000..69ae1ec --- /dev/null +++ b/docs/concepts/README.md @@ -0,0 +1,65 @@ +# Core Concepts + +VAPN sits on top of how the Internet actually routes traffic — a topic most +software developers never need to touch. This section teaches the foundations +**from first principles**, assuming you're a capable programmer who has simply +never worked with Internet routing. No prior knowledge of IP, CIDR, ASN, BGP, +or RIPE is assumed. + +Read these in order the first time; each builds on the last. If you only have +ten minutes, read [The Internet in one page](#the-internet-in-one-page) below +and then jump to whichever concept a guide sent you here for. + +## Learning path + +| # | Concept | You'll understand | +|---|---|---| +| 1 | [How the Internet works](how-the-internet-works.md) | IP addresses, subnets, **CIDR**, how a packet finds its way | +| 2 | [ASN & BGP](asn-and-bgp.md) | **Autonomous Systems**, **BGP**, how networks announce and reach each other | +| 3 | [RIPE, RIS & MRT](ripe-and-ris.md) | Who records global routing, and the **MRT** files that capture it | +| 4 | [Prefix ownership](prefix-ownership.md) | How VAPN decides which addresses belong to a provider | +| 5 | [GeoIP](geoip.md) | Turning IP addresses into countries and regions | +| 6 | [Measurement, consensus & trust](measurement-and-consensus.md) | Why one measurement isn't enough, and how VAPN builds public truth | + +## The Internet in one page + +Before the details, here's the whole chain VAPN depends on, in plain terms: + +1. **Everything online has an IP address** — a numeric label like a postal + address. Addresses come in blocks, written in **CIDR** notation like + `203.0.113.0/24` (256 addresses). → [1](how-the-internet-works.md) + +2. **The Internet is a network of networks.** Each big network — an ISP, a + cloud provider, a university — is an **Autonomous System** with a number, + its **ASN** (e.g. AS64500). A hosting provider owns one or more ASNs. + → [2](asn-and-bgp.md) + +3. **Networks tell each other which addresses they own** using a protocol + called **BGP**. "AS64500 announces `203.0.113.0/24`" means: to reach any + address in that block, send traffic toward AS64500. → [2](asn-and-bgp.md) + +4. **These announcements are recorded.** **RIPE** (a regional Internet + registry) runs **RIS**, a system of collectors that continuously capture + global BGP announcements into files in the **MRT** format. + → [3](ripe-and-ris.md) + +5. **VAPN reads those recordings** to learn, for each provider it monitors, + exactly which address blocks that provider announces to the world — its + *public network*. → [4](prefix-ownership.md) + +6. **Workers then probe representative addresses** from those blocks, from many + places, and VAPN combines the results into a trusted verdict. + → [6](measurement-and-consensus.md) + +```mermaid +flowchart TD + IP["IP addresses
203.0.113.0/24"] --> ASN["ASNs own address blocks
AS64500"] + ASN --> BGP["BGP announces them
to the whole Internet"] + BGP --> RIS["RIPE RIS records the
announcements as MRT files"] + RIS --> OWN["VAPN extracts each provider's
own prefixes"] + OWN --> PROBE["Workers probe them
from many vantage points"] + PROBE --> VERDICT["Consensus verdict:
healthy / degraded / outage"] +``` + +Everything VAPN does is a careful, trustworthy pipeline over those six ideas. +Start with [How the Internet works](how-the-internet-works.md). diff --git a/docs/concepts/asn-and-bgp.md b/docs/concepts/asn-and-bgp.md new file mode 100644 index 0000000..2479503 --- /dev/null +++ b/docs/concepts/asn-and-bgp.md @@ -0,0 +1,146 @@ +# ASN & BGP: The Internet's Network of Networks + +**Problem this solves:** VAPN monitors *providers*, and a provider's identity on +the Internet is its **ASN**(s). To know which addresses belong to a provider — +and therefore which addresses to probe — you have to understand what an ASN is +and how networks announce their addresses using **BGP**. This page explains +both from scratch. + +Read [How the Internet works](how-the-internet-works.md) first if you haven't; +this page assumes you know what an IP prefix is. + +## The Internet is a network of networks + +"The Internet" isn't one network. It's tens of thousands of independent +networks that agree to interconnect. Each independent network — a big ISP, a +cloud provider, a university, a hosting company — is called an **Autonomous +System (AS)**. + +"Autonomous" is the key word: each AS is run by one organization that makes its +own routing decisions internally. From the outside, an AS is a black box that +says "here are the address blocks you can reach through me." + +Every AS has a globally unique number, its **Autonomous System Number (ASN)**, +written like `AS64500`. ASNs are handed out by the same regional registries +that hand out address blocks (RIPE, ARIN, APNIC, …). A hosting provider you'd +find on VPS Advisor typically owns **one or a few ASNs**. + +> **Analogy:** if IP prefixes are streets, an **AS is a courier company** with +> a company number (its ASN). Each courier controls delivery within its own +> territory and hands packages off to other couriers at the borders. "The +> postal system" is all these couriers cooperating. + +This is why VAPN's source of truth is *ASNs*: a provider *is*, on the Internet, +its set of ASNs and the address blocks those ASNs announce. + +## BGP: how networks tell each other what they can reach + +Given thousands of autonomous networks, how does traffic get from any one to any +other? They have to exchange reachability information: "I can deliver to these +blocks; here's the path." The protocol they use is the **Border Gateway +Protocol (BGP)** — the routing protocol *between* Autonomous Systems. BGP is, +quite literally, what glues the Internet together. + +Here's the essential mechanic. An AS **announces** (or "advertises") the +prefixes it can deliver to, along with the **AS path** — the sequence of ASes a +packet would traverse to get there. + +``` +AS64500 announces: 203.0.113.0/24 AS path: [64500] +``` + +That means: "I, AS64500, originate `203.0.113.0/24` — to reach it, come to me." +AS64500 is the **origin AS** for that prefix. + +Its neighbors re-announce it, prepending themselves to the path: + +``` +AS64500 → AS200 → AS100 +AS100 now knows: 203.0.113.0/24 AS path: [100, 200, 64500] +``` + +Read the path right-to-left as "…via AS200, which originated at AS64500." Every +router that hears this announcement learns: *to reach `203.0.113.0/24`, forward +toward the neighbor that told me, and the traffic will end up at AS64500.* + +```mermaid +flowchart LR + O["AS64500
(origin)
owns 203.0.113.0/24"] -->|announce| T1["AS200
(transit)"] + T1 -->|re-announce| T2["AS100
(transit)"] + T2 -->|re-announce| R["Rest of the Internet"] + R -.->|"traffic flows back
along the path"| O +``` + +> **Analogy:** each courier company shouts to its neighbors, "I can deliver to +> Maple Street!" Neighbors relay the shout, adding "…and you get there through +> me." Eventually every hub knows a direction that leads to Maple Street, even +> though none of them has a global map. Routing is **rumor that converges.** + +### Route propagation and why it's not instant + +When AS64500 first announces a prefix, the announcement **propagates** outward +hop by hop. It takes seconds to a few minutes for the whole Internet to hear it. +The same is true when a route is *withdrawn* (an AS says "I can no longer reach +this block") — that news propagates too. + +This propagation is where **routing health** shows up: + +- If a provider's routes are **withdrawn** unexpectedly (a misconfiguration, an + upstream failure), parts of the Internet lose the path to it — an **outage**, + even if the provider's servers are fine. +- If routes keep **flapping** (announced, withdrawn, re-announced), the network + is unstable — routing churn that hurts reachability intermittently. + +VAPN doesn't speak BGP itself. Instead it reads *recordings* of these +announcements (via [RIPE RIS](ripe-and-ris.md)) to learn each provider's +current prefixes, and detects instability by comparing successive recordings +plus what workers actually observe. + +## The origin AS: who *owns* a prefix + +For VAPN, one field matters most: the **origin AS** — the AS at the end of the +path, the one that actually originates the announcement. If AS64500 is a +provider VAPN monitors, then every prefix whose origin is AS64500 is part of +that provider's public network and is fair game to probe. + +This sounds clean, and usually is, but the real world adds wrinkles: + +- **A prefix can be announced by more than one origin AS** at once — called a + **MOAS** (Multiple Origin AS). Sometimes legitimate (a company using two of + its own ASNs), sometimes a sign of a hijack or leak. +- **Bogus announcements** exist — private or reserved address blocks + ("bogons") that should never appear in global routing, or absurd prefix + lengths. + +VAPN's builder filters these carefully so it never probes something a provider +doesn't actually own. That filtering is the subject of +[Prefix ownership](prefix-ownership.md). + +## Why this matters to VAPN, concretely + +Put it together and you get VAPN's core data question, now fully grounded: + +> "For provider *P*, which owns ASNs *{A₁, A₂, …}*, what is the complete set of +> address prefixes currently originated by those ASNs on the global Internet?" + +Answer that accurately and you know exactly what *P*'s public network is — the +set of streets to send probes down. Answer it *wrong* (include someone else's +prefix, or miss one) and your measurements are meaningless or unfair. Getting +this right, safely, is why VAPN has a dedicated +[Routing Builder](../builder/README.md). + +## Key terms from this page + +| Term | Meaning | +|---|---| +| **Autonomous System (AS)** | An independently run network (ISP, cloud, host) | +| **ASN** | The globally unique number identifying an AS (`AS64500`) | +| **BGP** | The protocol ASes use to announce which prefixes they can reach | +| **Announce / withdraw** | Advertise a prefix, or retract it | +| **AS path** | The sequence of ASes a route traversed | +| **Origin AS** | The AS that originates a prefix — its owner, for VAPN's purposes | +| **MOAS** | Multiple Origin AS — a prefix announced by more than one origin | +| **Route propagation** | How announcements/withdrawals spread across the Internet over seconds-to-minutes | + +Next: [RIPE, RIS & MRT](ripe-and-ris.md) — where these announcements get +recorded, and the file format VAPN reads them from. diff --git a/docs/concepts/geoip.md b/docs/concepts/geoip.md new file mode 100644 index 0000000..de785fe --- /dev/null +++ b/docs/concepts/geoip.md @@ -0,0 +1,106 @@ +# GeoIP: Turning Addresses into Places + +**Problem this solves:** VAPN answers not just "is this provider healthy?" but +"is it healthy *in my region*?" To do that it must attach geography to two +things: the provider's prefixes, and the workers doing the measuring. That +mapping — IP address → place — is **GeoIP**, and VAPN uses MaxMind's GeoLite2 +databases for it. + +## What GeoIP is + +An IP address is just a number; it doesn't inherently encode a location. +**GeoIP** is the practice of looking an address up in a database that maps +address ranges to approximate locations (country, region, city, coordinates) and +to the registered network (ASN). Companies like **MaxMind** compile these +databases from registry data, network measurements, and other signals. + +VAPN uses two MaxMind GeoLite2 databases: + +| Database | Maps an IP to | Used by | +|---|---|---| +| **GeoLite2-City** | Country, city, latitude/longitude | The **builder**, to enrich prefixes | +| **GeoLite2-ASN** | The registered ASN/organization | The **coordinator**, to verify a worker's source network | + +The databases are distributed as **`.mmdb`** files (MaxMind DB format) — a +compact, memory-mappable binary optimized for fast range lookups. VAPN reads +them with a standard Go MaxMind library (`internal/routing/geo`). + +> **Accuracy caveat, stated up front:** GeoIP is *approximate*. Country-level +> data is usually reliable; city-level is a best-guess. VAPN treats geography as +> a **bucketing hint** for regional verdicts and worker diversity — never as +> ground truth about exactly where a server sits. Design choices lean on this: +> a region with thin or uncertain data yields `insufficient_data`, not a false +> verdict. + +## How VAPN uses it + +### Enriching prefixes (builder) + +When the [builder](../builder/README.md) extracts a provider's prefixes from +[RIS data](ripe-and-ris.md), it looks each one up in GeoLite2-City and attaches +`geo_country`, `geo_city`, and coordinates. This lets the platform reason about +*where* a provider's network lives — e.g. "this block is announced from +Frankfurt" — and later group verdicts by region. + +### Verifying workers (coordinator) + +When a worker talks to the coordinator, the coordinator sees the worker's public +source IP. Looking that up in GeoLite2-ASN gives two things: + +1. The worker's **source ASN** — the network it probes *from*. VAPN records this + to (a) spread each measurement across *different* source networks for + diversity, and (b) **exclude a worker from measuring its own provider's + network**, so a provider can't grade its own homework. See the + [security model](../architecture/05-security-trust-model.md). +2. The worker's **verified country**, compared against what the worker + self-reports. A mismatch is a mild trust signal. + +## Regional verdicts, made concrete + +Because prefixes and workers both carry geography, consensus can be computed per +region, not just globally: + +```mermaid +flowchart LR + P["Provider prefixes
geolocated by builder"] --> AGG + W["Workers
bucketed by region"] -->|"measurements tagged
with worker region"| AGG + AGG["Aggregation engine"] --> G["Global verdict"] + AGG --> R1["eu-west: healthy"] + AGG --> R2["ap-south: insufficient_data"] +``` + +A provider can be **healthy globally but degraded in one region** — exactly the +nuance a prospective customer cares about ("great provider, but how is it from +*where I am*?"). GeoIP is what makes that distinction possible. + +## Licensing and the independent update cadence + +MaxMind's GeoLite2 databases are free but require a **license key** and must not +be redistributed. Two consequences shape VAPN's design: + +- **Each deployer uses their own MaxMind license key.** The project never ships + or redistributes the databases. In production the `geoipupdate` container + refreshes them with the operator's key; in development, pre-downloaded copies + live under `data/geo-data/`. Workers that want a local copy use the + *operator's own* key too (optional — geo features degrade gracefully without + it). This is tracked as [risk R8](../architecture/08-risk-assessment.md). +- **GeoIP updates are independent of routing updates.** A GeoIP refresh never + requires rebuilding a routing snapshot, and vice versa. They're separate jobs + on separate cadences (routing daily, GeoIP weekly). This keeps a slow or + failed MaxMind download from blocking routing builds and keeps the two + concerns decoupled. See the + [snapshot lifecycle](../architecture/06-lifecycles.md#2-snapshot-lifecycle). + +## Key terms from this page + +| Term | Meaning | +|---|---| +| **GeoIP** | Mapping IP addresses to approximate geographic locations | +| **MaxMind / GeoLite2** | The provider and free databases VAPN uses | +| **`.mmdb`** | MaxMind's compact binary database file format | +| **GeoLite2-City / -ASN** | The two databases: location, and registered network | +| **Source ASN** | The network a worker probes *from*, derived from its IP | +| **Regional verdict** | A health verdict computed for one region, enabled by GeoIP | + +Next: [Measurement, consensus & trust](measurement-and-consensus.md) — how many +independent measurements become one trustworthy public verdict. diff --git a/docs/concepts/how-the-internet-works.md b/docs/concepts/how-the-internet-works.md new file mode 100644 index 0000000..a7c4edc --- /dev/null +++ b/docs/concepts/how-the-internet-works.md @@ -0,0 +1,151 @@ +# How the Internet Works (for Developers) + +**Problem this solves:** to measure whether a provider's *network* is +reachable, you first have to understand what "an address on the network" even +is, and how a packet you send finds its way there. This page builds that +foundation. If you've written an HTTP client but never thought about what +happens below the socket, this is for you. + +> **Analogy we'll use throughout:** the Internet is a global postal system. +> Addresses, postcodes, sorting hubs, and delivery routes all have direct +> equivalents. We'll build the analogy up piece by piece. + +## IP addresses: the postal addresses of the Internet + +Every device that talks on the Internet has at least one **IP address** — a +numeric label that uniquely identifies it, like a postal address identifies a +building. + +There are two versions in use: + +- **IPv4** — four numbers 0–255 separated by dots: `203.0.113.7`. That's 32 + bits, so about 4.3 billion possible addresses. We ran out of these years ago, + which is why they're rationed and traded. +- **IPv6** — eight groups of hexadecimal: `2001:db8::7`. That's 128 bits — an + effectively unlimited supply. + +VAPN handles both. When this documentation writes an address, assume the same +ideas apply to v4 and v6 unless stated otherwise. + +``` +203.0.113.7 +└──┬──┘ └┬┘ + network host (roughly: which building block, which door) +``` + +That split — "which block of the network" vs "which specific host in it" — is +the single most important idea for understanding routing. Let's make it precise. + +## Address blocks and CIDR notation + +Addresses aren't handed out one at a time; they're allocated in **contiguous +blocks**. A block is written in **CIDR** notation (Classless Inter-Domain +Routing), which looks like: + +``` +203.0.113.0 / 24 +└────┬────┘ └┬┘ + address prefix length (bits fixed) +``` + +The number after the slash is the **prefix length**: how many leading bits are +fixed ("the network part"). The rest are free ("the host part"). + +- `/24` fixes the first 24 of 32 bits, leaving 8 bits → **256** addresses + (`203.0.113.0` … `203.0.113.255`). +- `/16` fixes 16 bits, leaving 16 → **65,536** addresses. +- `/32` fixes all 32 bits → exactly **one** address. + +**Smaller number after the slash = bigger block.** Each step down doubles the +size. A `/23` is two `/24`s; a `/22` is four. + +> **Postal analogy:** `203.0.113.0/24` is "every address on Maple Street" — +> a whole street of 256 doors. `203.0.113.7/32` is "7 Maple Street" — one +> door. Routing works at the *street* level, not the door level, which is why +> blocks matter. + +In VAPN's vocabulary, an announced address block is called a **prefix**. When +you read "a provider's prefixes," think "the streets that provider owns." A +provider might own `203.0.113.0/24`, `198.51.100.0/22`, and a handful of IPv6 +blocks — collectively, its *public network*. + +### Why blocks instead of individual addresses? + +Because routers would drown otherwise. There are billions of addresses but only +around a million *blocks* announced globally. Routers keep a table of blocks, +not addresses — "to reach anything in `203.0.113.0/24`, go this way." Grouping +addresses into blocks is what keeps the global routing table a manageable size. +This is the same reason VAPN reasons about prefixes, not individual IPs. + +## How a packet finds its way + +Say your worker sends a probe to `203.0.113.7`. How does that packet cross the +planet to the right building? + +1. Your machine sees the destination isn't on its local network, so it hands the + packet to its **default gateway** (your router). +2. Your router doesn't know where `203.0.113.7` is either, so it forwards the + packet toward *its* provider (your ISP). +3. At each hop, a router consults its **routing table**: a big list of "for + this block of addresses, send toward that neighbor." It picks the entry that + most specifically matches the destination and forwards the packet one hop + closer. +4. This repeats — router to router, network to network — until the packet + reaches a router *inside* the destination's network, which delivers it to + the actual host. + +```mermaid +flowchart LR + W[Your worker] --> R1[Your router] + R1 --> ISP[Your ISP] + ISP --> T1[Transit network] + T1 --> T2[Another network] + T2 --> P["Provider's network
(owns 203.0.113.0/24)"] + P --> H[Host 203.0.113.7] +``` + +> **Postal analogy:** no single post office knows the location of every address +> on Earth. Each sorting hub only knows "mail for this region goes to that next +> hub." A letter hops hub to hub, each step narrowing it down, until it reaches +> the local office that knows the actual street. + +The crucial insight: **no router has a map of the whole Internet.** Each one +just knows the *next hop* for each block of addresses. The global journey +emerges from many local decisions. + +Which raises the question this whole system depends on: **how does a router +learn "for block X, send toward network Y"?** That's the job of the routing +protocol between networks — **BGP** — and the identity of those networks — +**ASNs** — which is exactly what the [next page](asn-and-bgp.md) covers. + +## Latency and reachability: what VAPN actually measures + +Two properties of that journey are what VAPN cares about: + +- **Reachability** — does a packet get there and back *at all*? If it does, the + chain of routes from your vantage point to the provider is intact. +- **Latency** — how long does the round trip take? This is the **RTT** + (round-trip time), measured in milliseconds. It reflects physical distance and + network congestion along the path. + +A worker measures both by sending an **ICMP echo request** (a "ping") to a +target address and timing the reply. Reachability = did a reply come back; +latency = how long it took. Simple in principle — but a *single* ping from a +*single* place is a weak signal, which is the problem +[consensus](measurement-and-consensus.md) exists to solve. + +## Key terms from this page + +| Term | Meaning | +|---|---| +| **IP address** | Numeric label identifying a device (`203.0.113.7`) | +| **IPv4 / IPv6** | The 32-bit and 128-bit address schemes | +| **CIDR** | Notation for an address block: `address/prefix-length` | +| **Prefix** | An announced address block; a provider's "streets" | +| **Prefix length** | Bits fixed as the network part; smaller = bigger block | +| **Routing table** | A router's list of "for this block, next hop is…" | +| **RTT** | Round-trip time — the latency of a probe | +| **Reachability** | Whether a packet gets there and back at all | + +Next: [ASN & BGP](asn-and-bgp.md) — who owns these blocks, and how the whole +Internet learns where they are. diff --git a/docs/concepts/measurement-and-consensus.md b/docs/concepts/measurement-and-consensus.md new file mode 100644 index 0000000..3bbef6d --- /dev/null +++ b/docs/concepts/measurement-and-consensus.md @@ -0,0 +1,258 @@ +# Measurement, Consensus & Trust + +**Problem this solves:** VAPN's entire output is a public verdict about a +provider's health. That verdict must be *trustworthy* even though it's built +from measurements taken by strangers on hardware VAPN doesn't control, some of +whom may be careless, misconfigured, or actively malicious. This page explains +how VAPN turns many shaky individual measurements into one verdict you can rely +on: through **redundancy**, **consensus**, and **trust**. + +This is the conceptual heart of the project. The mechanics are implemented in +`internal/aggregate` (consensus) and `internal/trust` (scoring); this page +teaches the ideas, and the [trust walkthrough](../walkthroughs/trust-calculation.md) +traces the numbers. + +## What a single measurement is + +A worker measures a target by sending an **ICMP echo request** — a "ping" — and +waiting for the echo reply. From one probe it learns two things: + +- **Reachable?** — did a reply come back at all. +- **Latency (RTT)** — how many milliseconds the round trip took. + +It repeats on an interval, batches the results, signs them, and uploads. A +single result looks like: + +```json +{ "target": "203.0.113.7", "probe_type": "icmp", + "measured_at": "2026-07-18T08:04:31Z", + "ok": true, "rtt_ms": 22.9, "packets_sent": 4, "packets_lost": 0 } +``` + +Simple. And on its own, **almost worthless.** Here's why. + +## Why a single measurement isn't enough + +### Why one *vantage point* isn't enough + +A worker measures the path *from itself* to the target. If that worker's local +ISP has a problem, the worker sees "unreachable" even though the provider is +perfectly healthy for everyone else. Conversely, a worker sitting in the same +data center as the provider sees "healthy" even during a global routing outage. + +> **A measurement is a statement about a path, not about a provider.** Any one +> path can be broken or privileged for reasons that have nothing to do with the +> provider. You need *many* paths, from *many* places, before you can say +> anything about the provider itself. + +### Why ICMP *alone* is insufficient + +Even with many vantage points, ICMP has real limitations you must design around: + +- **Some hosts deliberately drop ICMP.** A target that never answers pings + isn't necessarily down — it may just be configured to ignore them. Treating + silence as "outage" would produce constant false alarms. +- **ICMP can be de-prioritized.** Routers sometimes rate-limit or deprioritize + ICMP, so occasional loss or latency spikes don't necessarily reflect real + user-traffic conditions. +- **Reachability ≠ service health.** A pingable IP doesn't prove the web server, + API, or VM behind it is serving requests. + +VAPN handles these honestly: + +1. **A never-responding target is a non-signal, not an outage.** The + aggregation engine only counts targets that answered *someone* in the + trailing 24 hours ("responsive targets"). An address that never speaks ICMP + simply doesn't vote — it isn't evidence of a problem. +2. **ICMP is one protocol behind a protocol-agnostic engine.** The worker's + prober is an interface (`Probe(target, params) → Observation`); ICMP is the + first implementation. TCP-connect, traceroute, and HTTP(S) checks can be + added *behind the same interface* without changing scheduling, upload, or + aggregation. Measurement records already carry a `probe_type` and a typed + metrics blob so new protocols need no schema redesign. So "ICMP alone" is a + v1 starting point, not an architectural ceiling. + +## The fix: community measurement + redundancy + +Because no single path or worker can be trusted, VAPN measures every target with +**many workers at once** — a **redundancy group**. The scheduler deliberately +spreads a target's workers across **different regions and different source +networks (ASNs), ideally different operators**, so the group's opinions are +*independent*. It also refuses to let a worker measure its own provider's +network. + +```mermaid +flowchart TD + T["Target 203.0.113.7"] + T --> W1["Worker in Helsinki
AS-X, operator A"] + T --> W2["Worker in Virginia
AS-Y, operator B"] + T --> W3["Worker in Singapore
AS-Z, operator C"] + W1 & W2 & W3 --> C["Consensus:
combine independent views"] +``` + +Independence is what makes the combined result meaningful. Three workers on the +same ISP are barely better than one; three workers on three continents behind +three networks are a real cross-section of the Internet. + +## Consensus: from many views to one verdict + +**Consensus** is how VAPN combines a redundancy group's measurements into a +verdict. The rules are deliberately conservative — VAPN would rather say "not +sure" than cry wolf. Here's the actual model (see `internal/aggregate`): + +1. **Window the data.** Group observations into fixed time windows (default + **5 minutes**). Everything below happens per window, per provider, per + region, per probe type. +2. **Per target, workers vote by trust weight.** Each worker's observations of a + target reduce to an "ok-ratio" (fraction reachable). Workers vote with their + [trust](#trust) as weight (with a small floor so brand-new workers still + count a little). A target is considered **up** if **≥ 50% of the voting + weight** saw it reachable. +3. **Only responsive targets count.** As noted above, targets that answered + nobody recently are excluded — silence isn't an outage. +4. **Roll targets up to a provider verdict** based on the fraction of measured + targets that are up: + + | Up fraction | Verdict | + |---|---| + | ≥ 90% | **healthy** | + | ≥ 50% | **degraded** | + | < 50% | **outage** | + | too few distinct workers, or no measured targets | **insufficient_data** | + +5. **Attach confidence and metrics.** Each verdict carries a **confidence** + score (higher with more distinct workers and lower dissent), latency + percentiles (p50/p95/p99), and a loss rate. + +Two verdicts deserve emphasis: + +- **`insufficient_data` is a first-class outcome, not an error.** If fewer than + the minimum number of distinct workers (default **3**) measured a provider, + VAPN publishes `insufficient_data` — explicitly "we don't have enough to + say," which the website must render as *"not enough data,"* never as an + outage. +- **The verdict is a pure function of consensus, never of raw observations.** + No single worker's report ever becomes public. Only the windowed, trust- + weighted aggregate leaves the platform. + +### Detecting anomalies + +Consensus also produces the signals for **anomaly detection**: + +- **Reachability loss** — a transition into `degraded`/`outage` opens an anomaly + (and returning to `healthy` resolves it). +- **Latency regression** — the current window's p50 latency compared against the + provider's trailing 6-hour baseline; a large jump (default ≥ 2×) opens a + latency anomaly. +- **Routing churn** — big swings in a provider's prefix set between snapshots + (from the [builder](../builder/README.md)) indicate BGP instability. + +These drive the "recent instability" indicators on VPS Advisor. + +## Trust + +Consensus weights each worker by **trust** — a continuous score in **[0, 1]** +reflecting how reliable that worker has proven to be. Trust is what lets VAPN +lean on a good community worker while discounting a flaky or dishonest one, +*without* an administrator hand-grading anyone. It's recomputed continuously +(see `internal/trust`). + +Trust combines four components: + +| Component | What it measures | Effect | +|---|---|---| +| **Agreement** (dominant) | How well the worker's measurements match the *settled* consensus of its groups | The core signal — honest, accurate workers agree with the crowd over time | +| **Availability** | Heartbeat regularity and lease fulfillment | Reliable, always-on workers score higher | +| **Tenure** | How long the worker has been approved — a slow ramp | New workers start near the floor and rise over ~2 weeks | +| **Penalties** | Security events (invalid signatures, replays, policy violations) | Subtract sharply, decay slowly | + +The score is roughly: + +``` +score = availability × (0.2 + 0.3 × tenure) + 0.5 × agreement − penalty (clamped to [0,1]) +``` + +Three design choices matter: + +- **Agreement is scored against the *settled* window, not the instantaneous + majority.** A sharp worker that spots an outage a few minutes before the crowd + is *right*, and the final settled consensus rewards it rather than punishing + it for early disagreement. +- **Tenure caps Sybil value.** Because new workers start low and rise slowly, + spinning up a hundred fresh workers buys very little influence — you'd have to + operate them honestly for weeks. Combined with per-operator weight caps, this + blunts [Sybil attacks](../architecture/05-security-trust-model.md#3-threat-matrix-v1-scope). +- **Non-active workers have zero weight, always.** Only `active` workers + contribute. `quarantined` workers keep measuring in **shadow mode** (weight 0) + so they can rebuild agreement and earn their way back. + +### Reputation and the worker lifecycle + +Trust feeds the worker **lifecycle** — the reputation system administrators and +automation use to keep the fleet honest: + +```mermaid +stateDiagram-v2 + [*] --> pending: enroll + pending --> active: admin approves + active --> quarantined: trust collapse / anomaly (automatic) + quarantined --> active: agreement recovers / admin reinstates + active --> suspended: admin suspends + suspended --> active: admin reinstates + active --> retired: admin retires + suspended --> retired + quarantined --> retired + retired --> [*] +``` + +- **pending** — enrolled, awaiting human approval; heartbeats but no work. +- **active** — full participant; weight = trust. +- **quarantined** — measures in shadow mode (weight 0), earning trust back; + entered automatically on trust collapse or anomalies. +- **suspended** — locked out entirely (admin action). +- **retired** — terminal; keys revoked, history kept. + +The rule throughout: **administrators outrank automation.** The system may +*quarantine* automatically, but only a human retires or reinstates a worker. +Full detail in the [trust model](../architecture/05-security-trust-model.md#4-trust-model) +and [worker lifecycle](../worker/lifecycle.md). + +## Putting it together + +```mermaid +flowchart LR + subgraph Many independent workers + O1[obs] & O2[obs] & O3[obs] + end + O1 & O2 & O3 --> W["Weight each by trust"] + W --> V["Per-target vote (≥50% weight up)"] + V --> R["Provider verdict + confidence
(healthy/degraded/outage/insufficient_data)"] + R --> A["Anomaly detection"] + R --> P["Publish to VPS Advisor"] + R -->|"agreement feeds back"| T[Trust scores] + T --> W +``` + +A worker's honesty raises its trust; its trust raises its influence on +consensus; consensus defines truth; and truth, compared back against each +worker, updates trust. That feedback loop — not any single measurement — is what +makes VAPN's public verdicts trustworthy. + +## Key terms from this page + +| Term | Meaning | +|---|---| +| **Observation** | One signed measurement result from one worker | +| **Redundancy group** | The set of independent workers covering the same target | +| **Consensus** | The trust-weighted combination of a group's measurements into a verdict | +| **Responsive target** | A target that answered someone recently; only these vote | +| **Verdict** | `healthy` / `degraded` / `outage` / `insufficient_data` | +| **Confidence** | How sure the verdict is (more workers, less dissent → higher) | +| **Trust** | A worker's [0,1] reliability score; its weight in consensus | +| **Shadow mode** | Quarantined workers measuring at weight 0 to rebuild trust | +| **Anomaly** | A detected reachability loss, latency regression, or routing churn | + +You've now covered every concept VAPN rests on. To see them all working +together on one real measurement, follow the +[end-to-end walkthrough](../walkthroughs/end-to-end.md). To go deeper on the +design, read the [Architecture](../architecture/README.md). diff --git a/docs/concepts/prefix-ownership.md b/docs/concepts/prefix-ownership.md new file mode 100644 index 0000000..ee8147c --- /dev/null +++ b/docs/concepts/prefix-ownership.md @@ -0,0 +1,142 @@ +# Prefix Ownership: Deciding What Belongs to a Provider + +**Problem this solves:** the whole system rests on one claim — *"this address +belongs to the provider we're monitoring."* If that claim is wrong, VAPN either +probes addresses it shouldn't (unfair, potentially abusive) or misjudges a +provider's health. This page explains how VAPN establishes prefix ownership +correctly and conservatively. + +Prerequisites: [ASN & BGP](asn-and-bgp.md) (origin AS, MOAS) and +[RIPE, RIS & MRT](ripe-and-ris.md) (where the data comes from). + +## Ownership = origin AS, mostly + +The starting rule is simple and follows directly from how BGP works: + +> A prefix belongs to a provider if its **origin AS** (the AS that originates +> the announcement in the routing table) is one of the provider's monitored +> ASNs. + +VPS Advisor tells VAPN which ASNs each provider owns. The RIS `bview` dump tells +VAPN which prefixes each ASN originates. Join those two and you have each +provider's prefix set. Most of the global table resolves cleanly this way. + +But "mostly" is doing real work in that sentence. Three complications force VAPN +to be careful. + +## Complication 1: duplicate prefixes + +A single `bview` dump is recorded from many BGP *peers*, so the **same prefix +from the same origin AS appears many times** — once per peer that reported it. +A million-route table can contain several million raw records. + +VAPN **deduplicates**: for the purpose of "does provider *P* announce +`203.0.113.0/24`?", ten peers reporting it and one peer reporting it mean the +same thing — *yes*. The builder collapses all records for a given +`(prefix, origin AS)` into a single fact. + +``` +raw MRT records (from many peers) after dedup +203.0.113.0/24 origin AS64500 peer A ┐ +203.0.113.0/24 origin AS64500 peer B ┼──► 203.0.113.0/24 origin AS64500 +203.0.113.0/24 origin AS64500 peer C ┘ +``` + +> **Why duplicates exist, restated:** the dump isn't "the routing table" in the +> abstract — it's "what each of N peers told this collector." Agreement across +> peers is normal and expected; VAPN only needs the distinct facts. + +## Complication 2: bogons and bogus announcements + +Not every announcement in a global dump is legitimate. Some are **bogons** — +prefixes that must never appear in public routing: + +- **Private ranges** (`10.0.0.0/8`, `192.168.0.0/16`, `172.16.0.0/12`) — for + internal networks only. +- **Reserved / documentation ranges** (`203.0.113.0/24` is literally reserved + for docs; `127.0.0.0/8` is loopback). +- **Absurd prefix lengths** — e.g. an IPv4 `/32` or wildly large blocks that + indicate a leak or misconfiguration rather than a real allocation. + +If VAPN let a bogon through, a worker might try to probe a private or reserved +address — meaningless at best, and the kind of thing that makes a monitoring +system look like an attacker at worst. So the builder runs a **bogon filter** +that drops these before they can become probe targets. (The filter lives in +`internal/routing/bogon`.) + +## Complication 3: MOAS conflicts + +Recall from [BGP](asn-and-bgp.md#the-origin-as-who-owns-a-prefix) that a prefix +can be announced by **multiple origin ASes** at once — a **MOAS** (Multiple +Origin AS). This is ambiguous ownership, and VAPN refuses to guess: + +- Sometimes it's legitimate (one company announcing from two of its own ASNs, + anycast, migrations). +- Sometimes it's a **route leak or hijack** — someone announcing a prefix they + don't own. + +Either way, if a prefix's origin is ambiguous, treating it as definitely +belonging to a monitored provider could be wrong. So VAPN **flags MOAS +conflicts** (`flags: {moas_conflict: true}` on the prefix row) rather than +silently resolving them. Flagged prefixes are visible to administrators for +review and can be excluded from probing. The guiding principle: + +> **When ownership is ambiguous, surface it — never guess.** A wrong "yes" +> pointed a fleet of workers at someone else's network. + +The same principle applies one level up: VPS Advisor enforces that **one ASN +belongs to exactly one provider** (a uniqueness constraint). If two providers +claim the same ASN, VAPN hard-errors and asks a human to resolve it, rather than +splitting or duplicating measurements. See +[domain model](../architecture/02-domain-model.md). + +## From owned prefixes to probe targets + +Knowing a provider owns `203.0.113.0/24` (256 addresses) doesn't mean VAPN +probes all 256. That would be wasteful and could look like a scan. Instead the +builder derives a small set of **probe targets** — representative addresses per +prefix/region — with a recorded *rationale* for why each was chosen. Workers +probe these targets, not entire blocks. The number per provider is capped +(`MAX_TARGETS_PER_PROVIDER`, default 100). + +```mermaid +flowchart TD + A["Monitored ASNs
(from VPS Advisor)"] --> B["Prefixes originated by them
(from RIS bview)"] + B --> C[Deduplicate] + C --> D[Drop bogons] + D --> E[Flag MOAS conflicts] + E --> F[Enrich with GeoIP] + F --> G["Derive representative
probe targets (capped)"] + G --> H["Signed target list
workers download"] +``` + +## Snapshots make ownership a point-in-time fact + +Routing changes over time — providers add and drop prefixes. VAPN captures +ownership as an immutable, versioned **routing snapshot** (see +[snapshot lifecycle](../architecture/06-lifecycles.md#2-snapshot-lifecycle)). +Each snapshot is "the truth about who owned what, as of this build." Two +benefits: + +1. **Diffing** two snapshots reveals routing churn — prefixes that appeared or + vanished — feeding [anomaly detection](measurement-and-consensus.md). +2. **Reproducibility** — a measurement can be tied to the exact snapshot that + authorized its target, so "why did we probe this?" always has an answer. + +A worker even carries the current snapshot locally (as its signed SQLite file) +and uses it to *re-check* that a target is still legitimate before probing — +defense in depth against a stale or tampered assignment. + +## Key terms from this page + +| Term | Meaning | +|---|---| +| **Prefix ownership** | The claim that a prefix belongs to a provider, via origin AS | +| **Deduplication** | Collapsing many peer reports of the same prefix into one fact | +| **Bogon** | A prefix that must never appear in public routing (private/reserved/absurd) | +| **MOAS conflict** | A prefix with multiple origin ASes — ambiguous ownership, flagged not guessed | +| **Probe target** | A representative address chosen from a prefix; what workers actually probe | +| **Routing snapshot** | An immutable, versioned record of ownership at build time | + +Next: [GeoIP](geoip.md) — attaching geography to prefixes so verdicts can be +regional. diff --git a/docs/concepts/ripe-and-ris.md b/docs/concepts/ripe-and-ris.md new file mode 100644 index 0000000..82d828f --- /dev/null +++ b/docs/concepts/ripe-and-ris.md @@ -0,0 +1,148 @@ +# RIPE, RIS & MRT: Where Routing Is Recorded + +**Problem this solves:** VAPN needs to know every prefix each monitored provider +announces — but it doesn't participate in [BGP](asn-and-bgp.md) itself. Instead +it reads a global *recording* of BGP made by **RIPE RIS** and stored in **MRT** +files. This page explains who RIPE is, what RIS records, what an MRT file is, +and why VAPN uses it. + +## RIPE NCC: one of the Internet's registries + +Someone has to hand out IP address blocks and ASNs so they don't collide. That +job is split by region among five **Regional Internet Registries (RIRs)**: + +| RIR | Region | +|---|---| +| **RIPE NCC** | Europe, Middle East, Central Asia | +| ARIN | North America | +| APNIC | Asia-Pacific | +| LACNIC | Latin America & Caribbean | +| AFRINIC | Africa | + +**RIPE NCC** (Réseaux IP Européens Network Coordination Centre) is the European +one. Beyond allocating addresses and ASNs, RIPE runs a valuable public service +for the whole Internet: **RIS**. + +> You don't need to know registry politics. The one thing that matters: RIPE +> operates a free, public, global recording of BGP that anyone can download — +> and VAPN uses it. + +## RIS: the Routing Information Service + +The **Routing Information Service (RIS)** is a network of **route collectors** +that RIPE runs around the world. Each collector is a machine that peers with +many real networks and *listens* to their [BGP announcements](asn-and-bgp.md) — +but never announces anything itself. It's a passive tape recorder for global +routing. + +Each collector is named `rrc00`, `rrc01`, … (Remote Route Collector). VAPN's +builder defaults to `rrc00`, a well-connected multi-hop collector that sees a +broad view of the global routing table. + +RIS produces two kinds of data: + +- **Updates** — a continuous stream of individual announce/withdraw events (the + play-by-play). +- **RIB dumps** (also called **`bview`** dumps) — a periodic full snapshot of + the entire routing table as the collector currently sees it, taken every 8 + hours. RIB = Routing Information Base, the router's term for "my current table + of all known routes." + +**VAPN uses the `bview` dumps.** It doesn't need the second-by-second play-by- +play; it needs "what is the complete current set of prefixes for these ASNs?" — +which is exactly a full-table snapshot. Comparing two daily snapshots is also +how VAPN spots routing churn without processing the raw update firehose. + +```mermaid +flowchart LR + N1[Real network AS64500] -->|BGP| C[(RIS collector rrc00)] + N2[Real network AS200] -->|BGP| C + N3[Real network AS100] -->|BGP| C + C -->|"every 8h: full-table snapshot"| B["bview dump
(MRT file)"] + B -->|download| V[VAPN builder] +``` + +## MRT: the file format the recording lives in + +RIS dumps are stored in **MRT** format — **M**ulti-**T**hreaded **R**outing +**T**oolkit format, a long-standing standard (RFC 6396) for recording routing +information. Think of it as the standard "container format" for BGP data, the +way PCAP is the standard container for captured packets. + +An MRT `bview` file is a compact **binary** sequence of records. Each record, at +the level VAPN cares about, says essentially: + +``` +prefix 203.0.113.0/24 originated by AS64500 (plus AS path, timestamps, peer info) +``` + +A single full-table dump contains on the order of **a million** such entries — +the entire global routing table. The files are gzip-compressed and are tens to +a few hundred megabytes. + +### Why MRT (and why binary)? + +- **It's the standard.** Every routing tool, collector, and archive speaks MRT. + Using it means VAPN interoperates with RIPE's data directly, with no + bespoke format in between. +- **It's compact.** A million routes in a human-readable format would be + enormous; MRT's binary encoding keeps full dumps to a manageable size. +- **It's complete.** A `bview` dump is the *whole* table, so VAPN can be sure it + hasn't missed a prefix for a monitored ASN. + +### Why workers never touch MRT + +Parsing a million-entry binary file is heavy, and it requires the RIPE download, +a parser, and validation logic. VAPN does this **exactly once, centrally**, in +the [builder](../builder/README.md), and hands workers a tiny, pre-digested, +signed **SQLite** file containing only the finished target list. This is a +deliberate boundary: + +> **The builder is the only component that ever reads MRT or MaxMind data.** +> Workers consume finished artifacts only — they never parse routing files, +> never download from RIPE, and never make ownership decisions. + +This keeps community workers tiny (a few MB of RAM), keeps the expensive parsing +in one auditable place, and means the routing logic can be fixed or improved +without touching thousands of deployed workers. + +## From MRT to "provider *P*'s prefixes" + +The builder's job, covered in depth in [the builder guide](../builder/README.md) +and [prefix ownership](prefix-ownership.md), is roughly: + +1. Get the list of monitored ASNs from VPS Advisor. +2. Download the latest RIS `bview` MRT dump. +3. Stream through its ~1M records, **keeping only** those whose origin AS is in + the monitored set. +4. Deduplicate, validate (drop bogons, flag MOAS conflicts), enrich with + [GeoIP](geoip.md). +5. Store the result and publish a signed target artifact. + +Note step 3: VAPN reads the *whole* global table but *keeps* only the tiny slice +that belongs to monitored providers. It never builds a database of the entire +Internet — an explicit design constraint from the project brief. + +## A note on the pre-downloaded data + +To make development fast and offline-friendly, this repository ships a +pre-downloaded RIS dump and MaxMind databases under `data/` +(`data/ripe/latest-bview.gz`, `data/geo-data/`). The builder points at these by +default in development, so you can run the whole pipeline without fetching +anything from RIPE or MaxMind. In production the builder downloads fresh data on +its own schedule. + +## Key terms from this page + +| Term | Meaning | +|---|---| +| **RIPE NCC** | The regional Internet registry for Europe; runs RIS | +| **RIR** | Regional Internet Registry (RIPE, ARIN, APNIC, LACNIC, AFRINIC) | +| **RIS** | Routing Information Service — RIPE's global BGP recording | +| **Route collector / rrc** | A passive machine that records BGP announcements | +| **RIB / `bview` dump** | A full-table snapshot of routing, taken every 8h | +| **MRT** | The binary file format (RFC 6396) routing recordings are stored in | +| **Origin AS filtering** | Keeping only prefixes originated by monitored ASNs | + +Next: [Prefix ownership](prefix-ownership.md) — how VAPN turns a million raw MRT +records into a trustworthy, de-duplicated list of a provider's addresses. diff --git a/docs/demos/phase11.md b/docs/demos/phase11.md index b1329aa..0258ace 100644 --- a/docs/demos/phase11.md +++ b/docs/demos/phase11.md @@ -10,7 +10,8 @@ checklist. Docker is an implementation detail; contributors get a stable UX: ``` -curl -fsSL https://install.vpsadvisor.com | bash # deploy/worker/install.sh +# deploy/worker/install.sh +curl -fsSL https://raw.githubusercontent.com/HummingByteDev/vpsa-network-discovery/main/deploy/worker/install.sh | bash ``` ``` diff --git a/docs/demos/phase9.md b/docs/demos/phase9.md index fe31741..98c4186 100644 --- a/docs/demos/phase9.md +++ b/docs/demos/phase9.md @@ -15,7 +15,7 @@ implementation guide. systems), and admin decisions (approve/suspend/quarantine/retire made on the website dashboard are applied idempotently). - **Integration Guide**: - [docs/integration/vpsadvisor-integration-guide.md](../integration/vpsadvisor-integration-guide.md) + [docs/integration/django-integration.md](../integration/django-integration.md) — models, endpoints with schemas, dashboard pages, permissions, jobs, notifications, rollout order. The mock stub implements this exact contract, so the guide is executable: implement it and integration is a config change. diff --git a/docs/development/README.md b/docs/development/README.md new file mode 100644 index 0000000..eddee7b --- /dev/null +++ b/docs/development/README.md @@ -0,0 +1,164 @@ +# Development + +For contributors and maintainers. How the repository is organized, how to build +and test, the standards code is held to, and how changes flow from a branch to a +release. New to the *system*? Read [Core Concepts](../concepts/README.md) and the +[Architecture](../architecture/README.md) first — this guide assumes you know +what the components do. + +## Quick start for contributors + +```sh +git clone https://github.com/HummingByteDev/vpsa-network-discovery.git +cd vpsa-network-discovery + +make dev-up # full local stack (see below) +make test-db # one-time: create the vapn_test database +make check # vet + tests + build — run this before every commit +``` + +`make dev-up` brings up the whole loop locally via +[`deploy/compose/dev.compose.yaml`](../../deploy/compose/dev.compose.yaml): +PostgreSQL (`:5433`), MinIO (artifact store), a **mock VPS Advisor** +(`internal/mockadvisor`, serving the [integration contract](../integration/README.md) +from fixtures), the coordinator, the aggregator, and a few workers. Snapshot +signing keys come from `./bin/keygen`. It runs fully offline — the pre- +downloaded `data/` (RIS bview + GeoLite2) is mounted into the builder, so no +RIPE or MaxMind fetch is needed. Follow the guided tour in +[`docs/demos/phase1.md`](../demos/phase1.md) onward. + +## Repository structure + +``` +cmd/ one main package per binary + coordinator/ worker-facing API + scheduler (long-running) + aggregator/ consensus + trust + publisher (long-running) + builder/ RIPE→PostgreSQL→signed artifact (batch job) + worker/ the community probe agent (container) + vapn/ worker operator CLI (install/status/…) + vapnctl/ platform admin CLI + migrate/ migration runner + mockadvisor/ stub of the VPS Advisor contract (dev/CI) + keygen/ Ed25519 keypair generator + loadtest/ synthetic fleet load generator +internal/ implementation packages (each has a package doc comment) + advisor/ VPS Advisor client (pull catalog, push results) + aggregate/ consensus + anomaly detection + artifact/ SQLite export, manifest, signing, store (S3/fs) + builder/ snapshot build pipeline + RIS download + coordinator/ HTTP handlers, scheduler, admin, sync, rotation + observation/ observation types + validation + probe/ protocol-agnostic prober; ICMP implementation + publisher/ outbox drain to VPS Advisor + registry/ worker registry + routing/ bogon filter, geo lookup, MRT reader + scheduler/ assignment leasing + trust/ trust scoring + wireauth/ Ed25519 request signing/verification + platform/ shared infra: db, config, logging, metrics, migrate, http, audit, version +migrations/ one ordered SQL migration stream for the whole DB +deploy/ compose (dev) · prod (production stack) · worker (installer + systemd) +docs/ this documentation +data/ pre-downloaded RIS + GeoLite2 (dev convenience) +``` + +The mapping to subsystems is in +[architecture 07 §1](../architecture/07-deployment.md#1-artifacts). + +## Architecture principles & design philosophy + +These are the invariants contributions must preserve — they're what make the +system trustworthy, not incidental style: + +1. **VPS Advisor is the only source of truth for providers.** Never build a + provider registry here; cache with provenance and drop on delist. +2. **Only aggregated consensus leaves the platform.** A single worker's + observation is never public. `ProviderStatus` is a pure function of + `ConsensusWindow`s, never of raw observations. +3. **Workers are the hostile edge.** Assume malicious workers, stolen keys, and + bad measurements. Sign, timestamp, nonce, weight by trust, and require + consensus. Services trust each other (same zone); workers are never trusted. +4. **The builder is the only MRT/MaxMind reader.** Workers consume finished, + signed artifacts. Never move parsing to the edge. +5. **Never guess on ambiguity.** MOAS conflicts are flagged, not resolved; + thin data yields `insufficient_data`, not a verdict; admins outrank + automation. +6. **Protocol-agnostic measurement.** New probe types slot behind the `Prober` + interface and the typed `metrics` JSON without schema or scheduler changes. +7. **Idempotency and at-least-once everywhere data crosses a boundary.** Batch + ids, outbox with retries, idempotent website endpoints. +8. **Everything containerized; 12-factor config.** `VAPN_`-prefixed env vars; + every service prints its effective config (secrets redacted) at boot. + +If a change would violate one of these, it needs an explicit design discussion +first — they're load-bearing. + +## Coding standards + +- **Go**, formatted with `gofmt`; `go vet` clean (`make vet`). Match the + surrounding code's naming and comment density. +- **Package docs.** Every `internal/*` package carries a doc comment explaining + its responsibility and boundary — keep it accurate when you change behavior. +- **Errors** are wrapped with context (`fmt.Errorf("...: %w", err)`). +- **Config** goes through `internal/platform/config` (`VAPN_` prefix, typed + accessors, redacted dump) — never read `os.Getenv` directly in a service. +- **Logging** via `log/slog` (structured); no secrets in logs, ever. +- **SQL** lives in migrations (`migrations/NNNN_*.sql`), one ordered stream; + queries are parameterized. Migrations must be backward-compatible one version + back (expand → migrate → contract) so replicas can roll. +- **Least privilege:** one Postgres role per service (builder can't touch + `measurements`; coordinator can't write `aggregation`; etc.). + +## Testing + +```sh +make test # go test -p 1 ./... (DB tests share one database, serialized) +make check # vet + test + build +``` + +- DB integration tests target the **`vapn_test`** database (created by + `make test-db`), **never** the live dev `vapn` database — they truncate and + reshape what they touch. The default DSN is + `postgres://vapn:vapn-dev@localhost:5433/vapn_test`; override with + `VAPN_TEST_DB_DSN`. +- Unit tests live beside code (`*_test.go`). Notable suites: `aggregate` (the + consensus SQL), `wireauth` (signing/replay), `mrtreader`, `bogon`, + `coordinator` (security, scheduler simulation), `mockadvisor` (contract). +- **Contract parity:** the `mockadvisor` tests encode the + [integration contract](../integration/django-integration.md) — the same + expectations the website's staging must satisfy. +- Add tests with behavior changes; consensus/trust/auth changes especially need + them (they're the trust-critical core). + +## Branching, CI/CD & releases + +- **Branching:** feature branches off `main`; open a PR; `main` stays green. + Never commit or push directly to `main` without review. +- **Commits:** clear, imperative subject lines; explain *why* in the body when + it's not obvious. +- **CI** (`.github/`) runs `make check` (vet + tests against `vapn_test` + + build) and the contract tests on every PR. +- **Releases:** milestone/tag-based. Images are built per component + (`coordinator`, `aggregator`, `builder`, `worker`, `migrate`) and published to + the registry; the worker binary is published to GitHub releases for + `install.sh`. The version is stamped via `-ldflags` from `git describe` (see + the [Makefile](../../Makefile)). Full process and gates: + [release management](../operations/release-management.md) and the + [launch checklist](../operations/launch-checklist.md). + +## Contribution guide + +1. **Open an issue** describing the change and, for anything touching the + invariants above, the design. +2. **Branch, build, test:** `make check` must pass; add tests for behavior + changes. +3. **Keep docs in sync.** If you change a command, config var, endpoint, or a + subsystem's behavior, update the relevant doc (this docs tree is treated as + part of the code — see the [reference](../reference/README.md) pages). +4. **Open a PR** with a clear description and rationale; link the issue. +5. **Review** focuses on correctness, the invariants, and test coverage of the + trust-critical paths. + +Where to go next: [Reference](../reference/README.md) for CLI/config/schema +details, [Operations](../operations/README.md) for how it runs in production, +and the [Handbook](../handbook/README.md) for the whole system in one narrative. diff --git a/docs/getting-started/README.md b/docs/getting-started/README.md new file mode 100644 index 0000000..4e1a1e9 --- /dev/null +++ b/docs/getting-started/README.md @@ -0,0 +1,39 @@ +# Getting Started + +The fastest path from "I've heard of VAPN" to "I'm contributing measurements." +These guides are deliberately light on theory — when you want the *why*, follow +the links into [Core Concepts](../concepts/README.md). + +There are two ways to "get started," depending on who you are: + +- **Contribute a worker.** You have a Linux machine with a bit of spare + bandwidth and want to help measure provider health. This is the common case + and the focus of this section. → [Quick Start](quickstart.md) +- **Run the whole platform.** You are deploying VAPN itself (coordinator, + aggregator, builder, database). That is an operator task with its own guide. + → [Deployment guide](../operations/deployment.md) + +## Contributor path + +| Step | Guide | Time | +|---|---|---| +| 1 | [Quick Start](quickstart.md) — get a worker running | ~5 min | +| 2 | [Installation](installation.md) — the details behind the quick start | ~10 min | +| 3 | [What it costs & how privacy works](../worker/resources-and-privacy.md) | reading | +| 4 | [Updating](updating.md) — keep the worker current | ~2 min | +| 5 | [Uninstalling](uninstalling.md) — leave cleanly whenever you like | ~2 min | + +Stuck? → [Troubleshooting](troubleshooting.md) · [FAQ](faq.md) + +## What you'll need + +- A **Linux** machine (amd64 or arm64) — a cheap VPS, a home server, or a + spare box. It does **not** need to be powerful; a worker uses a few MB of + RAM and a trickle of bandwidth (see [resource usage](../worker/resources-and-privacy.md)). +- **[Docker](https://docs.docker.com/engine/install/)** installed and running. +- An **enrollment token** from your VPS Advisor dashboard + (My Workers → Create worker). This is a one-time secret that ties the worker + to your account. + +That's the whole shopping list. Everything else — keys, routing data, GeoIP, +assignments — the worker sets up automatically. diff --git a/docs/getting-started/faq.md b/docs/getting-started/faq.md new file mode 100644 index 0000000..0d958c5 --- /dev/null +++ b/docs/getting-started/faq.md @@ -0,0 +1,107 @@ +# Frequently Asked Questions + +General and worker-focused questions. Operator/platform questions are answered +throughout [Operations](../operations/README.md); integration questions in the +[Integration guide](../integration/README.md). + +## About the project + +**What is VAPN, in one sentence?** +A distributed system where community-run workers measure the public network +health of VPS providers from many locations, and a platform combines those +measurements into trusted, consensus-based verdicts published on VPS Advisor. + +**Is this the VPS Advisor website?** +No. VPS Advisor is a separate, already-live review platform. VAPN is the +measurement backend behind its *Provider Network Health* feature. See +[Documentation Home → Project background](../README.md#project-background). + +**Does VAPN scan the whole Internet?** +No. It only ever measures providers that exist on VPS Advisor, using only the +network routes those providers publicly announce. VPS Advisor is the sole +source of truth for which providers to monitor. See +[Core Concepts](../concepts/README.md). + +**Is it open source?** +Yes — the platform, worker, and all documentation are in one repository. See +[Development](../development/README.md) to contribute. + +## Running a worker + +**What do I need to run a worker?** +A Linux machine, Docker, and an enrollment token from VPS Advisor. That's all. +→ [Quick Start](quickstart.md). + +**How much CPU / RAM / bandwidth does it use?** +Very little — a few MB of RAM, negligible CPU, and a trickle of bandwidth. Full +numbers in [Resource usage](../worker/resources-and-privacy.md#resource-usage). + +**Why is my worker "Awaiting approval"?** +New workers are approved by a human as an anti-abuse measure. Once approved it +starts probing automatically; you don't need to do anything. See +[Quick Start](quickstart.md#what-awaiting-approval-means). + +**Can my worker see or measure *my* servers?** +No. Workers only probe addresses that appear in the signed routing snapshot, +which is derived exclusively from monitored providers' publicly announced +routes. Workers never choose their own targets. See +[Privacy](../worker/resources-and-privacy.md#privacy). + +**Will running a worker expose anything about my machine?** +The worker reports its measurements, its software version, and liveness. It +does not read your files or other network traffic, and your private key never +leaves your machine. See [Privacy](../worker/resources-and-privacy.md#privacy). + +**Can I pause participation?** +Yes: `vapn pause` stops probing and keeps your identity and trust; `vapn resume` +resumes. Pausing is better than uninstalling if you'll come back — uninstalling +creates a fresh identity next time. + +**How do I completely remove it?** +`vapn uninstall` removes containers, images, and all state, and offers to +unregister you cleanly. → [Uninstalling](uninstalling.md). + +**What happens if my machine reboots?** +The worker restarts automatically (`restart: unless-stopped`) and resumes on its +own — it re-reads its identity, re-downloads any new snapshot, and continues. + +## Trust and measurements + +**What is "trust" and why is mine low?** +Trust is a 0–1 score reflecting how reliable your worker has proven to be. New +workers start near the floor and ramp up over about two weeks (this caps the +value of spinning up many fake workers). Bad clocks and invalid signatures +lower it. → [Trust](../concepts/measurement-and-consensus.md#trust). + +**Why doesn't one worker's measurement become the public result?** +Because any single worker could be wrong, misconfigured, or malicious. Public +verdicts come only from **consensus** across many workers, weighted by trust. +→ [Consensus](../concepts/measurement-and-consensus.md#consensus-from-many-views-to-one-verdict). + +**My worker says a provider is down but the site says "healthy" — why?** +Your worker is one vantage point. If most trusted workers still reach the +provider, consensus is "healthy" and a regional issue near *you* may be the +cause. If enough workers agree, the verdict changes. That disagreement is the +system working as designed. + +**What does `insufficient_data` mean on a provider?** +Not enough distinct, trusted workers measured that provider (or region) in the +window to call it confidently. It is deliberately **not** shown as an outage — +absence of data is not evidence of a problem. + +## Security & privacy + +**Could a malicious worker poison the results?** +That's an explicit design assumption. Mitigations: redundancy (many workers per +target across different operators/networks), trust weighting, dissent scoring, +signed-and-timestamped measurements, and shadow-mode quarantine. See the +[Security & trust model](../architecture/05-security-trust-model.md). + +**Is my traffic to the coordinator encrypted?** +Yes — all worker↔coordinator traffic is HTTPS, and every request is +additionally signed with your worker's key so the platform can prove it came +from you and hasn't been altered or replayed. + +**Where can I look up a term I don't understand?** +The [Glossary](../reference/glossary.md) defines every networking and +project-specific term in plain language. diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md new file mode 100644 index 0000000..90e3f7f --- /dev/null +++ b/docs/getting-started/installation.md @@ -0,0 +1,120 @@ +# Worker Installation + +This guide explains worker installation in depth — the same flow as the +[Quick Start](quickstart.md), but with the reasoning behind each choice and the +options you can take. If you just want a worker running, the Quick Start is +enough; come back here when you want to know *what* you're installing and +*why*. + +## Requirements + +| Requirement | Why | Notes | +|---|---|---| +| **Linux**, amd64 or arm64 | The worker sends raw ICMP; it targets Linux hosts | A cheap VPS, a Raspberry Pi, or a spare box all work | +| **Docker** | The worker ships as a container so you don't manage Go, libraries, or system packages | `docker info` must succeed for your user | +| **An enrollment token** | Proves a real operator started this worker | From VPS Advisor → My Workers → Create worker | +| **Outbound HTTPS (443)** | The worker talks only to the coordinator and the artifact CDN | No inbound ports are opened | +| A synchronized **clock** | Signed requests carry a timestamp; large clock skew is rejected as a replay defense | `timedatectl set-ntp true` | + +> **What is CAP_NET_RAW?** Sending an ICMP echo ("ping") packet requires a +> raw socket, which needs one Linux capability: `CAP_NET_RAW`. The worker +> container requests exactly that one capability and nothing else — it is not +> privileged, cannot see your files, and cannot touch the host network stack +> beyond sending pings. See [privacy](../worker/resources-and-privacy.md#privacy). + +## Choose how you install + +The worker's canonical source is **GitHub** — auditable, versioned, and signed +by release tooling. There are three ways to install, in increasing order of +"I want to see exactly what runs": + +### 1. Piped installer (fastest) + +```sh +curl -fsSL https://raw.githubusercontent.com/HummingByteDev/vpsa-network-discovery/main/deploy/worker/install.sh | bash +``` + +### 2. Download, read, then run (recommended) + +```sh +curl -fsSL -o vapn-install.sh \ + https://raw.githubusercontent.com/HummingByteDev/vpsa-network-discovery/main/deploy/worker/install.sh +less vapn-install.sh # read it — it's short and commented +bash vapn-install.sh +``` + +Piping a script into a shell means trusting whatever the URL serves. Reading it +first costs a minute and removes the trust assumption. The script only detects +Docker, downloads the `vapn` binary for your architecture, and runs +`vapn install`. + +### 3. Clone the repository (for contributors / air-gapped installs) + +```sh +git clone https://github.com/HummingByteDev/vpsa-network-discovery.git +cd vpsa-network-discovery +make build # produces ./bin/vapn (needs Go) +./bin/vapn install +``` + +Use this if you want to build from source, pin to a specific commit, or install +somewhere without direct GitHub access. + +## What `vapn install` does, step by step + +`install.sh` only bootstraps the CLI; the real work is in `vapn install`, which +is interactive and idempotent: + +1. **System checks (`vapn doctor`).** Docker present and reachable, enough disk + space, coordinator reachable over HTTPS, clock synchronized. Any failure + stops installation with a specific fix. +2. **Collects two settings:** the **coordinator URL** + (`VAPN_COORDINATOR_URL`, e.g. `https://probe-api.vpsadvisor.example`) and + your **enrollment token** (`VAPN_ENROLLMENT_TOKEN`). These are written to + `~/.vapn/config.env`. +3. **Generates your identity.** On first boot the worker creates an + [Ed25519](../concepts/measurement-and-consensus.md#trust) keypair. The + **private key never leaves the container volume** (`~/.vapn`). The public + key is what you register. +4. **Registers** with the coordinator using the enrollment token + public key. + The token is now spent; the worker has a permanent ID and is `pending`. +5. **Downloads the routing snapshot** — the signed list of legitimate probe + targets — and verifies its signature against a pinned key before using it. +6. **Optionally downloads GeoLite2** if you supplied a MaxMind license key + (see below). Geo features degrade gracefully without one. +7. **Starts the container** (`restart: unless-stopped`) and **verifies** it + reports healthy. + +Everything the worker needs lives under `~/.vapn` (override with `VAPN_HOME`): +identity, config, the downloaded snapshot, and the generated +`docker-compose.yml`. + +## Optional: a MaxMind license key + +VAPN uses [MaxMind GeoLite2](../concepts/geoip.md) to attribute measurements to +countries/regions. The *platform* already geolocates providers; a worker only +needs a key if you want your worker's own location verified from a local +database rather than inferred. It is entirely optional — leave it unset and the +worker still contributes fully. To supply one, get a free license key from +MaxMind and set `VAPN_MAXMIND_LICENSE_KEY` when prompted. The key is **yours** +and is never redistributed by the project (see +[risk R8](../architecture/08-risk-assessment.md)). + +## After installation + +- The worker is `pending` until an admin approves it — see + [Quick Start → Awaiting approval](quickstart.md#what-awaiting-approval-means). +- Verify any time with `vapn status` and `vapn doctor`. +- Turn on [automatic updates](updating.md). +- Learn the [full command set](../worker/command-reference.md). + +## Uninstalling + +Leaving is one command and removes everything cleanly, including your identity: + +```sh +vapn uninstall +``` + +See [Uninstalling](uninstalling.md) for what it removes and how to unregister +politely so the platform stops assigning you work. diff --git a/docs/getting-started/quickstart.md b/docs/getting-started/quickstart.md new file mode 100644 index 0000000..5f44ba6 --- /dev/null +++ b/docs/getting-started/quickstart.md @@ -0,0 +1,92 @@ +# Quick Start — Run a Worker + +Goal: a healthy VAPN worker, probing from your machine, in about five minutes. +This page is the happy path. For the reasoning behind each step, see +[Installation](installation.md); if something goes wrong, see +[Troubleshooting](troubleshooting.md). + +## Before you begin + +You need three things: + +1. A **Linux** machine (amd64/arm64) you can run a long-lived container on. +2. **Docker** installed and running — verify with `docker info`. +3. An **enrollment token** from VPS Advisor: log in → **My Workers** → + **Create worker** → copy the token it shows you *once*. + +> **What's an enrollment token?** A one-time password that proves *you* (a +> real, logged-in operator) started this worker. The worker trades it for a +> permanent cryptographic identity on first boot, then it's useless. It is +> shown only once — if you lose it, regenerate it from the same page. + +## Install + +The installer lives in the project's GitHub releases. Run: + +```sh +curl -fsSL https://raw.githubusercontent.com/HummingByteDev/vpsa-network-discovery/main/deploy/worker/install.sh | bash +``` + +> **Why the long GitHub URL?** GitHub is the canonical, auditable source for +> the installer — no vanity redirect to trust. Prefer to read before you run? +> That's encouraged — see +> [Installation → Install from GitHub](installation.md#choose-how-you-install), +> which covers downloading and inspecting the script (or cloning the repo) +> first. + +The installer: + +1. Confirms Docker is present and the daemon is reachable. +2. Downloads the small `vapn` command-line tool for your architecture. +3. Hands over to `vapn install`, which asks for your **coordinator URL** and + **enrollment token**, runs system checks, starts the worker, and verifies + it comes up: + +``` +✓ Docker detected ✓ Coordinator reachable +✓ Disk space ✓ Clock synchronization +✓ Registration successful +Worker ID: 9f30… +Status: Awaiting approval +``` + +## What "Awaiting approval" means + +Your worker registered successfully and is now **pending**. A human +administrator reviews new workers before they contribute to public results — +this is a deliberate anti-abuse step (see [Trust model](../concepts/measurement-and-consensus.md#trust)). +You don't have to do anything: once approved, the worker starts probing on its +own. Approval status is visible on your VPS Advisor dashboard. + +## Check on it + +```sh +vapn status +``` + +You'll see health, the routing snapshot version it downloaded, how many +assignments it holds, and how many measurements it has submitted. A few useful +commands day-to-day: + +```sh +vapn logs -f # live logs +vapn pause # stop probing (identity kept; resume any time) +vapn resume +vapn doctor # re-run the system checks if something looks off +``` + +The full list is the [Worker command reference](../worker/command-reference.md). + +## You're done + +That's it — the worker is self-managing. It renews its own credentials, +downloads new routing snapshots (verifying each one cryptographically), retries +failed uploads, and recovers from reboots on its own. You should rarely need to +touch it. + +### Next steps + +- Understand exactly [what the worker does and what it costs you](../worker/resources-and-privacy.md). +- Turn on [automatic updates](updating.md). +- Curious how your packets become a public verdict? Follow the + [end-to-end walkthrough](../walkthroughs/end-to-end.md). diff --git a/docs/getting-started/troubleshooting.md b/docs/getting-started/troubleshooting.md new file mode 100644 index 0000000..12db773 --- /dev/null +++ b/docs/getting-started/troubleshooting.md @@ -0,0 +1,91 @@ +# Troubleshooting (Workers) + +Most worker problems are one of a handful of things: Docker permissions, a +clock that drifted, a firewall blocking egress, or simply waiting on manual +approval. This page is symptom-first. For platform-side operator troubleshooting +see [Operations → Runbooks](../operations/runbooks.md). + +> **Two commands solve most issues.** `vapn doctor` re-runs the system checks +> and tells you exactly what's wrong; `vapn logs` shows what the worker is +> actually doing. Start there. + +## Symptom → fix + +| Symptom | Likely cause | Fix | +|---|---|---| +| Stuck at **`Awaiting approval`** | Approval is a manual, human step | Check your VPS Advisor dashboard; nothing to do worker-side | +| **`Unreachable (last report …)`** in status | Worker crashed, host offline, or egress blocked | `vapn logs`, then `vapn doctor` | +| **Clock check fails** | No NTP; signed requests are time-bound | `sudo timedatectl set-ntp true` | +| **Docker permission errors** | Your user isn't in the `docker` group | `sudo usermod -aG docker $USER`, then log out/in | +| **`Coordinator unreachable`** | Strict egress firewall | Allow outbound HTTPS (443) to the coordinator domain | +| **Registration fails / token rejected** | Token spent, expired, or mistyped | Regenerate the token on VPS Advisor and re-run `vapn install` | +| **Snapshot verification failed** | Corrupt download or wrong pinned key | `vapn logs`; the worker retries automatically — persistent failure means report it | +| **No assignments after approval** | Scheduler still balancing, or you're the only worker for those targets | Wait a heartbeat cycle (~30 s); check `vapn status` | +| **Trust stuck low** | New workers start near the floor by design | [Tenure ramps over ~2 weeks](../concepts/measurement-and-consensus.md#trust); check for `bad_signature`/`replay` events with `vapn logs` | + +## Diagnosing step by step + +1. **Is the container running?** + ```sh + vapn status + ``` + If it shows the worker as down or restarting, go to logs. + +2. **What do the logs say?** + ```sh + vapn logs -f + ``` + Look for the last error line. Common ones map directly to the table above + (`clock skew`, `dial tcp … i/o timeout`, `403 suspended`, `401 signature`). + +3. **Do the system checks pass?** + ```sh + vapn doctor + ``` + Each failed check prints a one-line remedy. + +4. **Is it a network problem?** From the host: + ```sh + curl -fsSI https:///healthz + ``` + A hang or refusal means egress filtering; a 200 means the path is clear and + the problem is worker-side. + +## Specific situations + +### "My worker was quarantined" + +The platform quarantines workers whose measurements consistently disagree with +consensus, whose clock is wrong, or that submit invalid signatures. Quarantined +workers keep measuring in **shadow mode** (weight 0) and can earn trust back. +`vapn logs` and the trust events in `vapn status` explain why. Fix the +underlying issue (usually the clock or an outdated image) and trust recovers +over subsequent windows. See [Trust calculation](../walkthroughs/trust-calculation.md). + +### "It was working, now it's silent" + +Usually a host reboot or Docker restart. The worker is set to +`restart: unless-stopped`, so it should recover on its own; if it doesn't: + +```sh +vapn resume # if you had paused it +vapn logs -f # find the crash +vapn doctor # confirm environment still healthy +``` + +### "Updates keep rolling back" + +`vapn update` rolls back if the new image doesn't become healthy in two +minutes. That means the *new* version fails in your environment — capture logs +during the attempt (`vapn logs -f` in another terminal) and report them. Your +worker stays on the working previous version in the meantime. + +## Still stuck? + +- Re-read the relevant guide: [Installation](installation.md) · + [Updating](updating.md) · [FAQ](faq.md). +- Understand the lifecycle so the state names make sense: + [Worker lifecycle](../worker/lifecycle.md). +- Open an issue on the project's GitHub with `vapn doctor` output and the last + ~50 lines of `vapn logs` (they contain no secrets — the private key never + appears in logs). diff --git a/docs/getting-started/uninstalling.md b/docs/getting-started/uninstalling.md new file mode 100644 index 0000000..50529bd --- /dev/null +++ b/docs/getting-started/uninstalling.md @@ -0,0 +1,66 @@ +# Uninstalling a Worker + +You can leave the network at any time, and leaving is clean — no orphaned +containers, images, or files. There is no lock-in and no penalty. + +## The one command + +```sh +vapn uninstall +``` + +It walks you through two choices, then removes everything: + +1. **Unregister with the network? (recommended)** This tells the coordinator + your worker is retiring so it stops assigning you work and marks your worker + `retired` (keys revoked, history retained for audit). Declining just stops + the local container; the platform will notice the silence and flag the + worker `unreachable` on its own, but unregistering is the polite, immediate + way out. +2. **Keep a backup?** Offers to archive your identity and config first (see + below) in case you want to come back. + +Then it removes the containers, the worker image, and the entire `~/.vapn` +directory. + +## Just pausing, not leaving + +If you only want to stop probing for a while — a maintenance window, a busy +period on your link — don't uninstall. Pause instead: + +```sh +vapn pause # stops probing; keeps your identity and trust +vapn resume # back to work +``` + +While paused, your assignments redistribute to other workers within a minute, +and your accumulated [trust](../concepts/measurement-and-consensus.md#trust) is +preserved. Uninstalling and re-installing, by contrast, creates a *new* worker +identity that starts trust-building from scratch. + +## Backups (and what's in them) + +```sh +vapn backup +``` + +This archives your identity and configuration to a `tar.gz`. **It contains your +private key** — treat it like a password. A backup lets you restore the *same* +worker identity (and its trust history) on the same or another machine instead +of enrolling fresh. + +## Manual cleanup (if `vapn` is already gone) + +If you deleted the CLI before uninstalling, remove the pieces by hand: + +```sh +docker compose -f ~/.vapn/docker-compose.yml down --rmi all # stop + remove image +rm -rf ~/.vapn # remove state +sudo rm -f /usr/local/bin/vapn # remove the CLI +``` + +Your worker will then go silent and the platform will flag it `unreachable`; +an admin can retire it. To retire it immediately and cleanly, prefer +`vapn unregister` *before* manual cleanup. + +Thanks for the packets. diff --git a/docs/getting-started/updating.md b/docs/getting-started/updating.md new file mode 100644 index 0000000..48654b8 --- /dev/null +++ b/docs/getting-started/updating.md @@ -0,0 +1,66 @@ +# Updating a Worker + +Workers are versioned container images. Keeping current matters: the platform +can require a minimum worker version (old workers get drained until they +upgrade), and updates carry fixes and new probe types. Updating is safe by +design — it is **health-gated with automatic rollback**. + +## Update on demand + +```sh +vapn update +``` + +This pulls the latest worker image, restarts the container, and waits for it to +report healthy. **If the new version fails to become healthy within two +minutes, it is automatically rolled back to the previous image.** You cannot +break a working worker by updating it. + +## Automatic updates (recommended) + +The installer offers to enable a systemd timer that checks for updates daily at +a randomized hour (so the whole fleet doesn't update in lockstep). To enable it +later, install the units shipped in the repository: + +``` +deploy/worker/vapn-update.service +deploy/worker/vapn-update.timer +``` + +```sh +sudo cp deploy/worker/vapn-update.* /etc/systemd/system/ +sudo systemctl enable --now vapn-update.timer +``` + +The timer runs `vapn update --auto`, which is the same health-gated, +auto-rollback update — just unattended. + +> **Prefer to control timing yourself?** Skip the timer and run `vapn update` +> from your own cron or configuration-management tooling. The command is +> idempotent and safe to run when already up to date. + +## How the platform signals "please upgrade" + +Every heartbeat reports your worker's version. If your version is below the +platform's **minimum supported version**, the coordinator responds with an +`upgrade_required` control action: the worker is *drained* (its assignments +move to other workers) and refused new leases until it upgrades. This is how +old, potentially buggy workers are retired gracefully without cutting anyone +off abruptly. You'll see it in `vapn status` and `vapn logs`. + +## Verifying an update + +```sh +vapn version # CLI version +vapn status # shows running worker image/version + health +vapn logs -f # watch it come back up +``` + +If an update ever leaves you in a bad state (it shouldn't, thanks to +auto-rollback), see [Troubleshooting](troubleshooting.md). + +## Updating the CLI itself + +`vapn update` updates the *worker image*. To update the `vapn` CLI binary, +re-run the installer (it overwrites the binary in place) or `git pull && +make build` if you installed from source. diff --git a/docs/handbook/README.md b/docs/handbook/README.md new file mode 100644 index 0000000..71fc838 --- /dev/null +++ b/docs/handbook/README.md @@ -0,0 +1,692 @@ +# The VAPN Project Handbook + +*A complete guide to the VPS Advisor Probe Network — from Internet fundamentals +to production operation.* + +--- + +This handbook tells the whole story of VAPN in one place, in order, as a book. +It starts by assuming you know how to write software but have never worked with +Internet routing, builds the necessary foundations, and then introduces every +subsystem until you understand the entire platform — why it exists, how it +works, and how to run it. + +It is designed to be read cover to cover and to be exported as a PDF. Where a +topic has a dedicated reference elsewhere in the documentation, this handbook +teaches the essentials inline and links out for exhaustive detail, so you never +*need* to leave — but can when you want more. + +**How to read it.** If you're new, read straight through. If you know +networking, skim Part I and start at [Chapter 6](#chapter-6--project-goals). If +you're here for one subsystem, use the table of contents. Every chapter ends +knowing what the next one needs. + +## Table of contents + +**Part I — Foundations** +1. [Introduction](#chapter-1--introduction) +2. [Internet Fundamentals](#chapter-2--internet-fundamentals) +3. [Autonomous Systems & ASNs](#chapter-3--autonomous-systems--asns) +4. [BGP: How Networks Find Each Other](#chapter-4--bgp-how-networks-find-each-other) +5. [RIPE, RIS & MRT](#chapter-5--ripe-ris--mrt) + +**Part II — The System** +6. [Project Goals](#chapter-6--project-goals) +7. [System Architecture](#chapter-7--system-architecture) +8. [Component Walkthrough](#chapter-8--component-walkthrough) +9. [The Builder](#chapter-9--the-builder) +10. [Workers](#chapter-10--workers) +11. [Aggregation & Consensus](#chapter-11--aggregation--consensus) +12. [Trust & Security](#chapter-12--trust--security) +13. [VPS Advisor Integration](#chapter-13--vps-advisor-integration) + +**Part III — Running It** +14. [Deployment](#chapter-14--deployment) +15. [Operations](#chapter-15--operations) +16. [Development](#chapter-16--development) +17. [Troubleshooting](#chapter-17--troubleshooting) +18. [Future Roadmap](#chapter-18--future-roadmap) + +--- + +# Part I — Foundations + +# Chapter 1 — Introduction + +Imagine you're choosing a hosting provider. Reviews tell you about support and +price, but they can't answer the question that matters once your site is live: +**is this provider's network actually reliable — and reliable from where my +users are?** Traditional uptime monitors can't help either: they watch a server +*you already own*, not a provider you're *considering*. + +VAPN — the **VPS Advisor Probe Network** — exists to answer that question +honestly. It's a distributed system in which a global community runs small +programs called **workers** that measure the public network health of VPS +providers from many vantage points. No single worker is trusted on its own; +instead the platform combines many independent measurements into a +**consensus** verdict, weights each worker by how reliable it has proven to be +(**trust**), and publishes only the agreed-upon result to the VPS Advisor +website for the world to see. + +Three things make VAPN different from an uptime checker: + +- **It measures providers, not your servers.** The target is a provider's own + *public network* — the address blocks it announces to the Internet. +- **It's independent and communal.** Measurements come from strangers around the + world, not from the provider (who would grade their own homework) or a single + biased vantage point. +- **It assumes the measurers might lie.** Cryptographic signing, redundancy, + consensus, and a trust system make the *aggregate* trustworthy even when + individual participants aren't. + +> **A crucial boundary, stated once and for all.** VPS Advisor is an +> independent, already-live provider-review website. **VAPN is not that website.** +> VAPN is the measurement backend behind one of its features (Provider Network +> Health). VPS Advisor decides *which* providers exist and *which* networks they +> own; VAPN measures them and reports back. Two systems, one clean contract. + +To understand how VAPN measures a "provider's public network," you first need a +little of the networking that VAPN is built on. That's Part I. If you already +know IP, CIDR, ASN, BGP, and RIPE, skip to +[Chapter 6](#chapter-6--project-goals). + +# Chapter 2 — Internet Fundamentals + +The Internet is best understood as a **global postal system**, and we'll use +that analogy throughout Part I. + +**Every device online has an IP address** — a numeric label, like a building's +postal address. IPv4 addresses look like `203.0.113.7` (32 bits, ~4.3 billion of +them); IPv6 like `2001:db8::7` (128 bits, effectively unlimited). VAPN handles +both. + +**Addresses are allocated in blocks**, written in **CIDR** notation: +`203.0.113.0/24`. The number after the slash — the *prefix length* — says how +many leading bits are fixed as the "network part." A `/24` fixes 24 bits and +leaves 8, so it's 256 addresses; a `/16` is 65,536; a `/32` is a single address. +**Smaller number after the slash means a bigger block.** In postal terms, +`203.0.113.0/24` is "all 256 doors on Maple Street," and `203.0.113.7/32` is +"7 Maple Street." Routing happens at the *street* level, which is why blocks +matter. + +In VAPN's language, an announced address block is a **prefix**. A hosting +provider owns a handful of prefixes — collectively, its *public network*. + +**How does a packet reach an address?** No router has a map of the whole +Internet. Each one keeps a *routing table* — "for this block of addresses, hand +the packet to that neighbor" — and forwards each packet one hop closer. The +packet hops router to router, network to network, each making a purely local +decision, until it reaches a router inside the destination's network. Just like +a letter hops sorting hub to sorting hub, each hub knowing only "mail for that +region goes this way." + +Two properties of that journey are what VAPN measures: + +- **Reachability** — does a packet get there and back at all? +- **Latency (RTT)** — how many milliseconds the round trip takes. + +A worker measures both by sending an **ICMP echo request** (a "ping") and timing +the reply. Simple — but a single ping from a single place is a weak signal, +which is the problem the rest of the system exists to solve. + +The open question this raises: *how does a router learn "for block X, send toward +network Y"?* That requires knowing what the networks are (Chapter 3) and how they +tell each other what they can reach (Chapter 4). + +> Full treatment with diagrams: [How the Internet works](../concepts/how-the-internet-works.md). + +# Chapter 3 — Autonomous Systems & ASNs + +"The Internet" isn't one network; it's tens of thousands of independent networks +that agree to interconnect. Each one — an ISP, a cloud, a university, a hosting +company — is an **Autonomous System (AS)**: a black box run by one organization +that makes its own internal routing decisions. + +Every AS has a globally unique number, its **ASN**, written like `AS64500`. In +our analogy, an **AS is a courier company** and its ASN is the company number: +each courier controls delivery in its own territory and hands packages to other +couriers at the borders. + +This is the linchpin for VAPN. A hosting provider you'd review on VPS Advisor +*is*, on the Internet, **its ASN (or ASNs) and the address blocks those ASNs +announce.** That's why VAPN's source of truth is ASNs: VPS Advisor records "this +provider owns AS64500," and VAPN's job becomes "find every prefix AS64500 +announces, and measure it." + +A provider may own several ASNs; an ASN belongs to exactly one provider. VAPN +enforces that last rule strictly — an ASN claimed by two providers is an error a +human must resolve, never something VAPN guesses about. + +> Full treatment: [ASN & BGP](../concepts/asn-and-bgp.md). + +# Chapter 4 — BGP: How Networks Find Each Other + +Given thousands of autonomous networks, how does any one reach any other? They +exchange reachability information using the **Border Gateway Protocol (BGP)** — +the routing protocol *between* Autonomous Systems, and quite literally what glues +the Internet together. + +The mechanic: an AS **announces** the prefixes it can deliver to, with the **AS +path** showing the route. AS64500 announces `203.0.113.0/24` with path +`[64500]` — "I originate this block; come to me." Its neighbors re-announce it, +each prepending itself: `[100, 200, 64500]` means "reach it via AS200, which +reaches AS64500." Every router that hears this learns a *direction* toward the +block, without any router holding a global map. Routing is **rumor that +converges** — each courier shouts "I can deliver to Maple Street!" and neighbors +relay the shout, adding "…through me." + +The AS at the end of the path — the one that actually originates the +announcement — is the **origin AS**. For VAPN, the origin AS *is* ownership: if +AS64500 is a monitored provider, every prefix originated by AS64500 is part of +that provider's public network. + +Two BGP realities matter later: + +- **Announcements propagate over seconds to minutes**, and can be **withdrawn**. + An unexpected withdrawal means parts of the Internet lose the path to a + provider — a real outage, even if the provider's servers are fine. Repeated + announce/withdraw (**flapping**) means instability. +- **A prefix can have multiple origin ASes at once** — a **MOAS**. Sometimes + legitimate, sometimes a hijack or leak. Ambiguous ownership that VAPN must + handle carefully, never guess. + +VAPN doesn't speak BGP itself. It reads *recordings* of these announcements — +which is Chapter 5. + +> Full treatment: [ASN & BGP](../concepts/asn-and-bgp.md). + +# Chapter 5 — RIPE, RIS & MRT + +Someone has to allocate addresses and ASNs so they don't collide; that's split +by region among five **Regional Internet Registries**. **RIPE NCC** is the +European one, and — crucially for VAPN — it runs a free, public, global recording +of BGP called the **Routing Information Service (RIS)**. + +RIS is a network of **route collectors** — machines that peer with many real +networks and passively *listen* to their BGP announcements, recording everything +and announcing nothing. A tape recorder for global routing. Each collector +produces periodic **`bview` dumps**: full snapshots of the entire routing table +(about a million routes), taken every 8 hours, stored in the **MRT** format — the +standard binary container for routing data (RFC 6396). + +VAPN builds from these `bview` dumps because it needs *membership* ("which +prefixes do these ASNs announce right now?"), which a full-table snapshot answers +directly, and *churn* (which it gets by diffing successive snapshots) — neither +needs the raw second-by-second update stream. A dump is far simpler and safer to +process. + +One dump is ~1M binary records and tens to hundreds of MB. VAPN reads the whole +thing but **keeps only the tiny slice** originated by monitored ASNs — it never +builds a database of the entire Internet. And it does this parsing **exactly +once, centrally**, in the builder; workers never touch MRT. This single boundary +is why workers can be a few MB of RAM and why the routing logic can be improved +without redeploying the fleet. + +> Full treatment: [RIPE/RIS/MRT](../concepts/ripe-and-ris.md). To make +> development offline-friendly, the repo ships a pre-downloaded dump and MaxMind +> databases under `data/`. + +With the foundations in place, we can now describe what VAPN actually does. + +--- + +# Part II — The System + +# Chapter 6 — Project Goals + +VAPN's purpose is to answer, trustworthily and at scale: + +> *Is this provider's public network healthy — globally, and in my region — right +> now, and how has it behaved recently?* + +Concretely, the project must: + +- Measure the **public network health** of providers listed on VPS Advisor, + using only the routes those providers announce. +- Use **community-operated** workers from many vantage points. +- Produce **trust-weighted, consensus** verdicts — never expose a single + worker's raw opinion. +- Detect **anomalies** (reachability loss, latency regressions, routing churn). +- Push results back to VPS Advisor for display. + +And it must do so under hostile assumptions — **malicious workers exist, +credentials get stolen, measurements are unreliable** — while staying: +scalable (tens of thousands of providers, thousands of workers), maintainable, +secure, modular, observable, and well-documented. + +Equally important is what VAPN **does not** do: it doesn't scan the Internet for +providers, doesn't maintain its own provider registry, and doesn't touch +provider profiles, reviews, accounts, or billing. VPS Advisor owns all of that. +VAPN is measurement; VPS Advisor is truth-of-record and presentation. + +> Full brief: [`CLAUDE.md`](../../CLAUDE.md). Architecture rationale: +> [architecture 01 §1](../architecture/01-system-architecture.md). + +# Chapter 7 — System Architecture + +VAPN is deliberately split from VPS Advisor along a **control-plane / data-plane** +line — the one clarification the design made to the original brief: + +- **Control plane — VPS Advisor** (existing site, extended): human-facing, + low-volume — operator accounts, worker enrollment and approval, the provider + catalog, monitoring configuration, the admin dashboard, and ingestion of + aggregated results. +- **Data plane — VAPN** (this repository): machine-facing, high-volume — worker + heartbeats, assignment leases, observation uploads, snapshot downloads. + +Why split them? Thousands of workers probing every few seconds must never hit the +production review website. The split gives volume isolation, independent scaling, +and a small change surface on the existing site. Workers are *logically* +registered with VPS Advisor (accounts and approval live there) but *operationally* +talk only to VAPN's coordinator. + +```mermaid +flowchart TB + VA["VPS Advisor Website (control plane)
catalog · accounts · enrollment · admin · Results API"] + subgraph VAPN [VAPN platform (data plane)] + B[Snapshot Builder] --> PG[(PostgreSQL)] + CO[Coordinator + Scheduler] --> PG + AG[Aggregation Engine] --> PG + B --> AS[(Artifact Store)] + end + VA -->|provider/ASN pull| VAPN + VAPN -->|aggregated results push| VA + AS -->|signed snapshots| WK[Community Workers] + WK -->|signed observations| CO +``` + +Technology choices in brief: **Go** for every service and the worker (single +static binaries, tiny community Docker images, good raw-socket support); +**PostgreSQL 16+** everywhere server-side (partitioning, `cidr`/`inet` types, +LISTEN/NOTIFY); **SQLite** for the read-only artifact workers download; +**Ed25519** for worker request signing and artifact signing. + +The system is built to be *correct* at hundreds of workers and *not structurally +blocked* from tens of thousands: the coordinator is stateless (scale +horizontally), measurements are partitioned and batched, artifact distribution is +CDN-offloadable from day one, and aggregation is parallel by provider. + +> Full treatment: [architecture 01](../architecture/01-system-architecture.md). + +# Chapter 8 — Component Walkthrough + +Five components, each with a sharp boundary. This chapter names them; the next +chapters go deep on the interesting ones. + +- **Snapshot Builder** (`builder`) — a *batch job* that owns routing + intelligence. Pulls the monitored-ASN list, downloads RIPE RIS data, extracts + and validates each provider's prefixes, enriches with GeoIP, and publishes a + signed SQLite artifact. The only component that reads MRT or MaxMind data. + → [Chapter 9](#chapter-9--the-builder) +- **Coordinator** (`coordinator`) — a *long-running service*, the single endpoint + workers talk to. Authenticates workers, enforces lifecycle, runs the embedded + **scheduler** (targets → assignments → leases with redundancy), ingests signed + observations, serves snapshot metadata. Never computes public status; never + decides which providers are monitored. +- **Aggregation Engine** (`aggregator`) — a *long-running service* that owns + consensus and public truth. Windows observations, computes trust-weighted + verdicts and confidence, detects anomalies, updates trust, and pushes results + to VPS Advisor via an outbox. → [Chapter 11](#chapter-11--aggregation--consensus) +- **Worker** (`worker`) — a *community container* with minimal config. Generates + a keypair, registers, downloads and verifies snapshots, leases assignments, + probes, signs and uploads observations, self-updates. The probe engine is + protocol-agnostic. → [Chapter 10](#chapter-10--workers) +- **Artifact Store** — dumb, cacheable HTTPS storage for snapshots. Untrusted by + design (integrity comes from signatures), so it can sit behind a CDN. + +Behind them all: **PostgreSQL** (six schemas — routing, registry, scheduling, +measurements, aggregation, audit) and the **VPS Advisor** control plane. + +The end-to-end flow that ties them together — provider added → ASN synced → +snapshot built → workers probe → consensus computed → results published — is +told stage by stage in the +[end-to-end walkthrough](../walkthroughs/end-to-end.md). + +# Chapter 9 — The Builder + +The builder solves one hard, sensitive problem: **produce a trustworthy, +signed list of exactly which addresses belong to each monitored provider.** Get +it wrong and workers probe someone else's network. + +Each run, on a daily schedule: + +1. **Sync monitored ASNs** from VPS Advisor. +2. **Download the RIS `bview`** MRT dump (or use the pre-downloaded dev copy). +3. **Parse and origin-filter** — stream ~1M records, keep only prefixes + originated by a monitored ASN (the big reduction). +4. **Deduplicate** — a dump reports each prefix from many peers; collapse to one + fact per `(prefix, origin AS)`. +5. **Validate** — drop **bogons** (private/reserved/absurd prefixes that must + never be probed) and **flag MOAS conflicts** for review rather than guessing + ownership. +6. **Enrich with GeoIP** — attach country/city/coordinates from MaxMind + GeoLite2, so verdicts can later be regional. +7. **Load into PostgreSQL** under a new immutable snapshot version. +8. **Derive probe targets** — a small, capped set of representative addresses per + prefix (not every address in a block, which would look like a scan). +9. **Sanity-gate** — if the prefix count swung wildly versus the previous + snapshot, *hold for admin approval* (a route leak could be poisoning the + data). +10. **Export and sign** — write the worker-facing subset to a compact SQLite + artifact and sign a manifest (counts, sha256, Ed25519 signature, + min worker version). +11. **Publish atomically** — upload, verify readback, mark `published`, previous + → `superseded`. Any failure marks the build `failed` and leaves the previous + version fully in force; workers never see a half-built snapshot. + +Two validation layers protect the fleet: the *build-time* sanity gate (bad input) +and the *consume-time* signature check every worker performs (bad or tampered +artifact). If a bad snapshot ever slips through, an admin runs +`vapnctl snapshots rollback ` and, because publishing is atomic, +workers switch to the good version on their next heartbeat. + +> Full treatment: [The Routing Builder](../builder/README.md); the run in +> detail: [snapshot publishing](../walkthroughs/snapshot-publishing.md); +> the ownership logic: [prefix ownership](../concepts/prefix-ownership.md). + +# Chapter 10 — Workers + +A worker is a small Docker container run by a community member. It's intentionally +minimal — a few MB of RAM, negligible CPU, a trickle of bandwidth, no inbound +ports — because all the heavy lifting lives in the builder and coordinator. + +**Its life begins with enrollment.** The operator creates a worker on VPS Advisor +and receives a one-time **enrollment token**. On first boot the worker generates +an **Ed25519 keypair** — the private key never leaves the machine — registers with +the coordinator using the token and its public key, and enters the `pending` +state. A human admin then approves it (an anti-abuse gate), and it becomes +`active`. + +**Its steady state is a loop.** Every ~30 seconds it heartbeats, receiving config, +control actions, the current snapshot version, and lease renewals. It downloads +and cryptographically verifies new snapshots (rejecting anything whose signature +or hash doesn't match its pinned key). It leases **assignments** from the +scheduler — never choosing its own targets — probes each on its interval (ICMP +echo in v1), batches the results, **signs each observation and the batch**, and +uploads. If the coordinator is down, it queues locally and retries; on shutdown it +releases its leases cleanly. + +**Every request it makes is signed**, carrying a worker id, timestamp, nonce, and +Ed25519 signature over the method, path, and body hash. This proves the request +came from that worker, wasn't altered, and isn't a replay — on top of HTTPS. + +**The operator stays in control** through a small CLI: `vapn status`, `logs`, +`pause`/`resume`, `update` (health-gated with automatic rollback), +`unregister`, `uninstall`. Everything else — credential renewal, snapshot +downloads, retries, reboot recovery — the worker does itself. + +The probe engine is **protocol-agnostic**: ICMP is the first implementation of a +`Prober` interface, and TCP-connect, traceroute, and HTTP(S) checks can be added +behind the same interface without changing scheduling, upload, or aggregation. + +> Full treatment: [Community Workers](../worker/README.md), the +> [lifecycle](../worker/lifecycle.md), and the +> [measurement](../walkthroughs/measurement-lifecycle.md) and +> [authentication](../walkthroughs/worker-authentication.md) walkthroughs. + +# Chapter 11 — Aggregation & Consensus + +This is where many shaky individual measurements become one verdict you can rely +on. The aggregation engine's guiding posture is **conservative**: it would rather +say "not enough data" than cry wolf. + +**Why a single measurement can't be trusted.** A measurement is a statement about +a *path*, from one worker to one target — not about the provider. That path can +be broken (the worker's local ISP has a problem) or privileged (the worker sits in +the provider's own data center) for reasons unrelated to the provider. And ICMP +has quirks: some hosts deliberately drop pings, so silence isn't necessarily an +outage. + +**The fix is redundancy plus consensus.** Every target is measured by several +workers (default 3), deliberately spread across different regions and source +networks, and never by a worker on the provider's own network. Each window +(default 5 minutes), the engine: + +1. Reduces each worker's observations of a target to an ok-ratio. +2. Has workers **vote by trust weight**; a target is **up** if ≥ 50% of weight + saw it reachable. Only **responsive targets** (that answered *someone* + recently) count — a never-answering address is a non-signal, not an outage. +3. Rolls targets up to a provider **verdict** by the up-fraction: **healthy** + (≥ 90%), **degraded** (≥ 50%), **outage** (< 50%), or **insufficient_data** + (fewer than 3 distinct workers, or nothing measurable). +4. Attaches **confidence** (higher with more workers, less dissent) and latency + percentiles. + +Two rules are sacred: **`insufficient_data` is a real, honest outcome** (rendered +as "not enough data," never an outage), and **the public verdict is a pure +function of consensus, never of raw observations** — no single worker's report is +ever published. + +The same pipeline detects **anomalies** — a transition into degraded/outage opens +a reachability anomaly; a latency p50 ≥ 2× the 6-hour baseline opens a latency +anomaly; big prefix swings between snapshots signal routing churn — and it scores +each worker's **agreement** against the settled consensus, which feeds trust +(Chapter 12). Results are pushed to VPS Advisor through an **outbox** with +at-least-once, idempotent delivery, so website downtime never loses data. + +> Full treatment: [Measurement, consensus & trust](../concepts/measurement-and-consensus.md). + +# Chapter 12 — Trust & Security + +VAPN's security model starts from three assumptions taken literally: **malicious +workers exist, credentials get compromised, measurements are unreliable.** The +community runs the workers; the platform must stay trustworthy anyway. + +**Trust** is a continuous score in [0,1] per worker, its weight in consensus, +recomputed continuously from four parts: **agreement** (dominant — how well its +measurements match the *settled* consensus, so a worker that spots an outage early +isn't punished), **availability** (heartbeat regularity), **tenure** (a slow ramp +so new workers start near the floor and rise over ~2 weeks), and **penalties** +(bad signatures, replays — subtract sharply, decay slowly). Non-`active` workers +always weigh zero. The formula and worked examples are in the +[trust walkthrough](../walkthroughs/trust-calculation.md). + +**Reputation is a lifecycle**: `pending` → `active` → possibly `quarantined` +(shadow mode: measures at weight 0, earning trust back) → `suspended` or +`retired`. Automation may quarantine; only a human retires or reinstates — +**administrators always outrank automation.** + +**Authentication** is Ed25519 request signing over HTTPS: every request carries a +timestamp (±120 s window) and a per-worker nonce, giving replay protection without +perfectly synced clocks. Keys rotate (with an overlap window) and can be revoked +instantly. Observations are individually signed, so provenance survives forever. + +The **threat matrix** is explicit and each threat has a mitigation: fabricated +measurements (redundancy + trust weighting + shadow-mode quarantine); **Sybil** +attacks (manual approval + tenure ramp + per-operator weight caps); stolen keys +(rotation + revocation + anomaly-triggered re-verification); replays (timestamp + +nonce); a worker probing its own network (source-ASN exclusion); tampered +snapshots (signed manifests, downgrade protection); malicious admins (append-only +audit, attributed actions, separated tokens); coordinator DoS (per-worker rate +limits, batch-only uploads, stateless scaling); and poisoned routing data +(bogon/MOAS validation, sanity gate, rollback). + +Everything security-relevant lands in an **append-only audit log**, and security +aggregates flow to the VPS Advisor admin dashboard so site admins see the posture +without platform access. + +> Full treatment: [Security & trust model](../architecture/05-security-trust-model.md). + +# Chapter 13 — VPS Advisor Integration + +VAPN is only useful connected to VPS Advisor, and the two meet at a small, precise +contract. The website is the source of truth and human surface; the platform is +the measurement machine. **Four flows** cross the boundary: + +1. **Provider catalog** — platform *pulls* which providers/ASNs to monitor + (~2 min). +2. **Enrollment** — platform *pulls* pending worker enrollments (~2 min). +3. **Admin decisions** — platform *pulls* approve/suspend/quarantine/retire + decisions (~2 min). +4. **Results** — platform *pushes* verdicts, anomalies, and fleet telemetry + (~15 s / ~5 min). + +The design principles that keep this robust: **every push is idempotent** (the +outbox delivers at-least-once), **pulls tolerate the full list** (cursors are +optimizations), **identities align** (the website mints the worker UUID; the +platform adopts it), and **the platform stores no provider business data** beyond +IDs, names, ASNs, and priority. + +The website team implements this additively — new Django models (`monitoring_*`), +a handful of DRF endpoints, a scoped service credential, Celery housekeeping jobs, +three permissions, admin/operator/provider dashboard pages, and a public Network +Health section. Crucially, the **platform side is already built and tested against +a stub of this exact contract** (`internal/mockadvisor`), so implementing the +website side to spec makes integration a config change on the platform. Rollout is +incremental: catalog first (read-only), then results (public status live), then +enrollment/decisions (community onboarding) — with workers enrollable via the +admin CLI until the website UI ships. + +> Full treatment: [Django integration guide](../integration/django-integration.md) +> and the [API reference](../api/README.md). + +--- + +# Part III — Running It + +# Chapter 14 — Deployment + +The supported v1 deployment is deliberately modest: **a single Ubuntu VM running +Docker Compose behind Caddy.** Every component is a container image built from +the monorepo — `coordinator`, `aggregator`, `builder`, `worker`, `migrate` — +plus PostgreSQL, an S3-compatible artifact store, and monitoring (Prometheus + +Grafana). + +Only the coordinator and artifact store are Internet-exposed; the aggregator and +builder have no inbound surface. Caddy terminates TLS and restricts the admin +surface to an allowlisted CIDR. Secrets (database DSNs per least-privilege role, +the VPS Advisor service credential, the snapshot signing key, admin tokens) live +in an env file outside the repo. The artifact store is provider-agnostic — +Backblaze B2 is the production choice, but switching to R2, S3, or MinIO is an +environment change, not a code change — and is CDN-frontable for worker downloads +at scale. + +For larger scale the same images move to Kubernetes: the coordinator as an +autoscaled Deployment, the aggregator as a single leader, the builder as a +CronJob, migrations as a pre-upgrade job, PostgreSQL managed, and the artifact +store behind a CDN. + +> Full treatment: [deployment guide](../operations/deployment.md) and +> [architecture 07](../architecture/07-deployment.md). + +# Chapter 15 — Operations + +Running VAPN day to day is a small, well-defined set of concerns: + +- **Monitoring.** Structured JSON logs everywhere; Prometheus `/metrics` on every + service; `/healthz` and `/readyz` endpoints. Key alerts: snapshot age > 2× its + cadence, growing publication-outbox depth, a drop in active workers, a rising + `insufficient_data` ratio, and signature-failure spikes. +- **Backups.** Nightly base backups + WAL archiving. The registry and aggregation + data are precious; raw measurements are re-derivable and short-lived. +- **Upgrades.** Backward-compatible migrations (expand → migrate → contract) let + coordinator replicas roll; workers tolerate brief coordinator downtime by + queuing observations locally. Worker fleets move forward via health-gated + self-update and min-version enforcement. +- **Incident response & recovery.** A global scheduler kill switch + (`vapnctl scheduler pause`), instant snapshot rollback, and worker + suspend/quarantine give operators sharp levers. Runbooks cover the common + incidents. + +The administrative CLI `vapnctl` is the operational control plane — fleet status, +worker lifecycle, snapshot rollback, scheduler kill switch, audit query — mirrored +by the VPS Advisor admin dashboard for human operators. + +> Full treatment: [Operations](../operations/README.md) — deployment, monitoring, +> backup & DR, upgrades, release management, security hardening, runbooks, and the +> launch checklist. + +# Chapter 16 — Development + +VAPN is a Go monorepo: `cmd/` holds one main package per binary, `internal/` the +implementation packages (each with a doc comment stating its responsibility and +boundary), `migrations/` one ordered SQL stream, `deploy/` the compose/prod/worker +manifests, and `docs/` this documentation. + +A single command brings up the whole loop locally — `make dev-up` starts +PostgreSQL, MinIO, a **mock VPS Advisor** serving the integration contract from +fixtures, the coordinator, the aggregator, and workers, running fully offline +against the pre-downloaded `data/`. `make check` (vet + tests + build) is the +pre-commit gate; tests run against a dedicated `vapn_test` database, never the +live one. + +Contributions must preserve the load-bearing **invariants**: VPS Advisor is the +only source of truth for providers; only aggregated consensus leaves the platform; +workers are the hostile edge; the builder is the only MRT/MaxMind reader; never +guess on ambiguity; measurement is protocol-agnostic; idempotency everywhere data +crosses a boundary; everything containerized with 12-factor config. A change that +would violate one needs a design discussion first. + +> Full treatment: [Development guide](../development/README.md) and the +> [Reference](../reference/README.md) pages (CLI, configuration, schema). + +# Chapter 17 — Troubleshooting + +Most issues fall into a few buckets, and two commands solve most worker problems: +`vapn doctor` (re-runs the environment checks with specific fixes) and `vapn logs` +(shows what the worker is actually doing). + +Common worker symptoms: stuck "awaiting approval" (a human step — check the VPS +Advisor dashboard); "unreachable" (crash, offline host, or blocked egress); clock +check failing (enable NTP); Docker permission errors (add the user to the `docker` +group); coordinator unreachable (allow outbound HTTPS 443); quarantine (usually a +wrong clock or an outdated image — fix it and trust recovers). + +Common platform-side concerns: snapshots not publishing (check the builder logs; +the failing stage is named, and the previous published snapshot stays in force); a +build held for approval (investigate the prefix-count swing before forcing it); +growing outbox depth (VPS Advisor is rejecting or slow — check for 4xx, which +signals contract drift). + +> Full treatment: [worker troubleshooting](../getting-started/troubleshooting.md), +> the [FAQ](../getting-started/faq.md), and [operations runbooks](../operations/runbooks.md). + +# Chapter 18 — Future Roadmap + +VAPN v1 is intentionally correct-and-conservative, with clean seams for growth: + +- **More probe protocols.** ICMP is the first `Prober`; TCP-connect, traceroute, + and HTTP(S) checks slot in behind the same interface and the typed metrics + schema, moving beyond pure reachability toward service-level signals. +- **Richer routing intelligence.** RPKI origin validation layered onto the + existing bogon/MOAS checks; cross-collector agreement for stronger ownership; + BGP-hijack-aware interpretation of measurements. +- **Regional depth.** Finer region bucketing and per-region confidence as the + worker fleet grows denser. +- **Scale-out seams already present.** A Redis nonce cache for the coordinator, + CDN-fronted artifact distribution, and aggregation parallelized by provider — + all noted as explicit future seams, none a v1 dependency. +- **Operational maturity.** Deeper anomaly detection, historical analytics, and + tighter VPS Advisor dashboard integration. + +The north star, from the brief: VAPN should read like a **mature, production-ready +open-source distributed Internet-observability platform** — one that could power +VPS Advisor for many years and eventually serve tens of thousands of providers and +thousands of community workers worldwide. + +--- + +## Appendix — Where to go next + +- **Every term:** [Glossary](../reference/glossary.md) +- **Every command:** [CLI reference](../reference/cli.md) +- **Every setting:** [Configuration](../reference/configuration.md) +- **Every endpoint:** [API reference](../api/README.md) +- **Every design decision:** [Architecture](../architecture/README.md) +- **The friendly on-ramp:** [Getting Started](../getting-started/README.md) + +## Exporting this handbook as a PDF + +This handbook is a single self-contained Markdown file, which makes PDF export +straightforward. For example, with [Pandoc](https://pandoc.org): + +```sh +pandoc docs/handbook/README.md -o vapn-handbook.pdf \ + --toc --toc-depth=2 --number-sections \ + -V geometry:margin=1in -V documentclass=report +``` + +Or open it in any Markdown editor with PDF export (VS Code + a Markdown PDF +extension, Typora, Obsidian). The Mermaid diagrams render in tools with Mermaid +support; for plain LaTeX PDF export, pre-render diagrams or rely on the prose, +which is written to stand on its own. diff --git a/docs/integration/README.md b/docs/integration/README.md new file mode 100644 index 0000000..f1b2307 --- /dev/null +++ b/docs/integration/README.md @@ -0,0 +1,54 @@ +# VPS Advisor Integration + +This section is for the **VPS Advisor website engineering team**. It specifies +everything the website must add so VAPN (this repository, "the platform") can +operate against it. The website already exists and is in production — nothing +here redesigns or rebuilds it; it's purely additive. + +> **The key promise:** the platform side of every flow here is already +> implemented and tested against a stub of this exact contract +> (`internal/mockadvisor`). Implement to this specification and integration on +> the platform side is a config change (`VAPN_ADVISOR_URL` + credential) — the +> same contract tests that run against our stub in CI can run against your +> staging environment. + +## Documents + +| Document | For | +|---|---| +| [**Django integration guide**](django-integration.md) | The complete, canonical implementation: models, migrations, DRF endpoints, auth, Celery tasks, admin, permissions, signals, caching, testing, rollout | +| [API Reference → VPS Advisor endpoints](../api/README.md#a-vps-advisor-endpoints) | Endpoint inventory with schemas and errors | +| [Architecture 04 — API contracts](../architecture/04-api-contracts.md) | The design record behind the three API surfaces | + +## The boundary in one picture + +```mermaid +flowchart LR + subgraph Website [VPS Advisor website — you build the additive parts] + CAT[Provider catalog + ASNs] + ENR[Worker enrollment + approval] + RES[Results storage + Network Health UI] + end + subgraph Platform [VAPN — already built] + B[Builder] --- CO[Coordinator] --- AG[Aggregator] + end + CAT -->|"pull: providers/ASNs"| Platform + ENR -->|"pull: enrollments, decisions"| Platform + Platform -->|"push: status, anomalies, telemetry"| RES +``` + +Four flows cross the boundary. The website is the **source of truth and human +surface**; the platform is the **measurement machine**. + +| Flow | Direction | Cadence | +|---|---|---| +| Provider catalog | platform **pulls** | ~2 min | +| Worker enrollment & admin decisions | platform **pulls** | ~2 min | +| Aggregated results & anomalies | platform **pushes** | ~15 s outbox drain | +| Fleet telemetry | platform **pushes** | ~5 min | + +The platform never calls user-facing pages, stores no provider business data +beyond IDs/names/ASNs/priority, and tolerates website downtime (pulls retry; +pushes queue in an idempotent outbox with backoff). + +Start with the [Django integration guide](django-integration.md). diff --git a/docs/integration/django-integration.md b/docs/integration/django-integration.md new file mode 100644 index 0000000..3af79d0 --- /dev/null +++ b/docs/integration/django-integration.md @@ -0,0 +1,655 @@ +# VPS Advisor Django Integration Guide + +**Audience:** the VPS Advisor website engineering team (Django/DRF). +**Goal:** everything the website must add so VAPN can operate against it — +models, migrations, endpoints, authentication, tasks, admin, permissions, +testing, and rollout — in enough detail that you could implement it **without +ever reading the platform's code**. + +The platform side is already built and tested against a stub of this exact +contract ([`internal/mockadvisor`](../../internal/mockadvisor/server.go)). +Implement to this document and platform-side integration is just +`VAPN_ADVISOR_URL` + a credential. + +> This guide uses Django + Django REST Framework + Celery idioms because that's +> the VPS Advisor stack. Adapt names/types to your conventions; the **contract** +> (paths, payloads, semantics) is what must match. For the design rationale +> behind the three API surfaces, see +> [architecture 04 — API contracts](../architecture/04-api-contracts.md). + +## Contents + +1. [Mental model & the four flows](#1-mental-model) +2. [Authentication](#2-authentication) +3. [Database models & migrations](#3-database-models--migrations) +4. [Endpoints](#4-endpoints) — with request/response/validation/errors +5. [Permissions](#5-permissions) +6. [Celery tasks & background jobs](#6-celery-tasks--background-jobs) +7. [Signals](#7-signals) +8. [Caching](#8-caching) +9. [Django admin & dashboard pages](#9-django-admin--dashboard-pages) +10. [Management commands](#10-management-commands) +11. [Security considerations](#11-security-considerations) +12. [Testing strategy](#12-testing-strategy) +13. [Deployment & rollout order](#13-deployment--rollout-order) +14. [Operational workflow](#14-operational-workflow) + +--- + +## 1. Mental model + +The website is the **source of truth and human surface**; the platform is the +**measurement machine**. Four flows cross the boundary: + +| # | Flow | Direction | Cadence | Endpoints | +| --- | ------------------------------- | ------------------- | -------------- | -------------------------------------------- | +| 1 | Provider catalog | platform **pulls** | ~2 min | [4.1](#41-provider-catalog-platform-pulls) | +| 2 | Enrollment | platform **pulls** | ~2 min | [4.2](#42-enrollment) | +| 3 | Admin decisions | platform **pulls** | ~2 min | [4.3](#43-admin-decisions) | +| 4 | Results / anomalies / telemetry | platform **pushes** | ~15 s / ~5 min | [4.4](#44-results-ingestion-platform-pushes) | + +Design rules to internalize: + +- **You never store measurements or run probes.** You store _inputs_ (which + providers/ASNs to monitor) and _outputs_ (verdicts to display). +- **Every push endpoint must be idempotent** — the platform delivers + at-least-once from an outbox with retries. A replayed push must be a no-op. +- **Pulls are cheap and tolerant.** Returning the full list is always correct; + `updated_since`/`since` cursors are optimizations. +- **Identities align.** The website generates the worker `uuid`; the platform + adopts it verbatim. There is one worker identity, shared. + +## 2. Authentication + +Two distinct auth surfaces: + +### 2a. Platform ↔ website (server-to-server) + +One **service credential** scoped to the `/api/v1/monitoring/*` namespace. A +static bearer token is the minimum; HMAC or mTLS if your stack prefers. + +``` +Authorization: Bearer +``` + +Implement as a DRF authentication/permission class: + +```python +# monitoring/auth.py +from rest_framework.permissions import BasePermission +from django.conf import settings +import hmac + +class IsPlatformService(BasePermission): + """Authenticates the VAPN platform via a scoped bearer token.""" + def has_permission(self, request, view): + auth = request.headers.get("Authorization", "") + if not auth.startswith("Bearer "): + return False + token = auth.removeprefix("Bearer ") + # constant-time compare against the configured platform token(s) + return any(hmac.compare_digest(token, t) + for t in settings.MONITORING_PLATFORM_TOKENS if t) +``` + +**Rotation:** keep `MONITORING_PLATFORM_TOKENS` a list — issue a second token, +update the platform config, then drop the old one. Never a hard cutover. + +> **Alert on `401`/`403` from this credential.** A sudden spike means contract +> drift or a rotation gone wrong, not an attacker (the platform calls from a +> small set of egress IPs you'll be given — allowlist them too). + +### 2b. Humans (operators, providers, admins) + +Your existing session auth, plus three new [permissions](#5-permissions). No new +auth mechanism needed for people. + +## 3. Database models & migrations + +Add a `monitoring` app. Suggested models (adapt types/names): + +```python +# monitoring/models.py +import uuid +from django.db import models +from django.conf import settings + +class WorkerState(models.TextChoices): + PENDING = "pending" + ACTIVE = "active" + SUSPENDED = "suspended" + QUARANTINED = "quarantined" + RETIRED = "retired" + +class MonitoringWorker(models.Model): + # The website generates this id; the platform adopts it verbatim. + id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False) + operator = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.CASCADE, + related_name="monitoring_workers") + name = models.CharField(max_length=200) + state = models.CharField(max_length=16, choices=WorkerState.choices, + default=WorkerState.PENDING) + state_reason = models.TextField(blank=True) + created_at = models.DateTimeField(auto_now_add=True) + updated_at = models.DateTimeField(auto_now=True) + +class MonitoringEnrollment(models.Model): + id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False) + worker = models.ForeignKey(MonitoringWorker, on_delete=models.CASCADE, + related_name="enrollments") + token_hash = models.BinaryField() # sha256 of the one-time token + expires_at = models.DateTimeField() # suggest 24–72h + registered_at = models.DateTimeField(null=True, blank=True) # set on platform ack + created_at = models.DateTimeField(auto_now_add=True) + +class MonitoringDecision(models.Model): + id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False) + worker = models.ForeignKey(MonitoringWorker, on_delete=models.CASCADE, + related_name="decisions") + state = models.CharField(max_length=16, choices=WorkerState.choices) # minus pending + reason = models.TextField() + decided_by = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.SET_NULL, null=True) + decided_at = models.DateTimeField(auto_now_add=True, db_index=True) # platform polls ?since= + +class MonitoringProviderStatus(models.Model): + provider = models.OneToOneField("catalog.Provider", on_delete=models.CASCADE, + primary_key=True, related_name="monitoring_status") + document = models.JSONField() # verbatim platform payload (4.4) + updated_at = models.DateTimeField(auto_now=True) + +class MonitoringAnomaly(models.Model): + # (provider, kind, opened_at) is the idempotency key. + provider = models.ForeignKey("catalog.Provider", on_delete=models.CASCADE, + related_name="monitoring_anomalies") + kind = models.CharField(max_length=32) # reachability_loss|latency_regression|routing_churn + severity = models.CharField(max_length=16) + document = models.JSONField() + opened_at = models.DateTimeField() + resolved_at = models.DateTimeField(null=True, blank=True) + class Meta: + constraints = [models.UniqueConstraint( + fields=["provider", "kind", "opened_at"], name="uniq_anomaly")] + +class MonitoringFleetTelemetry(models.Model): + document = models.JSONField() + updated_at = models.DateTimeField(auto_now=True) # a single row is fine +``` + +### Provider model additions + +On your existing `Provider` (and a join table for ASNs): + +```python +class Provider(models.Model): + # ... existing fields ... + monitoring_enabled = models.BooleanField(default=False) # the opt-out toggle + monitoring_priority = models.IntegerField(default=100) # lower = probed more often + +class ProviderASN(models.Model): + provider = models.ForeignKey(Provider, on_delete=models.CASCADE, related_name="asns") + asn = models.BigIntegerField(unique=True) # ← CRITICAL: one ASN, one provider +``` + +> **The `asn` uniqueness constraint is not optional.** The platform hard-errors +> on an ASN claimed by two providers rather than guessing — [it will not split +> or duplicate measurements](../concepts/prefix-ownership.md#complication-3-moas-conflicts). +> Enforce it at the database level. + +Then `python manage.py makemigrations monitoring && migrate`. + +## 4. Endpoints + +All JSON; errors as **RFC 7807** `application/problem+json`; paginate with +`next_cursor` (the platform follows it when non-null). Base path +`/api/v1/monitoring/`. Wire them up: + +```python +# monitoring/urls.py +from django.urls import path +from . import views +urlpatterns = [ + path("providers", views.ProviderList.as_view()), + path("providers/", views.ProviderDetail.as_view()), + path("asns", views.ASNList.as_view()), + path("workers", views.WorkerList.as_view()), + path("workers//regenerate-token",views.RegenerateToken.as_view()), + path("enrollments/pending", views.PendingEnrollments.as_view()), + path("enrollments//registered", views.EnrollmentRegistered.as_view()), + path("admin/decisions", views.DecisionList.as_view()), + path("results/providers/", views.ResultUpsert.as_view()), + path("results/anomalies", views.AnomalyIngest.as_view()), + path("results/history", views.HistoryIngest.as_view()), + path("telemetry/fleet", views.FleetTelemetry.as_view()), +] +``` + +### 4.1 Provider catalog (platform pulls) + +**`GET /api/v1/monitoring/providers?enabled=true&updated_since=&cursor=`** + +- **Purpose:** the list of providers eligible for monitoring, with their ASNs. +- **Auth:** platform service token. +- **Query:** `enabled=true` filters to `monitoring_enabled`; `updated_since` + is an optimization; `cursor` for pagination. +- **Response 200:** + +```json +{ + "providers": [ + { + "provider_id": "7f9c...", + "name": "ExampleHost", + "asns": [64500, 64501], + "monitoring_enabled": true, + "priority": 10, + "updated_at": "2026-07-18T06:00:00Z" + } + ], + "next_cursor": null +} +``` + +- **Semantics:** a provider **absent** from the enabled list is treated as + opted-out/delisted and drained platform-side. Returning the full list is + always correct. +- **Errors:** `401` bad token. + +```python +# monitoring/views.py (illustrative) +class ProviderList(APIView): + permission_classes = [IsPlatformService] + def get(self, request): + qs = Provider.objects.prefetch_related("asns") + if request.query_params.get("enabled") == "true": + qs = qs.filter(monitoring_enabled=True) + if since := request.query_params.get("updated_since"): + qs = qs.filter(updated_at__gt=parse_datetime(since)) + page, next_cursor = cursor_paginate(qs.order_by("id"), request) + return Response({"providers": [serialize_provider(p) for p in page], + "next_cursor": next_cursor}) +``` + +**`GET /api/v1/monitoring/providers/{id}`** — the same object; `404` if unknown. + +**`GET /api/v1/monitoring/asns?updated_since=`** — flat mapping for cheap delta +sync (optional but recommended): + +```json +{ "asns": [{ "asn": 64500, "provider_id": "7f9c...", "monitoring_enabled": true, "updated_at": "..." }], "next_cursor": null } +``` + +### 4.2 Enrollment + +**Operator UI flow** (session auth + `monitoring.operator`): + +**`POST /api/v1/monitoring/workers`** — create a worker. + +- Generate a random **≥32-byte token**, show its plaintext **once**, store only + `sha256`. Create `MonitoringWorker` (state `pending`) + `MonitoringEnrollment`. +- **Response 201:** `{ "worker_id": "...", "enrollment_token": "", "coordinator_url": "https://probe-api..." }` + +**`POST /api/v1/monitoring/workers/{id}/regenerate-token`** — replace an +unused/expired token (same one-time display). + +**`GET /api/v1/monitoring/workers`** — the operator's own workers + state (join +in liveness from pushed telemetry/status). + +**Platform pull** (service token): + +**`GET /api/v1/monitoring/enrollments/pending`** — enrollments not yet +`registered_at` and not expired: + +```json +{ + "enrollments": [ + { + "enrollment_id": "en-123", + "worker_id": "9f30...", + "worker_name": "helsinki-1", + "operator_id": "user-uuid", + "token_hash": "hex sha256", + "expires_at": "2026-07-19T08:00:00Z" + } + ], + "next_cursor": null +} +``` + +**`POST /api/v1/monitoring/enrollments/{id}/registered`** — the platform has +provisioned the worker; set `registered_at`. **`204`. Idempotent** (a second +call is a no-op). + +### 4.3 Admin decisions + +Admin UI actions (approve / suspend / quarantine / retire, each with a +**required reason**) create `MonitoringDecision` rows. The platform polls: + +**`GET /api/v1/monitoring/admin/decisions?since=`** + +- Return decisions with `decided_at > since`, **oldest first**. +- **Response 200:** + +```json +{ + "decisions": [ + { + "decision_id": "d-1", + "worker_id": "9f30...", + "state": "active", + "reason": "verified operator", + "decided_at": "2026-07-18T09:00:00Z" + } + ], + "next_cursor": null +} +``` + +- The platform applies decisions **idempotently** (replays/no-ops safe) within + ~2 min. Lifecycle semantics to reflect in your UI: + [pending](../worker/lifecycle.md#pending) heartbeats but gets no work; + [suspended](../worker/lifecycle.md#suspended) is locked out; + [quarantined](../worker/lifecycle.md#quarantined-shadow-mode) measures at zero + weight; [retired](../worker/lifecycle.md#retired) is terminal. + +> **Optional webhook:** POST the same decision object to the platform to cut +> latency. Polling stays the fallback, so the webhook is a pure optimization — +> you never _depend_ on it. + +### 4.4 Results ingestion (platform pushes) + +All idempotent; respond `2xx` quickly (`202` fine). Non-2xx is retried with +backoff — transient `5xx` is harmless, persistent failure backs up the queue. + +**`PUT /api/v1/monitoring/results/providers/{provider_id}`** — upsert current +status: + +```json +{ "provider_id": "7f9c...", "as_of": "2026-07-18T08:05:00Z", "global": { "verdict": "healthy", "confidence": 0.97, "metrics": { "loss_rate": 0.001, "rtt_p50_ms": 21.4, "rtt_p95_ms": 38.2, "worker_count": 14, "dissent_ratio": 0.02 } }, "regions": [{ "region": "eu-west", "verdict": "healthy", "confidence": 0.99 }] } +``` + +- **Verdicts:** `healthy | degraded | outage | insufficient_data`. +- **Display guidance (important):** these are _public-network reachability_ + signals, **not SLA claims**. Always render `insufficient_data` as "not enough + data," never as an outage. Show confidence. **Accept and store unknown + fields** (regional breakdowns and future metrics arrive here). +- **Idempotent upsert** keyed by `provider_id`. Store the document verbatim. + +**`POST /api/v1/monitoring/results/anomalies`** — anomaly documents +(`kind: reachability_loss | latency_regression | routing_churn`, `severity`, +`opened_at`, nullable `resolved_at`, `evidence`). Idempotency key +`(provider, kind, opened_at)`. Drives "recent instability" UI + notifications. + +**`POST /api/v1/monitoring/results/history`** — periodic rollup batches for +charts (optional at launch; accept-and-store, payloads up to ~1 MB). + +**`POST /api/v1/monitoring/telemetry/fleet`** — admin-dashboard summary: + +```json +{ "as_of": "...", "workers_by_state": { "active": 41, "pending": 3 }, "software_versions": { "1.2.0": 39, "1.1.2": 2 }, "published_snapshot": "20260718T0800Z-1", "security_events_24h": { "replay": 0, "bad_signature": 2 } } +``` + +```python +class ResultUpsert(APIView): + permission_classes = [IsPlatformService] + def put(self, request, pk): + provider = get_object_or_404(Provider, pk=pk) + MonitoringProviderStatus.objects.update_or_create( + provider=provider, defaults={"document": request.data}) + return Response(status=202) # fast ack; render from stored doc later +``` + +## 5. Permissions + +Three new permissions (map to your groups/roles): + +| Permission | Grants | Typical holder | +| --------------------- | ------------------------------------------ | ------------------------------------------ | +| `monitoring.operator` | create/manage own workers | any eligible registered user (your policy) | +| `monitoring.provider` | the `monitoring_enabled` opt-out toggle | existing provider-claim role | +| `monitoring.admin` | approval queue, decisions, fleet dashboard | staff | + +```python +class HasMonitoringAdmin(BasePermission): + def has_permission(self, request, view): + return request.user.has_perm("monitoring.admin") +``` + +## 6. Celery tasks & background jobs + +```python +# monitoring/tasks.py +from celery import shared_task +from django.utils import timezone +from datetime import timedelta + +@shared_task +def expire_stale_enrollments(): + """Hourly: mark expired, unregistered enrollments unusable.""" + MonitoringEnrollment.objects.filter( + registered_at__isnull=True, expires_at__lt=timezone.now() + ).delete() + +@shared_task +def prune_monitoring_history(): + """Daily: trim old decisions/telemetry/history rows.""" + cutoff = timezone.now() - timedelta(days=90) + MonitoringDecision.objects.filter(decided_at__lt=cutoff).delete() + +@shared_task +def notify_worker_state_change(worker_id, new_state, reason): + """Fired from the decision signal (7) — email/notify the operator.""" + ... +``` + +Schedule with Celery Beat: + +```python +CELERY_BEAT_SCHEDULE = { + "expire-stale-enrollments": {"task": "monitoring.tasks.expire_stale_enrollments", + "schedule": crontab(minute=0)}, + "prune-monitoring-history": {"task": "monitoring.tasks.prune_monitoring_history", + "schedule": crontab(hour=3, minute=0)}, +} +``` + +> **You do not need a task to _push_ to the platform** — the platform pulls the +> catalog/enrollments/decisions itself. Your jobs are just housekeeping and +> notifications. + +## 7. Signals + +Use signals to react to admin decisions and status changes without coupling +views to notifications: + +```python +# monitoring/signals.py +from django.db.models.signals import post_save +from django.dispatch import receiver +from .models import MonitoringDecision, MonitoringAnomaly +from .tasks import notify_worker_state_change + +@receiver(post_save, sender=MonitoringDecision) +def on_decision(sender, instance, created, **kwargs): + if created: + instance.worker.state = instance.state + instance.worker.state_reason = instance.reason + instance.worker.save(update_fields=["state", "state_reason", "updated_at"]) + notify_worker_state_change.delay(str(instance.worker_id), + instance.state, instance.reason) + +@receiver(post_save, sender=MonitoringAnomaly) +def on_anomaly(sender, instance, created, **kwargs): + if created and instance.resolved_at is None: + # notify the provider (respect their opt-out), surface on admin feed + ... +``` + +## 8. Caching + +- **Provider catalog / ASN endpoints:** cache the serialized response briefly + (30–60 s). The platform polls every ~2 min and tolerates slight staleness, so + a short cache absorbs the load with no correctness cost. Invalidate on + provider/ASN save. +- **Public Network Health section:** cache the rendered status card per provider + (e.g. 60 s); it's read far more than it's written (~1 update/5 min/provider). +- **Do not cache** enrollment/decision pulls (they're low-volume and must be + fresh) or result upserts (writes). + +```python +from django.core.cache import cache +def enabled_providers_payload(): + key = "monitoring:providers:enabled" + if (cached := cache.get(key)) is None: + cached = build_enabled_providers_payload() + cache.set(key, cached, timeout=45) + return cached +``` + +## 9. Django admin & dashboard pages + +### Django admin (staff) + +```python +# monitoring/admin.py +from django.contrib import admin +from .models import MonitoringWorker, MonitoringDecision, MonitoringAnomaly + +@admin.register(MonitoringWorker) +class WorkerAdmin(admin.ModelAdmin): + list_display = ("id", "name", "operator", "state", "updated_at") + list_filter = ("state",) + search_fields = ("id", "name", "operator__username") + actions = ["approve", "suspend", "quarantine", "retire"] + # each action creates a MonitoringDecision(reason=...) so the platform syncs it +``` + +### Dashboard pages to add + +| Audience | Page | Contents | +| --------------------------- | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------- | +| **Operator** ("My Workers") | worker list | state/liveness, **create worker + one-time token (copy-once UX)**, regenerate token, retire own worker (writes a decision) | +| **Provider manager** | monitoring toggle | `monitoring_enabled` opt-out (copy: "takes effect within minutes, no commitment") + their status card | +| **Admin** | approval queue | pending workers with operator context; **decision actions with required reason** | +| **Admin** | fleet overview | from `MonitoringFleetTelemetry`: counts by state/version, snapshot in force, security events | +| **Admin** | worker detail | state history (from decisions), trust/security surfaced via telemetry, anomaly feed, audit trail | +| **Public** | Network Health section | rendered from `MonitoringProviderStatus` + recent anomalies, with the [display guidance](#44-results-ingestion-platform-pushes) | + +## 10. Management commands + +Handy operational commands to ship: + +```python +# monitoring/management/commands/monitoring_seed_token.py +class Command(BaseCommand): + """Issue an enrollment token from the CLI (bootstrap before the UI ships).""" + def handle(self, *args, **opts): + worker = MonitoringWorker.objects.create(operator=..., name=opts["name"]) + token = secrets.token_urlsafe(32) + MonitoringEnrollment.objects.create( + worker=worker, token_hash=hashlib.sha256(token.encode()).digest(), + expires_at=timezone.now() + timedelta(days=2)) + self.stdout.write(token) # show once +``` + +Others worth having: `monitoring_expire_enrollments` (manual trigger of the +task), `monitoring_fleet` (print the latest telemetry), `monitoring_check` +(verify the platform credential works end-to-end against staging). + +## 11. Security considerations + +- **Scope the service token** to `/api/v1/monitoring/*` only; allowlist the + platform's egress IPs (provided). +- **Never store plaintext enrollment tokens** — only `sha256`. Show plaintext + once, at creation. +- **Constant-time token comparison** (`hmac.compare_digest`). +- **Rate-limit** operator worker-creation to blunt token farming. +- **Require a reason on every decision** (audit trail) and keep decisions + append-only. +- **Enforce ASN uniqueness at the DB layer** ([3](#3-database-models--migrations)). +- **Alert on `4xx` from the platform credential** — it signals contract drift. +- **Validate result payloads loosely but store verbatim** — accept unknown + fields (forward compatibility), but sanity-check `verdict`/`kind` enums before + rendering to the public. + +Cross-reference the platform's [security model](../architecture/05-security-trust-model.md). + +## 12. Testing strategy + +- **Contract tests** are the centerpiece. The platform runs the same suite + against its stub and against your staging environment — mirror them: for each + endpoint, assert path, auth behavior (`401` without token), response shape, + idempotency (replay a push → no dupes), and pagination. +- **Idempotency tests:** `PUT results` twice → one row; `POST anomalies` with + same `(provider, kind, opened_at)` twice → one row; `POST enrollments/{id}/ +registered` twice → `204` both times, one state change. +- **Pagination tests:** more than one page → `next_cursor` set, following it + returns the rest, no overlaps/gaps. +- **Permission tests:** operator can't hit admin decisions; provider role can + toggle only their own provider. +- **Fixture parity:** seed the same provider/worker fixtures the platform's + `mockadvisor` uses so behavior matches. + +```python +def test_results_upsert_is_idempotent(client, provider, platform_token): + doc = {"provider_id": str(provider.id), "as_of": "...","global": {...}} + for _ in range(2): + r = client.put(f"/api/v1/monitoring/results/providers/{provider.id}", + doc, content_type="application/json", + HTTP_AUTHORIZATION=f"Bearer {platform_token}") + assert r.status_code == 202 + assert MonitoringProviderStatus.objects.filter(provider=provider).count() == 1 +``` + +## 13. Deployment & rollout order + +Ship incrementally — each step is independently useful and de-risks the next: + +1. **Models + 4.1 (catalog).** The platform can now run against production data + **read-only** — it learns which providers/ASNs to monitor and starts building + snapshots. No public change yet. +2. **4.4 (results).** Store and render Network Health → public status goes live. +3. **4.2 + 4.3 (enrollment + decisions).** Community workers onboard through + the website. _Until this ships,_ workers are enrolled via the platform admin + API (`vapnctl workers create`), so you're never blocked. +4. **Staging + service credential.** Provide a staging environment and token; + the platform runs its contract tests against it (same suite as CI). + +Provide the platform team: `VAPN_ADVISOR_URL`, the service token, and confirm +your egress-IP allowlist. + +## 14. Operational workflow + +Day-to-day once live: + +```mermaid +sequenceDiagram + participant Op as Operator + participant Site as VPS Advisor (you) + participant Plat as VAPN platform + Op->>Site: Create worker → one-time token + Plat->>Site: GET enrollments/pending (poll) + Plat->>Site: POST enrollments/{id}/registered + Note over Site: Operator sees "awaiting approval" + Admin->>Site: Approve (with reason) → MonitoringDecision + Plat->>Site: GET admin/decisions?since= (poll) → worker active + loop every few minutes + Plat->>Site: PUT results/providers/{id} (verdicts) + Plat->>Site: POST telemetry/fleet + end + Site-->>Op: Network Health on provider page + fleet dashboard +``` + +- **A provider opts in/out:** toggle `monitoring_enabled`; effective platform- + side within ~2 min. +- **An operator adds a worker:** create → token → it enrolls → admin approves → + it's measuring. Notifications keep the operator informed. +- **An anomaly opens:** the platform pushes it; you notify the provider (respect + opt-out) and show "recent instability." +- **Something's wrong:** watch for `4xx` from the platform credential (contract + drift) and stale `updated_at` on statuses (platform not pushing — check the + platform's [outbox depth](../operations/monitoring.md)). + +That's the whole integration. Questions on any endpoint's exact bytes: the +[API Reference](../api/README.md) has the canonical schemas, and +[`internal/mockadvisor`](../../internal/mockadvisor/server.go) is a runnable +reference implementation of this contract. diff --git a/docs/integration/vpsadvisor-integration-guide.md b/docs/integration/vpsadvisor-integration-guide.md deleted file mode 100644 index 0fcef2e..0000000 --- a/docs/integration/vpsadvisor-integration-guide.md +++ /dev/null @@ -1,264 +0,0 @@ -# VPS Advisor Integration Guide - -**Audience:** the VPS Advisor website engineering team. -**Goal:** everything the website must add so the Community Network Intelligence -Platform (this repository, "the platform") can operate against it. The -platform side of every flow described here is already implemented and tested -against a stub of this exact contract (`internal/mockadvisor`) — implement to -this document and integration is a config change on our side -(`VAPN_ADVISOR_URL` + credential). - -Related reading: [architecture/04-api-contracts.md](../architecture/04-api-contracts.md) -(contract overview), [architecture/01-system-architecture.md](../architecture/01-system-architecture.md) -(who calls whom and why). - ---- - -## 1. Integration overview - -The website is the **source of truth and human surface**; the platform is the -**measurement machine**. Four flows cross the boundary: - -| Flow | Direction | Cadence | Endpoints | -|---|---|---|---| -| Provider catalog | platform pulls | every ~2 min | §4.1 | -| Worker enrollment & admin decisions | platform pulls | every ~2 min | §4.2, §4.3 | -| Aggregated results & anomalies | platform pushes | ~15 s drain of an outbox | §4.4 | -| Fleet telemetry | platform pushes | every ~5 min | §4.4 | - -The platform never calls user-facing pages, never stores provider business -data beyond IDs/names/ASNs/priority, and tolerates website downtime: pulls -retry, pushes queue in an outbox with exponential backoff (at-least-once — -all push endpoints must be **idempotent**, see per-endpoint notes). - -## 2. Authentication - -One **service credential** for the platform (server-to-server): a static -bearer token at minimum (`Authorization: Bearer `), HMAC or mTLS if -your stack prefers. Scope it to the `/api/v1/monitoring/*` namespace only. -Rotate by issuing a second token, updating the platform config, then revoking -the old one. All human-facing pages use your existing session auth plus the -new permissions in §6. - -## 3. New database models - -Suggested shapes (adapt names/types to your conventions): - -``` -monitoring_worker - id uuid pk -- generated by the website; the platform - -- adopts this ID verbatim (identities align) - operator_id fk → user - name text - state enum: pending | active | suspended | quarantined | retired - state_reason text null - created_at, updated_at - -monitoring_enrollment - id uuid pk - worker_id fk → monitoring_worker - token_hash bytea -- sha256 of the one-time token; plaintext is - -- shown to the operator once, never stored - expires_at timestamptz -- suggest 24–72 h - registered_at timestamptz null -- set when the platform acks (§4.2) - -monitoring_decision - id uuid pk - worker_id fk → monitoring_worker - state enum (as above, minus pending) - reason text - decided_by fk → user - decided_at timestamptz -- the platform polls ?since= on this - -monitoring_provider_status -- current status per provider (upserted) - provider_id fk → provider, pk - document jsonb -- verbatim platform payload (§4.4) - updated_at - -monitoring_anomaly - id uuid pk (or platform-supplied key for idempotent upsert) - provider_id fk → provider - document jsonb - opened_at, resolved_at - -monitoring_history -- optional rollups for charts (§4.4 history) -monitoring_fleet_telemetry -- latest fleet document (single row is fine) -``` - -Provider model addition: `monitoring_enabled boolean` (the opt-out toggle — -exposed on the provider-manager dashboard, effective within ~2 min platform- -side), `monitoring_priority int` (default 100; lower = probed more often), -and one-or-more ASNs per provider (`provider_asn(provider_id, asn)` with a -**uniqueness constraint on asn** — one ASN can only belong to one provider; -the platform hard-errors on conflicts rather than guessing). - -## 4. Endpoints to implement - -All JSON; errors as RFC 7807 `application/problem+json`; paginate with -`next_cursor` (the platform follows it when non-null). - -### 4.1 Provider catalog (platform pulls) - -`GET /api/v1/monitoring/providers?enabled=true&updated_since=` - -```json -{ "providers": [ { - "provider_id": "7f9c…", "name": "ExampleHost", - "asns": [64500, 64501], - "monitoring_enabled": true, "priority": 10, - "updated_at": "2026-07-18T06:00:00Z" } ], - "next_cursor": null } -``` - -`GET /api/v1/monitoring/providers/{id}` — same object, 404 if unknown. -`GET /api/v1/monitoring/asns` — flat `{asn, provider_id, monitoring_enabled, -updated_at}` list (cheap delta sync; optional but recommended). - -Notes: `enabled=true` filters to `monitoring_enabled`. A provider absent from -the enabled list is treated as opted-out/delisted and drained. `updated_since` -is an optimization — returning the full list is always correct. - -### 4.2 Enrollment (operator UI + platform pull) - -Operator flow on the website: create worker → generate a random ≥32-byte -token → show it **once** → store only its sha256. - -- `POST /api/v1/monitoring/workers` (operator session) → creates - `monitoring_worker` (state `pending`) + `monitoring_enrollment`; responds - with the plaintext token for display. -- `POST /api/v1/monitoring/workers/{id}/regenerate-token` — replaces an - unused/expired token. -- `GET /api/v1/monitoring/workers` (operator session) — own workers + state - (join in the fleet telemetry/status the platform pushes for liveness). - -Platform pull: - -`GET /api/v1/monitoring/enrollments/pending` — enrollments not yet -`registered_at`, not expired: - -```json -{ "enrollments": [ { - "enrollment_id": "en-123", - "worker_id": "9f30…", - "worker_name": "helsinki-1", - "operator_id": "user-uuid", - "token_hash": "hex sha256", - "expires_at": "2026-07-19T08:00:00Z" } ], - "next_cursor": null } -``` - -`POST /api/v1/monitoring/enrollments/{id}/registered` → set `registered_at` -(the platform has provisioned the worker; the operator's container can now -redeem the token). 204. Idempotent. - -### 4.3 Admin decisions (admin UI + platform pull) - -Admin UI actions (approve / suspend / quarantine / retire, with a required -reason) write `monitoring_decision` rows. The platform polls: - -`GET /api/v1/monitoring/admin/decisions?since=` - -```json -{ "decisions": [ { - "decision_id": "d-1", "worker_id": "9f30…", - "state": "active", "reason": "verified operator", - "decided_at": "2026-07-18T09:00:00Z" } ], - "next_cursor": null } -``` - -Return decisions with `decided_at > since`, oldest first. The platform -applies them idempotently (replays and no-ops are safe) within ~2 min. -Lifecycle semantics the UI should reflect: `pending` workers heartbeat but -receive no work; `suspended` are locked out entirely; `quarantined` measure -in shadow mode with zero public weight; `retired` is terminal. - -Optional webhook: POST the same decision object to the platform to cut -latency; polling remains the fallback, so this is purely an optimization. - -### 4.4 Results ingestion (platform pushes) - -`PUT /api/v1/monitoring/results/providers/{provider_id}` — **idempotent -upsert** of the current status document: - -```json -{ "provider_id": "7f9c…", "as_of": "2026-07-18T08:05:00Z", - "global": { "verdict": "healthy", "confidence": 0.97, - "metrics": { "loss_rate": 0.001, "rtt_p50_ms": 21.4, "rtt_p95_ms": 38.2, - "worker_count": 14, "dissent_ratio": 0.02 } } } -``` - -Verdicts: `healthy | degraded | outage | insufficient_data`. **Display -guidance (important):** these are *public-network reachability* verdicts, not -SLA claims; always render `insufficient_data` as "not enough data", never as -an outage; show confidence. Regional breakdowns will be added to the same -document under `"regions"` — accept and store unknown fields. - -`POST /api/v1/monitoring/results/anomalies` — anomaly documents -(`kind: reachability_loss | latency_regression | routing_churn`, `severity`, -`opened_at`, `resolved_at` nullable, `evidence` object). Drives "recent -instability" UI and notifications. Store by (provider, kind, opened_at) to -stay idempotent. - -`POST /api/v1/monitoring/results/history` — periodic rollup batches for -historical charts (optional at launch; accept-and-store). - -`POST /api/v1/monitoring/telemetry/fleet` — fleet summary for the admin -dashboard: - -```json -{ "as_of": "…", "workers_by_state": {"active": 41, "pending": 3}, - "software_versions": {"1.2.0": 39, "1.1.2": 2}, - "published_snapshot": "20260718T0800Z-...", - "security_events_24h": {"replay": 0, "bad_signature": 2} } -``` - -All pushes: respond 2xx quickly (202 is fine); the platform retries non-2xx -with exponential backoff, so transient 5xx are harmless but persistent -failures back up the queue. - -## 5. Dashboard pages - -**Operator ("My Workers"):** list own workers with state/liveness, create -worker + one-time token display (copy-once UX), regenerate token, retire own -worker (writes a decision). - -**Provider manager:** the `monitoring_enabled` opt-out toggle (copy: takes -effect within minutes, no commitment) + their own status card. - -**Admin:** approval queue (pending workers with operator context), fleet -overview (from telemetry), worker detail (state history from decisions, -trust/security events surfaced via telemetry), decision actions with reason, -anomaly feed, audit trail of decisions. - -**Public provider pages:** the Network Health section rendered from -`monitoring_provider_status` + recent anomalies, with the display guidance -from §4.4. - -## 6. Permissions - -- `monitoring.operator` — create/manage own workers (any registered user you - deem eligible, or gated by an application flow — your policy). -- `monitoring.provider` — the opt-out toggle (existing provider-claim role). -- `monitoring.admin` — approval queue, decisions, fleet dashboard. - -## 7. Background jobs & notifications - -Jobs: expire stale enrollments (hourly); prune old decisions/telemetry -(daily). Notifications (recommended): operator — worker approved / suspended -/ went silent; provider — anomaly opened on their network (respect opt-out); -admin — pending approvals digest, security-event spikes. - -## 8. Deployment & rollout - -1. Ship models + §4.1 (catalog) first — the platform can then run against - production data read-only. -2. Ship §4.4 (results) — public status goes live. -3. Ship §4.2/§4.3 (enrollment + decisions) — community workers onboard via - the website; until then workers are enrolled via the platform admin API. -4. Provide a staging environment + service credential; we run the same - contract tests against it that run against our stub in CI. - -Operational notes: all platform calls originate from a small set of egress -IPs (we'll supply them); rate is low (a few requests/min plus pushes); -payloads are small except history batches (<1 MB). Log and alert on 4xx from -the platform credential — it indicates contract drift. diff --git a/docs/operations/README.md b/docs/operations/README.md index 8d137c2..26f1b01 100644 --- a/docs/operations/README.md +++ b/docs/operations/README.md @@ -1,8 +1,14 @@ # VAPN Operations +> Part of the [VAPN documentation](../README.md). New to the system? Start with +> [Architecture](../architecture/01-system-architecture.md) and the +> [end-to-end walkthrough](../walkthroughs/end-to-end.md). + Operator documentation for running the platform side of VAPN (coordinator, aggregator, builder, database, edge). For the community worker experience see -[docs/worker/](../worker/); for architecture see [docs/architecture/](../architecture/). +[Community Workers](../worker/README.md); for the routing builder see +[the builder guide](../builder/README.md); for architecture see +[docs/architecture/](../architecture/README.md). | Document | Covers | |---|---| diff --git a/docs/operations/runbooks.md b/docs/operations/runbooks.md index d6488b0..a44433b 100644 --- a/docs/operations/runbooks.md +++ b/docs/operations/runbooks.md @@ -55,7 +55,7 @@ dropped. 3. 5xx/timeouts → website-side incident; the queue drains itself once it recovers. Only act if growth threatens disk (it won't for days). 4. Persistent 4xx → contract drift; compare payloads against - docs/integration/vpsadvisor-integration-guide.md §4.4 and escalate to the + docs/integration/django-integration.md §4.4 and escalate to the website team. 4xx rows retry forever — after resolution they drain automatically. diff --git a/docs/reference/README.md b/docs/reference/README.md new file mode 100644 index 0000000..099ddf4 --- /dev/null +++ b/docs/reference/README.md @@ -0,0 +1,22 @@ +# Reference + +Look-up material: exact commands, configuration variables, database shapes, and +definitions. These pages are terse and complete — for teaching and rationale, +follow their links back into [Concepts](../concepts/README.md), +[Architecture](../architecture/README.md), and the +[Walkthroughs](../walkthroughs/README.md). + +| Page | Contents | +|---|---| +| [CLI reference](cli.md) | Every `vapn` (worker) and `vapnctl` (admin) command | +| [Configuration & environment variables](configuration.md) | The config model and every `VAPN_` variable, per service | +| [Database schema](database-schema.md) | All six PostgreSQL schemas, table by table | +| [Glossary](glossary.md) | Every networking and project term, in plain language | + +Related references elsewhere: + +- [API Reference](../api/README.md) — all HTTP endpoints and schemas. +- [Architecture 02 — Domain model](../architecture/02-domain-model.md) — + entities, ownership, and invariants. +- [Architecture 03 — Database design](../architecture/03-database-design.md) — + the DDL these pages summarize. diff --git a/docs/reference/cli.md b/docs/reference/cli.md new file mode 100644 index 0000000..9cfb308 --- /dev/null +++ b/docs/reference/cli.md @@ -0,0 +1,96 @@ +# CLI Reference + +Two command-line tools ship with VAPN: + +- **`vapn`** — the **worker operator** CLI (community-facing). Wraps Docker so + operators never touch containers directly. Full narrative: + [worker command reference](../worker/command-reference.md). +- **`vapnctl`** — the **platform administration** CLI. Drives the + [platform admin API](../api/README.md#c-platform-admin-api). + +Other binaries in `cmd/` are services or tools, not interactive CLIs: +`coordinator`, `aggregator`, `builder`, `worker`, `migrate`, `mockadvisor`, +`keygen`, `loadtest` — see [configuration](configuration.md) for how they're +run. + +--- + +## `vapn` (worker operator) + +``` +vapn +``` + +| Command | Description | +|---|---| +| `install` | Set up and start a worker (interactive) | +| `status` | Worker health at a glance | +| `doctor` | Run the system checks | +| `logs [-f]` | Worker logs (`-f` to follow) | +| `pause` | Stop probing (keeps identity; resume any time) | +| `resume` | Start probing again | +| `update [--auto]` | Update to the latest worker image (health-gated, auto-rollback) | +| `backup` | Archive worker identity/config to a `tar.gz` | +| `unregister` | Permanently retire this worker with the coordinator | +| `uninstall` | Remove everything (offers unregister + backup) | +| `version` | Print the CLI version | + +Home directory: `~/.vapn` (override with `VAPN_HOME`). Per-command detail and +recipes: [worker command reference](../worker/command-reference.md). + +--- + +## `vapnctl` (platform administration) + +``` +vapnctl [--url URL] [--token TOKEN] [args] +``` + +Reads `VAPN_COORDINATOR_URL` and `VAPN_ADMIN_TOKEN` from the environment if the +flags are omitted. The token is the platform [admin token](../api/README.md#c-platform-admin-api); +keep this CLI's access network-restricted. + +| Command | Description | +|---|---| +| `status` | Fleet overview | +| `workers list` | List workers | +| `workers show ` | Worker detail (state, trust, leases, events) | +| `workers create --name N` | Create a worker; prints a one-time enrollment token | +| `workers approve --reason R` | Transition to `active` | +| `workers suspend --reason R` | Transition to `suspended` | +| `workers quarantine --reason R` | Transition to `quarantined` (shadow mode) | +| `workers retire --reason R` | Transition to `retired` (terminal) | +| `workers rotate-key ` | Demand a key rotation at the worker's next heartbeat | +| `snapshots list` | List routing snapshots (version, status, counts) | +| `snapshots rollback ` | Re-publish a previous snapshot | +| `scheduler pause` | Global assignment kill switch — stop issuing work | +| `scheduler resume` | Resume issuing assignments | +| `audit [--category C] [--since RFC3339] [--limit N]` | Query the append-only audit log | + +### Common admin recipes + +```sh +# Enroll a worker before the website enrollment UI exists (rollout step 3) +vapnctl workers create --name helsinki-1 # prints one-time token + +# Approve a pending worker +vapnctl workers approve 9f30… --reason "verified operator" + +# Investigate a worker +vapnctl workers show 9f30… # state, trust, leases, events + +# Emergency: stop all probing fleet-wide +vapnctl scheduler pause # …later… vapnctl scheduler resume + +# Roll back a bad snapshot +vapnctl snapshots list +vapnctl snapshots rollback 20260717T0800Z-1 # audited + +# Review recent security events +vapnctl audit --category security --since 2026-07-18T00:00:00Z --limit 100 +``` + +These map to the [platform admin API](../api/README.md#c-platform-admin-api) and +mirror what the VPS Advisor admin dashboard drives via the +[decisions sync](../integration/django-integration.md#43-admin-decisions). Every +state-changing action is written to the [audit log](database-schema.md#schema-audit). diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md new file mode 100644 index 0000000..c7763ac --- /dev/null +++ b/docs/reference/configuration.md @@ -0,0 +1,143 @@ +# Configuration & Environment Variables + +Every VAPN service and CLI is configured through **`VAPN_`-prefixed environment +variables** (12-factor). There are no config files to hand-edit for the +services; the worker keeps its two settings in `~/.vapn/config.env`, written by +`vapn install`. + +## The configuration model + +Config is loaded through `internal/platform/config`, which gives every service +three guarantees: + +1. **Typed access with defaults** — `String`, `Int`, `Bool`, `Duration`, and + `Require` (mandatory). Bad values (a non-integer where an int is expected) + are collected and reported together at boot. +2. **Fail-fast on missing required keys** — a service refuses to start and lists + *every* missing/invalid variable at once, rather than crashing on the first. +3. **Redacted effective-config dump** — at startup each service logs its full + effective configuration with secret-like keys (anything containing `TOKEN`, + `SECRET`, `KEY`, `PASSWORD`, `DSN`, `CREDENTIAL`) shown as `[redacted]`. This + is the fastest way to confirm what a container actually received. + +``` +INFO effective configuration VAPN_DB_DSN=[redacted] VAPN_HTTP_ADDR=:8080 … +``` + +## Shared variables + +Used by multiple services: + +| Variable | Type | Default | Meaning | +|---|---|---|---| +| `VAPN_DB_DSN` | string (req.) | — | PostgreSQL connection string (per-service least-privilege role) | +| `VAPN_DB_WAIT` | duration | `60s` | How long to wait for the DB to become reachable at boot | +| `VAPN_HTTP_ADDR` | string | varies | Listen address (`:8080` coordinator, `:8082` aggregator) | +| `VAPN_LOG_LEVEL` | string | `info` | slog level | +| `VAPN_ADVISOR_URL` | string | — | VPS Advisor base URL | +| `VAPN_ADVISOR_TOKEN` | string | — | Service credential for the VPS Advisor API | + +### Artifact store (`VAPN_ARTIFACT_*`) + +Provider-agnostic S3-compatible storage for snapshot artifacts. Switching +providers is an environment change only, never code. Used by the builder +(writes) and coordinator (serves). + +| Variable | Meaning | +|---|---| +| `VAPN_ARTIFACT_S3_ENDPOINT` | S3 endpoint host | +| `VAPN_ARTIFACT_S3_REGION` | Region (`auto` for R2) | +| `VAPN_ARTIFACT_S3_BUCKET` | Bucket (default `vapn-artifacts`) | +| `VAPN_ARTIFACT_S3_ACCESS_KEY` / `_SECRET_KEY` | Credentials | +| `VAPN_ARTIFACT_S3_USE_SSL` | `true` in prod, `false` for local MinIO | +| `VAPN_ARTIFACT_DIR` | Filesystem store instead of S3 (tests/single host) | + +Provider examples (endpoint mapping) are in +[architecture 07 §2](../architecture/07-deployment.md#production-platform-side). + +## Coordinator + +| Variable | Type | Default | Meaning | +|---|---|---|---| +| `VAPN_HTTP_ADDR` | string | `:8080` | Worker-facing listen address | +| `VAPN_DB_DSN` | string (req.) | — | Database | +| `VAPN_ADMIN_TOKEN` | string (req.) | — | Bearer token for the [admin API](../api/README.md#c-platform-admin-api) / `vapnctl` | +| `VAPN_ADMIN_ALLOW_CIDR` | string | `127.0.0.1/32` | CIDR allowlist for the admin surface (set via Caddy in prod) | +| `VAPN_DEV_ENROLLMENT_TOKEN` | string | — | Dev-only shortcut enrollment token | +| `VAPN_SCHEDULER_INTERVAL` | duration | `30s` | How often the scheduler rebalances | +| `VAPN_REDUNDANCY` | int | `3` | Distinct workers per target | +| `VAPN_MAX_ASSIGNMENTS_PER_WORKER` | int | `64` | Cap on concurrent assignments per worker | +| `VAPN_GEOIP_ASN_MMDB` | string | — | Path to GeoLite2-ASN (worker source-ASN lookup) | +| `VAPN_ADVISOR_URL` / `_TOKEN` | string | — | VPS Advisor pull (catalog) + push | +| `VAPN_ADVISOR_SYNC_INTERVAL` | duration | `2m` | Provider/decision/enrollment poll cadence | + +## Aggregator + +| Variable | Type | Default | Meaning | +|---|---|---|---| +| `VAPN_HTTP_ADDR` | string | `:8082` | Health/metrics listen address | +| `VAPN_DB_DSN` | string (req.) | — | Database | +| `VAPN_TRUST_INTERVAL` | duration | `1m` | Trust recomputation cadence | +| `VAPN_WINDOW_SECONDS` | int | `300` | Consensus window length | +| `VAPN_MIN_WORKERS` | int | `3` | Distinct workers required for a verdict (else `insufficient_data`) | +| `VAPN_ADVISOR_URL` / `_TOKEN` | string | — | Results push target | + +Consensus tuning (`HealthyRatio` 0.9, `DegradedRatio` 0.5, `LatencyFactor` 2.0, +retention windows) has code defaults; see [`internal/aggregate`](../../internal/aggregate/aggregate.go) +and [trust calculation](../walkthroughs/trust-calculation.md). + +## Builder + +| Variable | Type | Default | Meaning | +|---|---|---|---| +| `VAPN_DB_DSN` | string (req.) | — | Database (builder role) | +| `VAPN_ADVISOR_URL` | string | `http://localhost:8081` | Monitored-ASN source | +| `VAPN_ADVISOR_TOKEN` | string (req.) | — | VPS Advisor credential | +| `VAPN_RIS_BVIEW_URL` | string | — | RIS `bview` download URL (prod: `data.ris.ripe.net/rrc00/latest-bview.gz`) | +| `VAPN_RIS_BVIEW_PATH` | string | `data/ripe/latest-bview.gz` | Local dump path/cache | +| `VAPN_RIS_BVIEW_MAX_AGE` | duration | `6h` | Reject dumps older than this | +| `VAPN_GEOIP_CITY_MMDB` | string | — | Path to GeoLite2-City | +| `VAPN_MAX_TARGETS_PER_PROVIDER` | int | `100` | Probe-target cap per provider | +| `VAPN_SANITY_MAX_DELTA_PCT` | int | `50` | Hold for approval if prefix count swings past this % | +| `VAPN_SANITY_FORCE` | bool | `false` | Bypass the sanity gate | +| `VAPN_RETAIN_SNAPSHOTS` | int | `5` | Old snapshots to keep | +| `VAPN_SNAPSHOT_SIGNING_KEY` | string (secret) | — | Base64 Ed25519 private key for signing artifacts | +| `VAPN_MIN_WORKER_VERSION` | string | `0.1.0` | Oldest worker version allowed to use the snapshot | + +## Worker + +Written to `~/.vapn/config.env` by `vapn install`. Only two are required. + +| Variable | Required | Meaning | +|---|---|---| +| `VAPN_COORDINATOR_URL` | yes | Coordinator endpoint the worker talks to | +| `VAPN_ENROLLMENT_TOKEN` | yes (first boot) | One-time enrollment token; spent on registration | +| `VAPN_MAXMIND_LICENSE_KEY` | no | Operator's own key for a local GeoIP DB (degrades gracefully) | +| `VAPN_HOME` | no | Override the `~/.vapn` home directory | +| `VAPN_STATE_DIR` | no | Override the state directory | +| `VAPN_WORKER_IMAGE` | no | Pin a specific worker image | +| `VAPN_WORKER_NAME` | no | Human-friendly worker name | + +## Snapshot signing keys (`SNAPSHOT_SIGNING_KEY` / `SNAPSHOT_PUBLIC_KEY`) + +Generate with `./bin/keygen`. The **builder** holds the private key +(`VAPN_SNAPSHOT_SIGNING_KEY`); the **worker image** pins the public key +(`VAPN_SNAPSHOT_PUBLIC_KEY`) so it can verify every artifact before use. Keep the +private key offline/secret; rotate by generating a new pair and shipping a worker +release with the new pinned public key. See +[builder installation](../builder/installation.md#step-1--generate-a-snapshot-signing-key). + +## Test / development + +| Variable | Meaning | +|---|---| +| `VAPN_TEST_DB_DSN` | Test database DSN (default `postgres://vapn:vapn-dev@localhost:5433/vapn_test`) — tests **never** touch the live dev DB | + +## How it's set in each environment + +- **Dev:** [`deploy/compose/dev.compose.yaml`](../../deploy/compose/dev.compose.yaml) + sets everything; pre-downloaded `data/` is mounted so no external fetch is + needed. +- **Production:** [`deploy/prod/.env.example`](../../deploy/prod/.env.example) + + `docker-compose.yml`; secrets live in an env file outside the repo (or a + secrets manager). Walkthrough: [deployment guide](../operations/deployment.md). diff --git a/docs/reference/database-schema.md b/docs/reference/database-schema.md new file mode 100644 index 0000000..c9b9f4a --- /dev/null +++ b/docs/reference/database-schema.md @@ -0,0 +1,119 @@ +# Database Schema Reference + +VAPN uses **one PostgreSQL cluster, one database, six schemas**. This page is a +table-by-table summary; the full DDL with column types, constraints, and index +notes is [architecture 03](../architecture/03-database-design.md), and the +authoritative source is [`migrations/`](../../migrations/) (one ordered SQL +stream applied by the `migrate` job). + +## Conventions + +- `bigint generated always as identity` PKs for high-volume tables; `uuid` + external IDs where values cross system boundaries (worker/provider IDs align + with VPS Advisor). +- `timestamptz` everywhere; `text` + `CHECK` for states that will evolve; native + `cidr`/`inet` for routing. +- **One Postgres role per service, least privilege:** the builder can't touch + `measurements`; the coordinator can't write `aggregation`; the aggregator reads + `measurements` and writes `aggregation` + `registry.trust_*`. + +## Schema map + +```mermaid +flowchart LR + routing["routing
provider, asn, prefix,
snapshot, probe_target"] + registry["registry
worker, worker_key, enrollment_token,
trust_score, trust_event, replay_nonce"] + scheduling["scheduling
assignment, lease"] + measurements["measurements
observation (partitioned),
upload_batch"] + aggregation["aggregation
consensus_window, provider_status,
anomaly, worker_agreement, publication_outbox"] + audit["audit
event (append-only)"] + routing --> scheduling --> measurements --> aggregation + registry --> scheduling + aggregation --> registry +``` + +## Schema `routing` + +Owned by the builder; the canonical routing intelligence. Prefixes are +per-snapshot and immutable — diffing snapshots yields routing-churn signals. + +| Table | Purpose | Key columns | +|---|---|---| +| `snapshot` | A versioned routing build | `version`, `status` (building/published/superseded/failed), `asn_count`, `prefix_count_*`, `artifact_sha256`, `artifact_signature` | +| `provider` | Cache of VPS Advisor providers | `provider_id` (uuid, verbatim), `monitoring_enabled`, `priority`, `delisted_at` | +| `asn` | Monitored ASNs → provider | `asn` (pk), `provider_id`, registry name/country | +| `prefix` | Prefixes per snapshot | `snapshot_id`, `prefix` (cidr), `origin_asn`, geo columns, `flags` jsonb (bogon, moas_conflict) — GiST index on `prefix` | +| `probe_target` | Derived probeable addresses | `snapshot_id`, `provider_id`, `prefix_id`, `address` (inet), `rationale`, `active` | + +## Schema `registry` + +The worker registry and trust state. + +| Table | Purpose | Key columns | +|---|---|---| +| `worker` | A worker node | `id` (uuid), `operator_id`, `state`, `software_version`, `verified_country`, `source_asn`, `last_heartbeat_at`, `config` jsonb | +| `worker_key` | Public keys with validity | `worker_id`, `public_key` (bytea), `valid_from/until`, `revoked_at` — partial unique index: one current key per worker | +| `enrollment_token` | One-time tokens | `token_hash` (sha256, pk), `worker_id`, `expires_at`, `used_at` — plaintext never stored | +| `trust_score` | Current trust per worker | `worker_id` (pk), `score` [0,1], `components` jsonb, `computed_at` | +| `trust_event` | Append-only trust/security events | `worker_id`, `event_type` (approved, suspended, disagreement, bad_signature, replay…), `actor`, `detail` | +| `replay_nonce` | Replay protection | `(worker_id, nonce, seen_at)`, short TTL pruning | + +## Schema `scheduling` + +| Table | Purpose | Key columns | +|---|---|---| +| `assignment` | Instruction to probe target T with type P at interval I | `target_id`, `provider_id`, `probe_type`, `interval_seconds`, `redundancy_group` (uuid), `status` (open/leased/draining/closed) | +| `lease` | A worker's time-bounded claim | `assignment_id`, `worker_id`, `expires_at` (renewed by heartbeat), `released_at` — partial unique index: one live lease per (assignment, worker) | + +## Schema `measurements` + +The write hotspot — **partitioned by day**, internal-only, immutable. + +| Table | Purpose | Key columns | +|---|---|---| +| `observation` | One signed measurement | `worker_id`, `assignment_id`, `provider_id`, `target` (inet), `probe_type`, `measured_at` (worker clock), `received_at`, `ok`, `rtt_ms`, `packets_sent/lost`, `metrics` jsonb, `signature` — PK `(id, measured_at)`, range-partitioned by `measured_at` | +| `upload_batch` | Batch dedup / idempotency | batch id, `received_at` | + +**Retention:** raw partitions dropped after N days (default 14); aggregates keep +history. Inserts are bulk (COPY-style). + +## Schema `aggregation` + +Consensus, public status, anomalies, and the publication outbox. Owned by the +aggregator. + +| Table | Purpose | Key columns | +|---|---|---| +| `consensus_window` | Trust-weighted aggregate per (provider, region, probe_type, window) | `verdict`, `confidence`, `worker_count`, `dissent_ratio`, `loss_rate`, `rtt_p50/p95/p99` — unique on the tuple (idempotent settle) | +| `provider_status` | Current rollup, one row per (provider, region) | `verdict`, `confidence`, `since`, `metrics` jsonb, `updated_at` | +| `anomaly` | Detected events | `kind` (reachability_loss/latency_regression/routing_churn), `severity`, `opened_at`, `resolved_at`, `evidence` jsonb | +| `worker_agreement` | Per-worker agreement vs settled consensus | `worker_id`, `window_start`, `agreement`, `targets` — feeds trust scoring | +| `publication_outbox` | At-least-once push to VPS Advisor | `kind`, `payload` jsonb, `attempts`, `next_attempt_at`, `acked_at` | + +## Schema `audit` + +| Table | Purpose | Key columns | +|---|---|---| +| `event` | Append-only audit trail (no updates/deletes) | `category` (auth/admin/snapshot/security), `actor`, `action`, `subject`, `detail` jsonb, `created_at` | + +Every auth failure, state transition, admin action, snapshot publication, and +policy change lands here. Queryable via `vapnctl audit`; security aggregates flow +to the VPS Advisor admin dashboard via [fleet telemetry](../integration/django-integration.md#44-results-ingestion-platform-pushes). + +## Migrations + +| File | Adds | +|---|---| +| `0001_routing.sql` | routing schema | +| `0002_registry.sql` | registry schema | +| `0003_scheduling.sql` | scheduling schema | +| `0004_measurements.sql` | measurements (partitioned) | +| `0005_aggregation.sql` | aggregation schema | +| `0006_audit.sql` | audit schema | +| `0007_upload_batch.sql` | upload batch idempotency | +| `0008_worker_agreement.sql` | per-worker agreement table | + +Migrations are backward-compatible one version back (expand → migrate → +contract) so coordinator replicas can roll during an upgrade. Applied by +`./bin/migrate` (or the `migrate` container). Backups: nightly base + WAL +archiving — see [backup & restore](../operations/backup-restore.md). diff --git a/docs/reference/glossary.md b/docs/reference/glossary.md new file mode 100644 index 0000000..d9eef00 --- /dev/null +++ b/docs/reference/glossary.md @@ -0,0 +1,200 @@ +# Glossary + +Every networking and project-specific term used in this documentation, defined +in plain language. Networking terms link to the concept page that teaches them. + +## Networking + +**AS (Autonomous System)** — an independently operated network (an ISP, cloud, +or hosting company) that makes its own routing decisions. → +[ASN & BGP](../concepts/asn-and-bgp.md) + +**ASN (Autonomous System Number)** — the globally unique number identifying an +AS, e.g. `AS64500`. A provider *is*, on the Internet, its set of ASNs. → +[ASN & BGP](../concepts/asn-and-bgp.md) + +**BGP (Border Gateway Protocol)** — the protocol ASes use to announce which +address prefixes they can reach; it's what glues the Internet's networks +together. → [ASN & BGP](../concepts/asn-and-bgp.md) + +**bogon** — an address block that must never appear in public routing (private, +reserved, or absurd). The builder drops these so they can't become targets. → +[Prefix ownership](../concepts/prefix-ownership.md) + +**`bview` / RIB dump** — a full-table snapshot of the routing table, taken by a +RIS collector every 8 hours. VAPN builds from these. → +[RIPE/RIS/MRT](../concepts/ripe-and-ris.md) + +**CIDR** — notation for an address block: `address/prefix-length`, e.g. +`203.0.113.0/24` (256 addresses). Smaller number after the slash = bigger block. +→ [How the Internet works](../concepts/how-the-internet-works.md) + +**GeoIP** — mapping IP addresses to approximate geographic locations and to the +registered network. VAPN uses MaxMind GeoLite2. → [GeoIP](../concepts/geoip.md) + +**ICMP echo (ping)** — the request/reply used to test reachability and measure +round-trip time; VAPN's first probe type. → +[Measurement](../concepts/measurement-and-consensus.md) + +**IP address** — the numeric label identifying a device on the network +(`203.0.113.7`). IPv4 (32-bit) and IPv6 (128-bit). → +[How the Internet works](../concepts/how-the-internet-works.md) + +**MOAS (Multiple Origin AS)** — a prefix announced by more than one origin AS — +ambiguous ownership. VAPN flags these rather than guessing. → +[Prefix ownership](../concepts/prefix-ownership.md) + +**MRT** — the binary file format (RFC 6396) that RIS routing recordings are +stored in. Only the builder parses it. → [RIPE/RIS/MRT](../concepts/ripe-and-ris.md) + +**origin AS** — the AS that originates a prefix's announcement — its owner, for +VAPN's purposes. → [ASN & BGP](../concepts/asn-and-bgp.md) + +**prefix** — an announced address block; a provider's "streets." VAPN reasons +about prefixes, not individual addresses. → +[How the Internet works](../concepts/how-the-internet-works.md) + +**RIB (Routing Information Base)** — a router's table of all known routes; +captured as a `bview` dump. → [RIPE/RIS/MRT](../concepts/ripe-and-ris.md) + +**RIPE NCC** — the Regional Internet Registry for Europe; runs the RIS service +VAPN reads routing data from. → [RIPE/RIS/MRT](../concepts/ripe-and-ris.md) + +**RIR (Regional Internet Registry)** — one of five bodies (RIPE, ARIN, APNIC, +LACNIC, AFRINIC) that allocate addresses and ASNs. → +[RIPE/RIS/MRT](../concepts/ripe-and-ris.md) + +**RIS (Routing Information Service)** — RIPE's network of passive collectors that +record global BGP announcements. → [RIPE/RIS/MRT](../concepts/ripe-and-ris.md) + +**route collector / rrc** — a RIS machine that records BGP without announcing +anything; VAPN defaults to `rrc00`. → [RIPE/RIS/MRT](../concepts/ripe-and-ris.md) + +**route propagation** — how BGP announcements/withdrawals spread across the +Internet over seconds to minutes. → [ASN & BGP](../concepts/asn-and-bgp.md) + +**RTT (round-trip time)** — the latency of a probe, in milliseconds. → +[How the Internet works](../concepts/how-the-internet-works.md) + +**reachability** — whether a packet gets to a target and back at all. → +[Measurement](../concepts/measurement-and-consensus.md) + +## VAPN concepts + +**aggregation engine (`aggregator`)** — the service that computes trust-weighted +[consensus](../concepts/measurement-and-consensus.md#consensus-from-many-views-to-one-verdict), detects +anomalies, updates trust, and publishes results. + +**anomaly** — a detected reachability loss, latency regression, or routing +churn, opened/resolved by the aggregator. → +[Measurement](../concepts/measurement-and-consensus.md#detecting-anomalies) + +**artifact / snapshot artifact** — the compact, signed **SQLite** file (plus +manifest) that workers download; the worker-facing subset of a routing snapshot. +→ [Builder](../builder/README.md) + +**assignment** — an instruction to a worker: probe target T with probe type P at +interval I. → [Measurement lifecycle](../walkthroughs/measurement-lifecycle.md) + +**builder (Snapshot Builder)** — the batch service that turns RIPE data into a +signed target list; the only component that reads MRT/MaxMind data. → +[Builder](../builder/README.md) + +**confidence** — how sure a verdict is (higher with more distinct workers and +less dissent). → [Measurement](../concepts/measurement-and-consensus.md#consensus-from-many-views-to-one-verdict) + +**consensus** — the trust-weighted combination of a redundancy group's +measurements into a verdict. → +[Measurement](../concepts/measurement-and-consensus.md#consensus-from-many-views-to-one-verdict) + +**consensus window** — a fixed time bucket (default 5 min) over which consensus +is computed. → [Trust calculation](../walkthroughs/trust-calculation.md) + +**control plane / data plane** — the split between VPS Advisor (identity, +catalog, enrollment, admin — control) and this platform's high-volume +worker-facing Coordinator API (data). → +[Architecture 01 §2](../architecture/01-system-architecture.md#2-api-plane-split-clarification-of-the-brief) + +**coordinator** — the long-running HTTP service workers talk to: auth, +heartbeats, scheduling, observation intake. → +[Architecture 01 §4.2](../architecture/01-system-architecture.md#42-coordinator-coordinator) + +**Ed25519** — the public-key signature scheme workers use to sign requests and +observations, and the builder uses to sign artifacts. → +[Worker authentication](../walkthroughs/worker-authentication.md) + +**enrollment token** — a one-time secret proving a real operator started a +worker; traded for a permanent identity on first boot. → +[Worker installation](../walkthroughs/worker-installation.md) + +**heartbeat** — the ~30 s worker→coordinator call carrying liveness/version and +returning config, leases, snapshot version, and control actions. → +[Measurement lifecycle](../walkthroughs/measurement-lifecycle.md) + +**insufficient_data** — the verdict when too few distinct trusted workers +measured a provider; shown as "not enough data," never an outage. → +[Measurement](../concepts/measurement-and-consensus.md#consensus-from-many-views-to-one-verdict) + +**lease** — a worker's time-bounded claim on an assignment, renewed by +heartbeat; expiry triggers reassignment. → +[Measurement lifecycle](../walkthroughs/measurement-lifecycle.md) + +**observation** — one signed measurement result from one worker; internal-only, +never public on its own. → +[Measurement](../concepts/measurement-and-consensus.md#what-a-single-measurement-is) + +**operator** — the human/community account (on VPS Advisor) that runs workers. + +**probe engine / prober** — the protocol-agnostic component inside the worker; +ICMP is the first `Prober` implementation. → +[Measurement lifecycle](../walkthroughs/measurement-lifecycle.md) + +**probe target** — a representative address chosen from a prefix; what workers +actually probe. → [Prefix ownership](../concepts/prefix-ownership.md) + +**quarantine / shadow mode** — a lifecycle state where a worker measures at +weight 0 to rebuild trust. → [Worker lifecycle](../worker/lifecycle.md#quarantined-shadow-mode) + +**redundancy group** — the set of independent workers covering the same target, +spread across regions/networks/operators. → +[Measurement](../concepts/measurement-and-consensus.md#the-fix-community-measurement--redundancy) + +**replay protection** — timestamp window + per-worker nonce uniqueness that stops +captured requests from being reused. → +[Worker authentication](../walkthroughs/worker-authentication.md) + +**routing snapshot** — an immutable, versioned record of prefix ownership at +build time; the source of the worker artifact. → +[Snapshot publishing](../walkthroughs/snapshot-publishing.md) + +**sanity gate** — the builder's hold-for-approval check when a snapshot's prefix +count swings past a threshold (route-leak protection). → +[Builder](../builder/README.md) + +**scheduler** — the coordinator module that turns targets into assignments and +leases them with a redundancy factor. → +[Architecture 01 §4.2](../architecture/01-system-architecture.md#42-coordinator-coordinator) + +**source ASN** — the network a worker probes *from*; used for measurement +diversity and to stop a worker measuring its own provider. → +[GeoIP](../concepts/geoip.md) + +**Sybil attack** — one actor running many fake nodes to sway results; blunted by +manual approval, the tenure ramp, and per-operator weight caps. → +[Security & trust model](../architecture/05-security-trust-model.md) + +**trust** — a worker's [0,1] reliability score; its weight in consensus. → +[Measurement](../concepts/measurement-and-consensus.md#trust) + +**verdict** — a provider's health state: `healthy`, `degraded`, `outage`, or +`insufficient_data`. → [Measurement](../concepts/measurement-and-consensus.md#consensus-from-many-views-to-one-verdict) + +**VPS Advisor** — the independent, already-live review website that is the source +of truth for providers and the consumer of VAPN's results. **This repository is +not VPS Advisor.** → [Documentation Home](../README.md#project-background) + +**worker** — a community-run container that probes targets and uploads signed +measurements. → [Community Workers](../worker/README.md) + +**`vapn` / `vapnctl`** — the worker-operator CLI and the platform-admin CLI. → +[CLI reference](cli.md) diff --git a/docs/walkthroughs/README.md b/docs/walkthroughs/README.md new file mode 100644 index 0000000..74933e0 --- /dev/null +++ b/docs/walkthroughs/README.md @@ -0,0 +1,42 @@ +# Walkthroughs + +Concepts and architecture tell you *what* the pieces are. Walkthroughs show you +the pieces *in motion* — following real data and control flow through the +system, stage by stage, in plain English. Read these once the +[Core Concepts](../concepts/README.md) make sense; they're the bridge from +theory to the running system. + +## Start here + +**[End-to-end: a provider becomes a public verdict](end-to-end.md)** — the +master walkthrough. Follow one provider from the moment it's added on VPS +Advisor, through routing, probing, and consensus, to the health badge on its +public page. Every other walkthrough zooms into one stage of this one. + +```mermaid +flowchart TD + A[Provider added on VPS Advisor] --> B[ASNs synced to VAPN] + B --> C[Builder downloads RIPE data] + C --> D[Snapshot built + signed] + D --> E[Snapshot published] + E --> F[Workers detect + download it] + F --> G[Workers probe targets] + G --> H[Signed measurements uploaded] + H --> I[Aggregation computes consensus] + I --> J[Results published to VPS Advisor] + J --> K[Provider page shows network health] +``` + +## Focused walkthroughs + +| Walkthrough | Zooms into | Audience | +|---|---|---| +| [Worker installation](worker-installation.md) | What actually happens from `install.sh` to a probing worker | Contributors, operators | +| [Snapshot publishing](snapshot-publishing.md) | The builder run: RIPE → PostgreSQL → signed artifact → published | Operators | +| [Worker authentication](worker-authentication.md) | Enrollment, keypairs, request signing, replay protection, rotation | Developers, security reviewers | +| [Measurement lifecycle](measurement-lifecycle.md) | Heartbeat → lease → probe → sign → upload → persist | Developers | +| [Trust calculation](trust-calculation.md) | How a worker's score is computed, window by window, with numbers | Developers, operators | +| [Software updates](software-updates.md) | Health-gated worker updates and min-version enforcement | Contributors, operators | + +Each is self-contained but assumes the concepts. If a term is unfamiliar, the +[Glossary](../reference/glossary.md) has it. diff --git a/docs/walkthroughs/end-to-end.md b/docs/walkthroughs/end-to-end.md new file mode 100644 index 0000000..ae6bd5e --- /dev/null +++ b/docs/walkthroughs/end-to-end.md @@ -0,0 +1,226 @@ +# End-to-End: A Provider Becomes a Public Verdict + +This is the master walkthrough. We follow a single provider — call it +**ExampleHost**, owning **AS64500** — from the moment it's added on VPS Advisor +all the way to the network-health badge on its public page. Every stage links to +the concept behind it and the focused walkthrough that goes deeper. + +If you read only one document to understand how VAPN works as a *system*, read +this one. + +```mermaid +flowchart TD + A["1. Provider added on VPS Advisor"] --> B["2. ASN synced to VAPN"] + B --> C["3. Builder downloads RIPE data"] + C --> D["4. Snapshot built + validated + signed"] + D --> E["5. Snapshot published to artifact store"] + E --> F["6. Workers detect + verify + download"] + F --> G["7. Workers lease + probe targets"] + G --> H["8. Signed measurements uploaded"] + H --> I["9. Aggregation computes consensus"] + I --> J["10. Trust updated; anomalies detected"] + J --> K["11. Results pushed to VPS Advisor"] + K --> L["12. Provider page shows network health"] +``` + +--- + +## Stage 1 — A provider is added on VPS Advisor + +A hosting company claims its listing on **VPS Advisor** (the website — *not* +this project) and an admin records its **ASN(s)**: ExampleHost owns `AS64500`. +Someone toggles **monitoring enabled** for it. + +Nothing in VAPN has happened yet. VPS Advisor is the +[source of truth](../README.md#project-background) for *which* providers exist +and *which* ASNs they own; VAPN never discovers providers on its own. + +> **Concept:** [ASN & BGP](../concepts/asn-and-bgp.md) — why a provider *is* its +> ASNs on the Internet. + +## Stage 2 — The ASN is synced into VAPN + +Every couple of minutes, VAPN's **coordinator** pulls the provider catalog from +VPS Advisor's `GET /api/v1/monitoring/providers` endpoint (using its service +credential). It sees ExampleHost, `AS64500`, `monitoring_enabled=true`, +priority. It stores this as a **cache with provenance** — just the ID, name, +ASNs, priority, and enabled flag — never a full copy of provider business data. + +At this instant ExampleHost is *known* but not yet *measurable*: there are no +prefixes and no probe targets for it until the next builder run includes it. + +> **Deeper:** [Provider sync lifecycle](../architecture/06-lifecycles.md#3-provider-sync-lifecycle) +> · [Integration §4.1](../integration/django-integration.md#41-provider-catalog-platform-pulls). + +## Stage 3 — The builder downloads RIPE routing data + +On its schedule (daily), the **[Routing Builder](../builder/README.md)** runs: + +1. It asks VPS Advisor for the current list of monitored ASNs — `AS64500` is now + on it. +2. It downloads the latest **RIPE RIS `bview`** dump — a full snapshot of the + global routing table in **MRT** format, ~1M routes (in dev, the pre- + downloaded `data/ripe/latest-bview.gz`). + +> **Concept:** [RIPE, RIS & MRT](../concepts/ripe-and-ris.md) — what this data +> is and why the builder, not workers, reads it. + +## Stage 4 — A snapshot is built, validated, and signed + +The builder streams through the million MRT records and keeps only the slice it +cares about — prefixes **originated by `AS64500`** (and the other monitored +ASNs). Say ExampleHost announces `203.0.113.0/24` and `198.51.100.0/23`. The +builder then: + +- **Deduplicates** (many RIS peers reported the same prefix). +- **Drops bogons** and **flags MOAS conflicts** — never guessing ownership. +- **Enriches** each prefix with **[GeoIP](../concepts/geoip.md)** (country, + city, coordinates). +- **Loads** the result into the canonical `routing` schema in PostgreSQL under a + new **snapshot version**. +- **Derives probe targets** — a small, capped set of representative addresses + per prefix/region (not every one of the 256+512 addresses). +- **Sanity-gates** the build: if the prefix count swung wildly versus the + previous snapshot, it *holds for admin approval* rather than publishing + possibly-poisoned data. +- **Exports a signed SQLite artifact** + a manifest (version, counts, sha256, + Ed25519 signature, minimum worker version). + +> **Concept:** [Prefix ownership](../concepts/prefix-ownership.md). **Deeper:** +> [Snapshot publishing walkthrough](snapshot-publishing.md). + +## Stage 5 — The snapshot is published + +The builder uploads the artifact + manifest to the **artifact store** (an +S3-compatible bucket, CDN-frontable), verifies the readback, marks the new +snapshot `published`, and marks the previous one `superseded`. Publication is +**atomic from the workers' view**: until this moment they keep using the old +snapshot; the instant it completes, the new version is the current one. If any +step failed, the snapshot is marked `failed` and the old one stays fully in +force. + +The coordinator will now advertise the new version to workers in heartbeats. + +## Stage 6 — Workers detect, verify, and download it + +Each **community worker**, on its ~30-second heartbeat, is told "current +snapshot is version X." If it doesn't have X yet, it: + +1. Downloads the artifact from the store (or CDN). +2. **Verifies** the sha256 and the **Ed25519 signature** against a public key + *pinned into the worker image* — a tampered or substituted artifact is + rejected. It also checks the version is newer (downgrade protection). +3. **Atomically swaps** it in (download to temp, verify, rename). + +The worker now holds ExampleHost's targets locally and can even re-check that a +target is legitimate before probing it. The artifact store itself is untrusted +by design — integrity comes entirely from the signature. + +> **Concept:** [Trust — snapshot integrity](../concepts/measurement-and-consensus.md). +> **Deeper:** [Worker authentication](worker-authentication.md). + +## Stage 7 — Workers are assigned targets and probe them + +The coordinator's embedded **scheduler** turns targets into **assignments** and +leases them to workers with a **redundancy factor** (default 3): each of +ExampleHost's targets is measured by several workers, deliberately spread across +different regions and source networks, and never by a worker on ExampleHost's +own network. + +A worker leases some ExampleHost targets and starts probing — an **ICMP echo** +to each target on the assignment's interval, recording reachability and RTT. + +> **Concept:** [Why community measurement + redundancy](../concepts/measurement-and-consensus.md#the-fix-community-measurement--redundancy). +> **Deeper:** [Measurement lifecycle](measurement-lifecycle.md). + +## Stage 8 — Signed measurements are uploaded + +The worker batches its observations (~60 s or a size threshold), **signs each +one and the batch** with its private key, and uploads to +`POST /api/v1/observations`. The coordinator verifies the signature and the +worker's state, checks the timestamp/nonce (replay protection), and persists the +batch to the time-partitioned `measurements` schema via bulk insert. Invalid or +replayed data is rejected at the door and recorded as a trust event. + +Crucially: these raw observations are **internal only**. No single one will ever +appear on the public site. + +## Stage 9 — Aggregation computes consensus + +Every window (default 5 minutes) the **aggregation engine** settles the just- +completed window for ExampleHost: + +- Per target, workers vote by **trust weight**; a target is **up** if ≥ 50% of + weight saw it reachable. Only **responsive targets** (answered someone in 24 h) + count. +- The provider verdict follows from the up-fraction: **healthy** (≥ 90%), + **degraded** (≥ 50%), **outage** (< 50%), or **insufficient_data** (fewer + than 3 distinct workers, or nothing measurable). +- It records confidence, latency p50/p95/p99, and loss rate. + +Say 12 trusted workers all reached ExampleHost's targets at ~22 ms: verdict +**healthy**, confidence high. + +> **Concept:** [Consensus](../concepts/measurement-and-consensus.md#consensus-from-many-views-to-one-verdict). + +## Stage 10 — Trust is updated; anomalies are detected + +The same pipeline: + +- Scores **per-worker agreement** against the settled consensus and feeds it + into each worker's **trust** score (agreement is the dominant term). Workers + that matched the crowd gain; a worker that reported "unreachable" while + everyone else saw "reachable" loses a little. +- Compares this window to history: a transition into degraded/outage opens a + **reachability anomaly**; a latency p50 ≥ 2× the 6-hour baseline opens a + **latency anomaly**; returning to healthy resolves them. + +> **Deeper:** [Trust calculation](trust-calculation.md). + +## Stage 11 — Results are pushed to VPS Advisor + +The engine writes the provider's current status document to a **publication +outbox** and a publisher drains it to VPS Advisor's Results API +(`PUT /api/v1/monitoring/results/providers/{id}`), with **at-least-once, +idempotent** delivery and exponential backoff if the site is down. Anomalies go +to `POST …/results/anomalies`; a fleet-telemetry summary goes to the admin +dashboard endpoint periodically. + +```json +PUT …/results/providers/7f9c… +{ "as_of": "2026-07-18T08:05:00Z", + "global": { "verdict": "healthy", "confidence": 0.97, + "metrics": { "rtt_p50_ms": 21.4, "loss_rate": 0.001, "worker_count": 12 } } } +``` + +> **Deeper:** [Integration §4.4](../integration/django-integration.md#44-results-ingestion-platform-pushes). + +## Stage 12 — The provider page shows network health + +VPS Advisor stores that document and renders the **Network Health** section on +ExampleHost's public page: a healthy badge, confidence, latency, and any recent +instability — with the display guidance that `insufficient_data` is shown as +"not enough data," never as an outage, and that these are *public-network +reachability* signals, not SLA claims. + +The loop then repeats forever: new windows refine the verdict, new snapshots +track routing changes, workers come and go, and trust keeps everyone honest. + +--- + +## The whole thing, in one breath + +VPS Advisor says *who* to watch → the builder learns each provider's real +public prefixes from RIPE and publishes a signed target list → a global +community of workers probes those targets from many independent vantage points → +every measurement is signed and combined into a trust-weighted consensus → only +that consensus, never any individual's report, is published back to VPS Advisor +for the world to see. + +Next, pick any stage to go deeper: +[installation](worker-installation.md) · +[publishing](snapshot-publishing.md) · +[authentication](worker-authentication.md) · +[measurement](measurement-lifecycle.md) · +[trust](trust-calculation.md) · +[updates](software-updates.md). diff --git a/docs/walkthroughs/measurement-lifecycle.md b/docs/walkthroughs/measurement-lifecycle.md new file mode 100644 index 0000000..11ceffb --- /dev/null +++ b/docs/walkthroughs/measurement-lifecycle.md @@ -0,0 +1,98 @@ +# Walkthrough: Measurement Lifecycle + +The steady-state loop of an active worker: how one probe goes from "I have +capacity" to a durable, signed row in the database. This expands +[Stages 7–8](end-to-end.md#stage-7--workers-are-assigned-targets-and-probe-them) +of the end-to-end flow. + +```mermaid +sequenceDiagram + participant W as Worker + participant C as Coordinator + participant DB as measurements schema + loop every ~30s + W->>C: POST /heartbeat (version, resources) + C-->>W: config, control actions,
snapshot version, lease renewals + end + W->>C: POST /assignments/lease (capacity, capabilities) + C-->>W: assignment batch (targets, probe type, interval, expiry) + loop on each assignment's interval + W->>W: ICMP echo to target, record ok + RTT + end + W->>W: batch observations (~60s / size threshold), sign each + batch + W->>C: POST /observations (signed batch, idempotency key) + C->>C: verify signature, worker state, timestamp/nonce + C->>DB: bulk insert observations + C-->>W: 200 (per-item accept/reject) +``` + +## The loop, step by step + +1. **Heartbeat (~30 s).** The worker calls `POST /api/v1/workers/heartbeat` + reporting its version and resource stats. The response carries **config** + (rate limits, intervals), **control actions** (rotate key, drain, suspend, + upgrade-required), the **current snapshot version**, and **lease renewals**. + Heartbeat is the worker's single source of truth about what it should be + doing. Missing heartbeats → leases expire → the scheduler reassigns the work. + +2. **Lease assignments.** The worker calls `POST /api/v1/assignments/lease` with + its capacity and capabilities. The **scheduler** returns a batch of + assignments — each an instruction: *probe target T with probe type P every I + seconds until this lease expires* — respecting `MAX_ASSIGNMENTS_PER_WORKER` + (default 64) and the redundancy factor (each target covered by N distinct + workers across regions/networks). + +3. **Probe.** For each assignment, on its interval, the worker's **probe + engine** runs the probe. In v1 that's an **ICMP echo** (a few packets) via + `internal/probe/icmp`, recording whether a reply came back (`ok`) and the + round-trip time (`rtt_ms`). Before probing, the worker can re-check the + target against its local signed snapshot — defense against a stale or tampered + assignment. The engine is protocol-agnostic (a `Prober` interface), so + TCP/traceroute/HTTP probes slot in later without changing this loop. + +4. **Batch + sign.** Observations accumulate and are flushed roughly every 60 s + or when a size threshold is hit. Each observation *and* the batch are + **signed** ([auth walkthrough](worker-authentication.md)). Batching keeps + upload volume low at fleet scale. + +5. **Upload.** `POST /api/v1/observations` with a **batch id** (idempotent — a + retried batch won't double-insert). The coordinator verifies the request + signature, the worker's state (`403` if suspended), and timestamp/nonce, then + validates each observation. It responds `207`-style with per-item + accept/reject so one bad row doesn't sink the batch. + +6. **Persist.** Accepted observations are bulk-inserted (COPY) into the + time-partitioned `measurements.observation` table (daily partitions, + internal-only, immutable). This is the platform's write hotspot, which is why + uploads are batched and inserts are bulk. + +7. **Consume.** The [aggregation engine](trust-calculation.md) reads these rows + each window to compute consensus — but that's a separate pipeline; the worker + is done once its batch is accepted. + +## Resilience built into the loop + +- **Coordinator downtime is survivable.** If uploads fail, the worker **queues + observations locally** (bounded disk buffer) and retries with backoff — brief + platform maintenance loses no data. +- **Idempotent uploads.** Batch ids mean retries are safe. +- **Lease expiry, not lease loss.** If a worker goes quiet, its leases simply + expire and the work flows to others; nothing is stuck. +- **Graceful shutdown.** On `SIGTERM` the worker releases its leases + (`POST /assignments/release`), flushes its upload queue, and exits — so a + restart or `vapn pause` hands work back cleanly. + +## What a stored observation looks like + +``` +worker_id assignment_id target probe measured_at ok rtt_ms sig +9f30… 812 203.0.113.7 icmp 2026-07-18T08:04:31Z t 22.9 … +``` + +Immutable, signed, internal. It will influence a public verdict only *after* +being combined with many others into [consensus](trust-calculation.md) — never +on its own. + +Related: [worker lifecycle](../worker/lifecycle.md) · +[trust calculation](trust-calculation.md) · +[API reference — Coordinator](../api/README.md#b-coordinator-api). diff --git a/docs/walkthroughs/snapshot-publishing.md b/docs/walkthroughs/snapshot-publishing.md new file mode 100644 index 0000000..ea79580 --- /dev/null +++ b/docs/walkthroughs/snapshot-publishing.md @@ -0,0 +1,98 @@ +# Walkthrough: Snapshot Publishing + +A single **builder** run, start to finish — how raw RIPE routing data becomes a +signed target list workers can trust. This expands +[Stages 3–5](end-to-end.md#stage-3--the-builder-downloads-ripe-routing-data) of +the end-to-end flow. Conceptual background: +[RIPE/RIS/MRT](../concepts/ripe-and-ris.md) and +[prefix ownership](../concepts/prefix-ownership.md). To run the builder, see the +[builder guide](../builder/README.md). + +```mermaid +flowchart TD + A["Sync monitored ASN list
(VPS Advisor Provider API)"] --> B["Fetch RIS bview MRT dump
(or pre-downloaded data/ripe/)"] + B --> C["Parse MRT, keep only
prefixes from monitored ASNs"] + C --> D["Deduplicate"] + D --> E["Validate: bogon filter,
MOAS conflict flagging"] + E --> F["Enrich: GeoIP
(country/city/coords)"] + F --> G["Load into routing.* schema
[status: building]"] + G --> H["Derive probe targets
(capped per provider)"] + H --> I{"Sanity gate:
prefix-count swing
within threshold?"} + I -->|no| HOLD["Hold for admin approval"] + I -->|yes| J["Export signed SQLite artifact
+ manifest (sha256, Ed25519)"] + J --> K["Upload to artifact store,
verify readback"] + K --> L["Mark published;
previous → superseded"] + L --> M["Coordinator advertises
new version in heartbeats"] +``` + +## The run, stage by stage + +1. **Sync monitored ASNs.** The builder calls VPS Advisor for the current ASN + list. Only these ASNs' prefixes will survive the filter — VAPN never builds a + database of the whole Internet. + +2. **Fetch the RIS dump.** A full-table `bview` MRT dump from RIPE RIS (`rrc00` + by default). It checks the dump's age against `VAPN_RIS_BVIEW_MAX_AGE`; in + development it uses the pre-downloaded `data/ripe/latest-bview.gz`. + +3. **Parse + filter.** It streams ~1M MRT records (via + `internal/routing/mrtreader`) and keeps only those whose **origin AS** is + monitored. This is the big reduction: a million routes down to a few thousand. + +4. **Deduplicate.** Collapse the many per-peer reports of each + `(prefix, origin AS)` into single facts. + +5. **Validate.** Drop **bogons** (private/reserved/absurd) via + `internal/routing/bogon`; **flag MOAS conflicts** instead of resolving them. + +6. **Enrich with GeoIP.** Look up each prefix in GeoLite2-City for + country/city/coordinates (from the builder's local copy, refreshed on its own + cadence with the deployer's MaxMind key). + +7. **Load into PostgreSQL** under a new `routing.snapshot` row, status + `building`. Prefixes are per-snapshot and immutable — diffing snapshots later + yields routing-churn signals. + +8. **Derive probe targets.** Choose representative addresses per prefix/region, + each with a recorded rationale, capped at `MAX_TARGETS_PER_PROVIDER` + (default 100). + +9. **Sanity gate.** If the prefix count changed by more than + `VAPN_SANITY_MAX_DELTA_PCT` (default 50%) versus the last snapshot, **hold** + for admin approval — a wild swing may mean a route leak poisoned the data. + `VAPN_SANITY_FORCE=true` overrides (use carefully). See + [risk R-routing](../architecture/08-risk-assessment.md). + +10. **Export + sign.** Write the worker-facing subset to a compact **SQLite** + artifact (`prefixes`, `targets`, `meta`) and a manifest with counts, sha256, + and an **Ed25519 signature** over the manifest, plus `min_worker_version`. + +11. **Publish atomically.** Upload artifact + manifest to the store, verify the + readback, mark the snapshot `published`, previous → `superseded`. If *any* + step failed, mark it `failed` and leave the previous published version fully + in force — workers never see a half-built snapshot. + +12. **Advertise.** The coordinator picks up the new `published` version and + advertises it in heartbeats; workers download and verify it + ([auth walkthrough](worker-authentication.md)); the scheduler drains + assignments on removed targets and issues new ones. + +## Why it's designed this way + +- **One place parses MRT.** Expensive, security-sensitive parsing happens once, + centrally, auditable — never on thousands of workers. +- **Atomic publish + failed-stays-safe.** Workers always run on a complete, + verified snapshot; a broken build is a no-op, not an outage. +- **Signed artifacts over an untrusted store.** The artifact store needs no + trust; the signature is the integrity guarantee, so it can sit behind any CDN. +- **Sanity gate + rollback.** Bad routing data (leaks, hijacks) can't silently + redirect the fleet; a human is in the loop for anomalous builds, and + `vapnctl snapshots rollback ` re-points to a known-good snapshot. + +## Cadence + +Routing snapshots build **daily** (RIS `bview` is 8-hourly, but daily is plenty +for prefix membership, and churn comes from diffs). GeoIP refresh is an +**independent weekly** job — a GeoIP update never triggers a routing rebuild and +vice versa. Full detail: +[snapshot lifecycle](../architecture/06-lifecycles.md#2-snapshot-lifecycle). diff --git a/docs/walkthroughs/software-updates.md b/docs/walkthroughs/software-updates.md new file mode 100644 index 0000000..d8f773d --- /dev/null +++ b/docs/walkthroughs/software-updates.md @@ -0,0 +1,79 @@ +# Walkthrough: Software Updates + +How a worker updates safely, and how the platform retires old worker versions +without cutting anyone off. Operator-facing how-to: +[Getting Started → Updating](../getting-started/updating.md). + +## Worker self-update: health-gated with auto-rollback + +`vapn update` (manual) and `vapn update --auto` (the systemd timer) run the same +safe procedure: + +```mermaid +flowchart TD + A[Record current image as 'previous'] --> B[Pull latest worker image] + B --> C[Restart worker with new image] + C --> D{Healthy within 2 min?} + D -->|yes| E[Keep new image, done] + D -->|no| F[Roll back to previous image] + F --> G[Worker keeps running old, working version] +``` + +Steps: + +1. **Remember the current image** so rollback is possible. +2. **Pull** the latest tag. +3. **Restart** the worker container on the new image. +4. **Health-gate**: wait up to two minutes for the worker to report healthy + (via its heartbeat/health check). +5. **Decide**: healthy → keep it; not healthy → **automatically roll back** to + the previous image. + +The guarantee: **you cannot break a working worker by updating it.** A bad +release simply rolls back and the worker keeps contributing on the last good +version, while its logs capture why the new one failed. + +The auto-update timer runs daily at a **randomized hour** so the whole fleet +doesn't restart in lockstep (which would briefly dent coverage and hammer the +registry). See [`deploy/worker/vapn-update.timer`](../../deploy/worker/vapn-update.timer). + +## Platform-side: minimum version enforcement + +The platform can require a **minimum worker version** and drain anything older — +the lever for sunsetting buggy or insecure releases fleet-wide. + +```mermaid +sequenceDiagram + participant W as Worker (old version) + participant C as Coordinator + W->>C: heartbeat (version = 1.1.2) + C->>C: 1.1.2 < min_worker_version (1.2.0)? + C-->>W: control action: upgrade_required + Note over W,C: leases drained, new leases refused + W->>W: vapn update (or auto-timer) → 1.2.0 + W->>C: heartbeat (version = 1.2.0) + C-->>W: normal operation resumes +``` + +- Every **heartbeat reports the worker's version**. +- If it's below the minimum, the coordinator responds `upgrade_required`: the + worker is **drained** (its assignments move to current workers) and **refused + new leases** until it upgrades. It's not banned — it just can't contribute + until current. This shows up in `vapn status` and `vapn logs`. +- The minimum is also baked into each **snapshot manifest** + (`min_worker_version`) so a worker too old to parse a new artifact won't try. + +The two mechanisms compose: the platform *asks* old workers to upgrade +(min-version drain), and the worker *can* upgrade safely and unattended +(health-gated auto-update). Together they let the fleet move forward without an +operator having to babysit anything and without a bad release taking workers +down. + +## Updating the platform's own services + +Worker updates are one thing; upgrading the coordinator/aggregator/builder is an +operator task with its own rules — backward-compatible migrations +(expand→migrate→contract), rolling deploys, and workers tolerating brief +coordinator downtime by queueing observations locally. That's covered in +[Operations → Upgrades](../operations/upgrades.md) and +[release management](../operations/release-management.md). diff --git a/docs/walkthroughs/trust-calculation.md b/docs/walkthroughs/trust-calculation.md new file mode 100644 index 0000000..59d8ddd --- /dev/null +++ b/docs/walkthroughs/trust-calculation.md @@ -0,0 +1,143 @@ +# Walkthrough: Trust Calculation + +How a worker's **trust score** is computed, window by window, with the actual +formula and worked numbers. This expands +[Stages 9–10](end-to-end.md#stage-9--aggregation-computes-consensus) of the +end-to-end flow and makes concrete the +[trust concept](../concepts/measurement-and-consensus.md#trust). Implementation: +`internal/trust` (scoring) and `internal/aggregate` (agreement + consensus). + +## The two pipelines involved + +Trust emerges from two cooperating pipelines that run each window (default +5 min): + +1. **Consensus** (`aggregate.ComputeWindow`) settles what actually happened — + per target, whether it was "up" by trust-weighted vote — and records how well + **each worker agreed** with that settled result into + `aggregation.worker_agreement`. +2. **Scoring** (`trust.ComputeAll`) recomputes every worker's overall trust from + its recent agreement plus availability, tenure, and penalties. + +```mermaid +flowchart LR + OBS[Raw observations] --> CW["ComputeWindow:
settle consensus per target"] + CW --> WA["Record per-worker agreement
vs settled result"] + WA --> TC["ComputeAll:
recompute trust scores"] + TC -->|"weights next window's"| CW +``` + +That feedback loop is the whole idea: consensus judges workers; the resulting +trust weights the next consensus. + +## Step 1 — agreement against the settled window + +For each target in the window, consensus computes whether it was **up** +(≥ 50% of trust weight saw it reachable). Then, for each worker, it measures how +close that worker's own ok-ratio was to the settled truth: + +``` +agreement_for_target = 1 − |worker_ok_ratio − settled_up(0 or 1)| +``` + +A worker averages this across all targets it measured, and the result is stored +per window. Examples for one target that settled **up**: + +| Worker saw | ok_ratio | Settled | Agreement | +|---|---|---|---| +| reachable every time | 1.0 | 1 (up) | **1.00** | +| reachable 3 of 4 | 0.75 | 1 (up) | 0.75 | +| unreachable every time | 0.0 | 1 (up) | **0.00** | + +Crucially this is scored against the **settled** window, not the instantaneous +majority — a worker that correctly detected an outage a little early still +matches the final settled result and is *not* penalized for being ahead of the +crowd. + +## Step 2 — the four components + +`trust.ComputeAll` recomputes every non-retired worker's score from four +components (all in [0,1] except penalty which subtracts): + +| Component | How it's computed | Intuition | +|---|---|---| +| **agreement** | mean `worker_agreement` over the last 24 h of settled windows; **0.5** (neutral) if the worker has no history yet | did you match reality? | +| **availability** | heartbeat recency: **1.0** if seen < 5 min ago, **0.5** if < 1 h, else **0.0** | are you actually online? | +| **tenure** | `d / (d + 14)` where `d` = days since approval | how long have you been around? | +| **penalty** | `0.1 ×` count of `bad_signature`/`replay` events in the last 7 days, capped at **0.5** | are you misbehaving? | + +## Step 3 — the formula + +``` +score = clamp( availability × (0.2 + 0.3 × tenure) + 0.5 × agreement − penalty , 0, 1 ) +``` + +Read it as: agreement is the dominant term (weight 0.5); availability gates a +tenure-scaled base (a worker that isn't online contributes nothing from that +term); penalties bite directly. + +### Worked examples + +**A brand-new, honest, online worker** (approved today, no history, no +penalties): + +``` +availability = 1.0 +tenure = 0 / (0 + 14) = 0.0 +agreement = 0.5 (neutral default) +penalty = 0.0 +score = 1.0 × (0.2 + 0.3×0.0) + 0.5×0.5 − 0 = 0.20 + 0.25 = 0.45 +``` + +It starts mid-range but low-ish — enough to contribute a little (consensus gives +every worker a weight floor of 0.1), not enough to dominate. + +**A seasoned, reliable worker** (approved 60 days ago, always agrees, always +online): + +``` +availability = 1.0 +tenure = 60 / (60 + 14) ≈ 0.81 +agreement = ~1.0 +penalty = 0.0 +score = 1.0 × (0.2 + 0.3×0.81) + 0.5×1.0 = 0.443 + 0.5 = 0.94 +``` + +Near the top — this worker's vote carries real weight. + +**A worker with a wrong clock** (submitting replays/bad signatures): + +``` +availability = 1.0 +tenure = 0.81 +agreement = 0.9 +penalty = min(0.5, 0.1 × 6 events) = 0.5 +score = 0.443 + 0.45 − 0.5 = 0.39 +``` + +The penalty drags an otherwise-good worker down and, if it persists, triggers +quarantine. + +## Step 4 — how trust is *used* + +- **Weight in consensus** = the worker's score, floored at 0.1 for `active` + workers so newcomers count a little; **0 for any non-`active` state**. +- **Per-operator cap** limits how much total weight one operator's workers can + contribute — blunting Sybil attacks even further. +- **Quarantine (shadow mode)**: a worker whose trust collapses keeps measuring + at weight 0 and rebuilds agreement over subsequent windows — it can earn its + way back without an admin, though only an admin can `retire` or `reinstate`. + +## Why these choices + +| Choice | Defends against | +|---|---| +| Agreement scored vs *settled* window | Punishing correct-but-early workers | +| Tenure ramp (`d/(d+14)`) | Sybil attacks — fresh workers can't buy influence | +| Neutral 0.5 for no-history workers | Neither trusting nor distrusting the unknown | +| Penalty decays over 7 days | One bad day shouldn't be permanent; sustained abuse is | +| Zero weight unless `active` | Suspended/pending/retired workers can't sway results | +| Per-operator weight cap | One actor running many honest-looking nodes | + +The design record is [Security & trust model §4](../architecture/05-security-trust-model.md#4-trust-model); +the concept is [trust](../concepts/measurement-and-consensus.md#trust). diff --git a/docs/walkthroughs/worker-authentication.md b/docs/walkthroughs/worker-authentication.md new file mode 100644 index 0000000..3a14239 --- /dev/null +++ b/docs/walkthroughs/worker-authentication.md @@ -0,0 +1,121 @@ +# Walkthrough: Worker Authentication + +How VAPN knows a request really came from a specific worker, hasn't been +altered, and isn't a replay — using **Ed25519 request signing**. This is the +security backbone of the worker↔coordinator channel. Design reference: +[Security & trust model](../architecture/05-security-trust-model.md); the code +is `internal/wireauth`. + +## Why signing, on top of HTTPS? + +HTTPS encrypts the channel and authenticates the *server* to the worker. It does +**not**, on its own, prove which *worker* sent a request or stop a captured +request from being replayed. VAPN assumes workers can be malicious or +compromised, so every request carries a per-worker **signature** in addition to +running over HTTPS. Two independent guarantees: + +- **HTTPS**: nobody on the wire can read or tamper in transit. +- **Ed25519 signature**: the coordinator can prove *this exact request* was + produced by the holder of *this worker's* private key, and that the body + wasn't changed. + +## The keypair + +At first boot the worker generates an **Ed25519 keypair** (a modern, fast, +32-byte public-key signature scheme). The **private key never leaves** the +worker's volume. The **public key** is registered with the coordinator during +[enrollment](worker-installation.md). From then on the worker signs; the +coordinator verifies with the stored public key. + +## Every request is signed + +Every worker request except `register` carries these headers: + +``` +X-Worker-Id: +X-Timestamp: +X-Nonce: <128-bit random, unique per request> +X-Signature: Ed25519( method | path | timestamp | nonce | sha256(body) ) +``` + +The coordinator verifies by: + +1. Looking up the worker's currently-valid public key. +2. Recomputing the signed string from the request and checking the signature. +3. Checking the **timestamp** is within the ±120 s window (clock-skew tolerant, + replay-hostile). +4. Checking the **nonce** hasn't been seen for this worker within the window + (`registry.replay_nonce`, TTL-pruned). + +Any failure → the request is rejected (`401` bad signature/expired, +`409` replayed nonce) **and** a `bad_signature` or `replay` **trust event** is +recorded, which lowers the worker's [trust](trust-calculation.md). + +```mermaid +sequenceDiagram + participant W as Worker + participant C as Coordinator + W->>W: build signed string:
method|path|ts|nonce|sha256(body) + W->>W: sign with private key + W->>C: request + X-Worker-Id/Timestamp/Nonce/Signature + C->>C: load worker's public key + C->>C: verify signature + C->>C: check timestamp within ±120s + C->>C: check nonce unused in window + alt all pass + C-->>W: 200 (process request) + else any fail + C-->>W: 401 / 409 + record trust event + end +``` + +## Why timestamp *and* nonce? + +They defend different replays: + +- **Timestamp** bounds how long any captured request is usable — outside ±120 s + it's dead. It also stops an attacker from replaying *old healthy* observations + during an outage, because observations bind `measured_at`. +- **Nonce** stops replay *within* the window — the same request can't be + submitted twice even a second apart. + +Together they give tight replay protection without requiring perfectly +synchronized clocks (hence the worker's clock check at install time). + +## Observations are individually signed too + +Beyond the request signature, **each observation inside a batch is also signed**. +So even if a measurement were ever relayed through some future ingestion path, +its per-measurement provenance survives — you can always prove which worker +produced which data point. This is why raw observations remain attributable +forever in the audit trail. + +## Key rotation + +Keys can be rotated without downtime: + +1. The worker submits its **next** public key, signed by its **current** key + (`POST /api/v1/workers/keys/rotate`). +2. Both keys are valid during a bounded **overlap window** (`registry.worker_key` + allows one current + overlapping keys), so in-flight requests don't fail. +3. After the overlap, only the new key is valid. + +Rotation happens on a routine schedule and can be **demanded by the server** — +a heartbeat control action forces rotation on suspected compromise +(`vapnctl workers rotate-key `). **Revocation** is the sharp lever: revoking +all of a worker's keys cuts it off instantly, paired with a state transition to +`suspended` or `quarantined`. + +## What this buys the system + +| Guarantee | Mechanism | +|---|---| +| Only the real worker can act as itself | Private key never leaves the worker; signature verified against registered public key | +| Requests can't be tampered in flight | Signature covers method, path, and body hash | +| Captured requests can't be replayed | Timestamp window + per-worker nonce uniqueness | +| Compromise is recoverable | Rotation (overlap) + instant revocation + state transition | +| Attacks are visible and penalized | Failures recorded as trust events, surfaced in audit + telemetry | + +Related: [enrollment](worker-installation.md) · +[trust calculation](trust-calculation.md) · +[Security & trust model](../architecture/05-security-trust-model.md). diff --git a/docs/walkthroughs/worker-installation.md b/docs/walkthroughs/worker-installation.md new file mode 100644 index 0000000..002fa3a --- /dev/null +++ b/docs/walkthroughs/worker-installation.md @@ -0,0 +1,91 @@ +# Walkthrough: Worker Installation + +What *actually* happens between running the installer and having a probing +worker. This traces [Stage 6](end-to-end.md#stage-6--workers-detect-verify-and-download-it) +of the end-to-end flow from the worker's side. For the operator-facing how-to, +see [Getting Started → Installation](../getting-started/installation.md). + +```mermaid +sequenceDiagram + participant U as Operator (browser) + participant S as VPS Advisor + participant I as install.sh / vapn CLI + participant W as Worker container + participant C as Coordinator + + U->>S: Create worker (My Workers) + S-->>U: One-time enrollment token + U->>I: run install.sh + paste token + I->>I: check Docker, disk, clock, coordinator + I->>W: start container with token + coordinator URL + W->>W: generate Ed25519 keypair (private key stays local) + W->>C: POST /register (token + public key + facts) + C->>S: verify token hash against pending enrollment + C-->>W: worker_id, state = pending + Note over W,C: worker heartbeats; no work until approved + U->>S: Admin approves worker + S-->>C: decision synced (state = active) + C-->>W: next heartbeat: active + snapshot version + W->>W: download + verify snapshot, lease work, probe +``` + +## Step by step + +1. **Operator creates the worker on VPS Advisor.** The website generates a + random ≥32-byte **enrollment token**, shows it once, and stores only its + sha256. A `monitoring_worker` row exists in state `pending`. + +2. **The installer bootstraps the CLI.** `install.sh` verifies Docker, detects + the architecture, downloads the `vapn` binary from GitHub releases, and hands + off to `vapn install`. → [Installer source](../../deploy/worker/install.sh). + +3. **System checks run** (`vapn doctor`): Docker reachable, disk space, + coordinator reachable over HTTPS, clock synchronized. Each failure blocks + with a specific fix. + +4. **The worker generates its identity.** On first boot it creates an **Ed25519 + keypair**. The **private key never leaves** the `~/.vapn` volume — not in + logs, not to the coordinator, never. This is what makes measurements + attributable to *you* without the platform being able to impersonate you. + +5. **The worker registers.** `POST /api/v1/workers/register` with the enrollment + token, its public key, and facts (version, self-reported location). The + coordinator checks the token hash against VPS Advisor's pending enrollments, + creates the worker record, and returns a `worker_id` in state `pending`. The + token is now spent. + +6. **It waits, heartbeating.** A `pending` worker may heartbeat but gets no + assignments and contributes zero weight. The operator sees "awaiting + approval." + +7. **An admin approves it** on VPS Advisor. That decision syncs to the + coordinator (poll every ~2 min, or webhook), which flips the worker to + `active`. + +8. **The worker goes to work.** On its next heartbeat it's told it's active and + which snapshot version is current. It downloads and cryptographically + verifies the snapshot ([auth walkthrough](worker-authentication.md)), leases + assignments, and starts probing ([measurement walkthrough](measurement-lifecycle.md)). + +## Why enrollment is split this way + +Enrollment deliberately spans two systems: the **operator identity and approval +live on VPS Advisor** (humans, accounts, anti-abuse), while the **worker +operates against the coordinator** (high-volume, machine-facing). The one-time +token is the bridge — it lets the coordinator trust "a real operator started +this" without the worker ever holding a website session. See the +[API plane split](../architecture/01-system-architecture.md#2-api-plane-split-clarification-of-the-brief). + +## What's on disk afterward + +Everything lives under `~/.vapn` (override with `VAPN_HOME`): + +| Path | Contents | +|---|---| +| `config.env` | coordinator URL, enrollment token (once), settings | +| `state/` | identity (private key!), downloaded snapshot, upload queue | +| `docker-compose.yml` | the generated worker service definition | + +Back it up with `vapn backup` (it contains your private key — guard it). +Related: [command reference](../worker/command-reference.md) · +[worker lifecycle](../worker/lifecycle.md). diff --git a/docs/worker/README.md b/docs/worker/README.md index ea0813b..76f0db2 100644 --- a/docs/worker/README.md +++ b/docs/worker/README.md @@ -1,89 +1,74 @@ -# Running a VAPN Worker +# Community Workers -Thank you for contributing measurement capacity to the VPS Advisor Probe -Network. A worker is a small Docker container that pings VPS providers' -public networks from your vantage point and reports signed measurements. -It uses a few MB of RAM, negligible CPU, and a trickle of bandwidth. +A **worker** is a small program you run that measures VPS providers' public +network health from your location and reports signed results back to the +network. Thousands of workers, run by the community from many places, are what +make VAPN's verdicts trustworthy — no single one is trusted on its own. -**You stay in control**: pause, resume, or remove the worker at any time. -Providers being measured can opt out at any time too — the target list your -worker uses comes exclusively from providers listed on VPS Advisor. +Thank you for considering running one. This section is everything you need, +from "what is it" to "how do I leave." -## Install +> **Just want it running?** → [Quick Start](../getting-started/quickstart.md) +> (a worker in ~5 minutes). This section is the reference behind it. -Requirements: Linux (amd64/arm64), Docker, and an **enrollment token** from -your VPS Advisor dashboard (My Workers → Create worker). +## What a worker is, in one paragraph -```sh -curl -fsSL https://install.vpsadvisor.com | bash -``` - -The installer detects Docker, downloads the `vapn` CLI, runs the system -checks, asks for your token, starts the worker, and verifies it comes up: +It's a Docker container that, once approved, receives **assignments** from the +platform (never chooses its own targets), sends a few lightweight +[ICMP "ping"](../concepts/measurement-and-consensus.md#what-a-single-measurement-is) +packets to the assigned provider addresses, times the replies, cryptographically +signs the results with a key that never leaves your machine, and uploads them. +It uses a few MB of RAM, negligible CPU, and a trickle of bandwidth. **You stay +in control**: pause, resume, or remove it whenever you like. -``` -✓ Docker detected ✓ Coordinator reachable -✓ Disk space ✓ Clock synchronization -✓ Registration successful -Worker ID: 9f30… -Status: Awaiting approval -``` +## This section -Approval is manual (an admin reviews new workers); your worker begins -probing automatically once approved — nothing more to do. - -## Day to day +| Guide | What it covers | +|---|---| +| [Installation](installation.md) | Requirements and the full install flow (also: [Quick Start](../getting-started/quickstart.md)) | +| [Command reference](command-reference.md) | Every `vapn` command with examples | +| [Worker lifecycle](lifecycle.md) | Every state a worker moves through and why | +| [Resource usage & privacy](resources-and-privacy.md) | Exactly what it costs you and what it does (and doesn't) see | +| [Updating](../getting-started/updating.md) | Keeping it current, safely | +| [Uninstalling](../getting-started/uninstalling.md) | Leaving cleanly | +| [Troubleshooting](../getting-started/troubleshooting.md) · [FAQ](../getting-started/faq.md) | When something's off | + +## Why run one? + +- **Help buyers make better decisions.** Your vantage point adds a real, + independent data point about provider health from your part of the world. +- **It's nearly free to run.** A spare VPS or home server has plenty of headroom; + see [resource usage](resources-and-privacy.md#resource-usage). +- **It's safe and private by design.** Your worker only probes addresses from a + signed, platform-provided list, reports nothing about your machine beyond its + own liveness and version, and your private key never leaves your host. See + [privacy](resources-and-privacy.md#privacy). + +## The short version of day-to-day ```sh vapn status # health, snapshot, assignments, measurements submitted vapn logs -f # live logs -vapn pause # stop probing (your assignments shift to other workers) +vapn pause # stop probing (identity + trust kept; resume any time) vapn resume -vapn update # pull latest image; health-gated with automatic rollback +vapn update # health-gated update with automatic rollback vapn doctor # re-run the system checks -vapn backup # archive identity + config (contains your private key!) +vapn uninstall # remove everything (offers a clean unregister) ``` -Updates are automatic if you enable the timer (the installer offers it, or -see `deploy/worker/vapn-update.*`): daily check, randomized hour, and if an -updated worker fails to report healthy within two minutes it is rolled back -to the previous image automatically. +Full details: [command reference](command-reference.md). -Everything lives in `~/.vapn`; the worker is self-managing — it renews -credentials, downloads new routing snapshots (cryptographically verified -against a pinned key), retries failed uploads, and recovers from reboots -(`restart: unless-stopped`) and network interruptions on its own. You -should rarely need to SSH in. +## How your worker earns its place -## Leaving +New workers are **manually approved** (anti-abuse) and then build **trust** over +time by agreeing with the consensus of other workers. Trust determines how much +your measurements count. Misbehaving workers (bad clocks, tampered binaries) +lose trust and are quarantined automatically. None of this needs your attention +in the normal case — but if you're curious how it works, read +[the trust concept](../concepts/measurement-and-consensus.md#trust) and the +[lifecycle](lifecycle.md). -```sh -vapn uninstall -``` - -Offers to unregister your worker with the network (recommended) and to keep -a backup, then removes containers, images, and `~/.vapn`. No orphaned -resources, no hard feelings — thanks for the packets. - -## Privacy & conduct - -- The worker only probes IP addresses from the signed routing snapshot — - never targets of its own choosing; ICMP echo only, a few packets per - minute per target. -- It reports measurements, its version, and liveness. It does not read - anything else on your machine. -- Your worker's key never leaves your machine; the platform can verify your - measurements but cannot impersonate you. -- Misbehaving workers (bad clocks, tampered binaries) lose trust weight and - are quarantined server-side — if that happens to you, `vapn doctor` and - `vapn logs` usually explain why. - -## Troubleshooting - -| Symptom | Try | -|---|---| -| `Awaiting approval` for long | Approval is human; check your VPS Advisor dashboard | -| `Unreachable (last report …)` in status | `vapn logs`, then `vapn doctor` | -| Clock check fails | Enable NTP (`timedatectl set-ntp true`) | -| Docker permission errors | Add your user to the `docker` group, re-login | -| Behind strict egress firewall | Allow HTTPS (443) to the coordinator domain | +Everything lives in `~/.vapn`; the worker is self-managing — it renews +credentials, downloads and verifies new routing snapshots, retries failed +uploads, and recovers from reboots on its own. You should rarely need to SSH in. +Thanks for the packets. diff --git a/docs/worker/command-reference.md b/docs/worker/command-reference.md new file mode 100644 index 0000000..f86c6ee --- /dev/null +++ b/docs/worker/command-reference.md @@ -0,0 +1,170 @@ +# Worker Command Reference + +Every `vapn` command, what it does, and when to use it. `vapn` is the small CLI +the installer places on your host; it wraps Docker so you never have to think +about containers directly. + +Run `vapn` with no arguments (or `vapn help`) to print this list. The worker's +home directory is `~/.vapn` (override with `VAPN_HOME`). + +## Quick index + +| Command | One-liner | +|---|---| +| [`install`](#install) | Set up and start a worker (interactive) | +| [`status`](#status) | Worker health at a glance | +| [`doctor`](#doctor) | Run the system checks | +| [`logs`](#logs) | Show worker logs (`-f` to follow) | +| [`pause`](#pause) | Stop probing, keep identity | +| [`resume`](#resume) | Start probing again | +| [`update`](#update) | Update the worker image (health-gated, auto-rollback) | +| [`backup`](#backup) | Archive identity + config | +| [`unregister`](#unregister) | Permanently retire this worker with the network | +| [`uninstall`](#uninstall) | Remove everything | +| [`version`](#version) | Print the CLI version | + +--- + +## `install` + +```sh +vapn install +``` + +Interactive setup: runs the [system checks](#doctor), prompts for the +**coordinator URL** and **enrollment token**, writes `~/.vapn/config.env`, +generates the worker's keypair, registers with the coordinator, downloads and +verifies the routing snapshot, starts the container, and verifies it comes up. +Idempotent — safe to re-run to reconfigure. Usually invoked for you by +`install.sh`. → [Installation](installation.md). + +## `status` + +```sh +vapn status +``` + +A snapshot of worker health: running/paused state, current routing snapshot +version, number of held assignments, measurements submitted, last heartbeat, and +any pending control actions (e.g. `upgrade_required`). Your first stop when +checking in. → interpreting it: [lifecycle](lifecycle.md). + +## `doctor` + +```sh +vapn doctor +``` + +Re-runs the environment checks: Docker present and reachable, disk space, +coordinator reachable over HTTPS, clock synchronized. Each failure prints a +specific remedy. Run it whenever something looks off. → +[Troubleshooting](../getting-started/troubleshooting.md). + +## `logs` + +```sh +vapn logs # recent logs +vapn logs -f # follow live +``` + +Streams the worker container's logs. Safe to share when reporting a problem — +**the private key never appears in logs**. Look here for the last error line +when diagnosing. + +## `pause` + +```sh +vapn pause +``` + +Stops probing but **keeps your identity and accumulated trust**. Your +assignments redistribute to other workers within about a minute. Use this for +maintenance windows or when you need your bandwidth. Prefer this over uninstall +if you'll be back — resuming keeps your trust; reinstalling starts trust over. + +## `resume` + +```sh +vapn resume +``` + +Resumes probing after a `pause`. The worker re-leases assignments on its next +cycle. + +## `update` + +```sh +vapn update # manual, health-gated update +vapn update --auto # what the systemd timer runs (unattended) +``` + +Pulls the latest worker image, restarts, and waits for health. **If the new +version isn't healthy within two minutes, it automatically rolls back** to the +previous image — you can't break a working worker by updating. See +[Updating](../getting-started/updating.md) and the +[software-updates walkthrough](../walkthroughs/software-updates.md). + +## `backup` + +```sh +vapn backup +``` + +Archives your identity and configuration to a `tar.gz`. **The archive contains +your private key — treat it like a password.** Use it to move a worker to +another host or to restore the same identity (and its trust) instead of +enrolling fresh. + +## `unregister` + +```sh +vapn unregister +``` + +Tells the coordinator this worker is retiring: it's marked `retired`, keys +revoked, no more assignments — history retained for audit. The polite, immediate +way to leave. `uninstall` offers to do this for you. + +## `uninstall` + +```sh +vapn uninstall +``` + +Removes everything: containers, the worker image, and `~/.vapn`. Interactively +offers to [`unregister`](#unregister) first (recommended) and to keep a +[`backup`](#backup). → [Uninstalling](../getting-started/uninstalling.md). + +## `version` + +```sh +vapn version +``` + +Prints the `vapn` CLI version. (To see the running *worker image* version, use +[`status`](#status).) + +--- + +## Common recipes + +```sh +# First install, then confirm it's healthy and awaiting approval +vapn install && vapn status + +# Something's wrong — diagnose +vapn doctor ; vapn logs -f + +# Going away for a bit (keep identity/trust) +vapn pause # …later… vapn resume + +# Move this worker to a new machine +vapn backup # copy the tar.gz over, restore into ~/.vapn there + +# Leave for good, cleanly +vapn uninstall # accept the unregister prompt +``` + +Related: [lifecycle](lifecycle.md) · +[resource usage & privacy](resources-and-privacy.md) · +[FAQ](../getting-started/faq.md). diff --git a/docs/worker/installation.md b/docs/worker/installation.md new file mode 100644 index 0000000..13de4b7 --- /dev/null +++ b/docs/worker/installation.md @@ -0,0 +1,84 @@ +# Worker Installation (Details) + +The friendly step-by-step lives in +[Getting Started → Installation](../getting-started/installation.md) and +[Quick Start](../getting-started/quickstart.md). This page adds the container- +level details for people who want to know exactly what runs, or who install by +hand rather than through `vapn install`. + +## The three install methods (recap) + +Canonical source is **GitHub**. In increasing order of "show me exactly what +runs": + +1. **Piped:** `curl -fsSL | bash` +2. **Download, read, run:** save the script, `less` it, then `bash` it. +3. **Clone + build:** `git clone … && make build && ./bin/vapn install`. + +Full commands and reasoning: +[Getting Started → Choose how you install](../getting-started/installation.md#choose-how-you-install). + +## What the container actually looks like + +`vapn install` generates a `~/.vapn/docker-compose.yml` roughly like this: + +```yaml +services: + worker: + image: ghcr.io/hummingbytedev/vapn-worker:latest + cap_add: [NET_RAW] # the ONE capability it needs (ICMP) + environment: + VAPN_ENROLLMENT_TOKEN: "…" # spent on first boot + VAPN_COORDINATOR_URL: "https://probe-api.vpsadvisor.example" + # VAPN_MAXMIND_LICENSE_KEY: "…" # optional; your own key + volumes: [worker-state:/state] # identity + snapshot + upload queue + restart: unless-stopped # survives reboots +volumes: { worker-state: } +``` + +Key points: + +- **`cap_add: [NET_RAW]` and nothing else.** Sending an ICMP echo needs a raw + socket, which needs `CAP_NET_RAW`. The container is **not** privileged, does + not use host networking, and cannot see your files. See + [privacy](resources-and-privacy.md#privacy). +- **One persistent volume** holds the worker's identity (its private key), the + downloaded routing snapshot, and the local upload queue. Back it up with + `vapn backup`. +- **`restart: unless-stopped`** means the worker returns automatically after a + reboot or Docker restart. + +## Configuration surface + +A worker needs only two settings; everything else is automatic: + +| Variable | Required | Purpose | +|---|---|---| +| `VAPN_COORDINATOR_URL` | yes | The platform endpoint the worker talks to | +| `VAPN_ENROLLMENT_TOKEN` | yes (first boot) | One-time proof a real operator started it | +| `VAPN_MAXMIND_LICENSE_KEY` | no | Your own key for a local GeoIP DB (degrades gracefully without) | +| `VAPN_HOME` | no | Override the `~/.vapn` location | + +The complete list is in the +[configuration reference](../reference/configuration.md). + +## Requirements checklist + +| Requirement | Check | +|---|---| +| Linux amd64/arm64 | `uname -m` | +| Docker running for your user | `docker info` | +| Outbound HTTPS (443) to the coordinator | `curl -fsSI https:///healthz` | +| Synchronized clock | `timedatectl` shows NTP active | +| An enrollment token | VPS Advisor → My Workers → Create worker | + +Any of these failing is caught by `vapn doctor` with a specific fix. See +[Troubleshooting](../getting-started/troubleshooting.md). + +## After install + +- Worker is `pending` until [manually approved](lifecycle.md#pending). +- `vapn status` / `vapn doctor` to verify. +- Turn on [automatic updates](../getting-started/updating.md). +- Learn the [commands](command-reference.md) and the + [lifecycle](lifecycle.md). diff --git a/docs/worker/lifecycle.md b/docs/worker/lifecycle.md new file mode 100644 index 0000000..1248db2 --- /dev/null +++ b/docs/worker/lifecycle.md @@ -0,0 +1,121 @@ +# Worker Lifecycle + +Every state a worker moves through, what it can and can't do in each, and who +controls the transitions. Understanding this makes `vapn status`, approval +delays, and quarantine make sense. The design record is +[architecture 06 §1](../architecture/06-lifecycles.md#1-worker-lifecycle); the +concept is [trust & reputation](../concepts/measurement-and-consensus.md#reputation-and-the-worker-lifecycle). + +## The states at a glance + +```mermaid +stateDiagram-v2 + [*] --> pending: enroll (operator + token) + pending --> active: admin approves + active --> quarantined: trust collapse / anomaly (automatic) + quarantined --> active: recovers / admin reinstates + active --> suspended: admin suspends + suspended --> active: admin reinstates + active --> retired: admin retires + suspended --> retired + quarantined --> retired + retired --> [*] +``` + +| State | Can authenticate? | Gets assignments? | Weight in consensus | +|---|---|---|---| +| **pending** | register + heartbeat only | none | 0 | +| **active** | full | yes | trust-based | +| **suspended** | rejected (403) | none (leases revoked) | 0 | +| **quarantined** | full | yes (shadow) | 0 — results recorded to rebuild trust | +| **retired** | rejected, keys revoked | none | 0, forever | + +## pending + +Where every worker starts, right after [enrollment](../walkthroughs/worker-installation.md). +It can heartbeat (so the platform knows it's alive) but receives no work and +counts for nothing until a **human admin approves it**. This manual gate is a +deliberate anti-abuse step — it's the main thing stopping someone from flooding +the network with fake workers. + +*What you see:* `vapn status` shows "Awaiting approval." *What to do:* nothing — +check your VPS Advisor dashboard for the approval decision. + +## active + +The normal, healthy state. The worker leases assignments, probes, uploads signed +measurements, and its measurements count in [consensus](../concepts/measurement-and-consensus.md#consensus-from-many-views-to-one-verdict) +weighted by its [trust](../concepts/measurement-and-consensus.md#trust). Trust rises as it agrees with the +crowd and stays online; it can fall on misbehavior. + +*What you see:* assignments held, measurements climbing, healthy heartbeats. + +## quarantined (shadow mode) + +Entered **automatically** when a worker's trust collapses or it triggers +anomalies (a wrong clock, tampered binary, or consistent disagreement with +consensus). A quarantined worker keeps measuring — but at **weight 0**, so its +results can't affect public verdicts. Those results are still recorded and +scored for agreement, so the worker can **earn its way back** to `active` as its +agreement recovers over subsequent windows. + +*What you see:* `vapn status`/`vapn logs` explain why (usually a clock or +version issue). *What to do:* fix the underlying cause; trust recovers on its +own. Only an admin can force reinstatement early. + +## suspended + +An **admin action** (not automatic): the worker is locked out entirely — its +requests are rejected (`403`) and its leases revoked. Used when a worker or +operator needs to be stopped immediately. Reversible: an admin can reinstate it +to `active`. + +## retired + +Terminal. An **admin** (or the operator via `vapn unregister`) retires the +worker: keys revoked, no further work, history kept for audit. There's no path +back — a retired worker's operator installs a *new* worker (new identity, +trust from scratch) if they return. + +## Who controls transitions + +The governing rule: **administrators outrank automation.** + +| Transition | Trigger | +|---|---| +| pending → active | admin approval (human) | +| active → quarantined | automatic (trust collapse / anomaly) | +| quarantined → active | automatic recovery **or** admin reinstate | +| active/quarantined → suspended | admin | +| suspended → active | admin | +| any → retired | admin, or operator's `vapn unregister` | + +Automation may *quarantine* (a reversible, zero-weight holding state) but never +*retires* or *reinstates* on its own — those decisions are always a human's. + +## The `unreachable` attribute (not a state) + +Separately from these states, a worker that stops heartbeating is flagged +**`unreachable`** on the admin dashboard. This is an *attribute*, not a lifecycle +state — an `active` worker that goes silent is still `active`, just unreachable, +and resumes normally when it comes back. Its leases expire and the work flows to +others in the meantime. + +## Operator controls vs platform states + +Your `vapn` commands map onto this lifecycle: + +| You run | Effect on lifecycle | +|---|---| +| `vapn install` | Creates a worker in `pending` | +| `vapn pause` / `resume` | No state change — just stops/starts probing locally (keeps `active` + trust) | +| `vapn unregister` | Moves the worker to `retired` | +| `vapn uninstall` | Offers `unregister` (→ `retired`), then removes everything locally | + +Note that **pausing is not suspension**: pause is your local choice and +preserves everything; suspension is an admin lockout. Choose `pause` for +temporary breaks. + +Related: [command reference](command-reference.md) · +[trust calculation](../walkthroughs/trust-calculation.md) · +[security & trust model](../architecture/05-security-trust-model.md). diff --git a/docs/worker/resources-and-privacy.md b/docs/worker/resources-and-privacy.md new file mode 100644 index 0000000..fc3306e --- /dev/null +++ b/docs/worker/resources-and-privacy.md @@ -0,0 +1,100 @@ +# Worker Resource Usage & Privacy + +The two questions every prospective worker operator asks: **"What will it cost +me?"** and **"What can it see?"** This page answers both plainly. If anything +here doesn't match what you observe, that's a bug worth reporting. + +## Resource usage + +A worker is intentionally tiny. It's a single static Go binary in a minimal +container that spends almost all its time asleep between probes. + +| Resource | Typical usage | Notes | +|---|---|---| +| **Memory** | a few MB | No routing files to hold — the worker carries only a small signed SQLite target list | +| **CPU** | negligible | Sending pings and signing small batches; idle otherwise | +| **Bandwidth** | a trickle | A few ICMP packets per minute per assigned target, plus small signed uploads and a ~30 s heartbeat | +| **Disk** | tens of MB | The worker image, the current snapshot, and a bounded local upload queue | +| **Network ports** | **none inbound** | The worker only makes *outbound* HTTPS (443) connections; it opens no listening ports | + +### Why it's so light + +- **Workers never parse routing data.** The heavy lifting (parsing ~1M MRT + records) happens once in the [builder](../builder/README.md); workers get a + pre-digested target list. → [why workers never touch MRT](../concepts/ripe-and-ris.md#why-workers-never-touch-mrt) +- **Probes are small and paced.** ICMP echo is a handful of tiny packets, and + probe intervals are set by policy — the platform actively prevents anything + that would resemble a scan or abuse. +- **Uploads are batched.** Observations are collected and sent roughly once a + minute, signed, in one request — not one request per probe. + +### Controlling the load + +- `vapn pause` stops all probing instantly (and returns your assignments to the + fleet) — use it when you need your bandwidth. +- The platform caps assignments per worker (`MAX_ASSIGNMENTS_PER_WORKER`) and + enforces per-target and per-worker rate limits via **probe policy**, so a + worker can't be handed an abusive amount of work. + +## Privacy + +This is the part people care about most, so here it is directly. + +### What the worker does + +- **Probes only platform-provided targets.** It sends ICMP echo ("ping") + packets **only** to addresses in the signed routing snapshot, which is derived + exclusively from [monitored providers' publicly announced routes](../concepts/prefix-ownership.md). + It **never chooses its own targets** and never scans. +- **Reports only its own measurements and liveness.** It uploads the probe + results (reachability, RTT), its software version, and a heartbeat. That's the + entire outbound story. + +### What the worker does *not* do + +- It does **not** read your files, your other network traffic, your processes, + or anything else on your machine. It runs as an unprivileged container with a + single Linux capability, `CAP_NET_RAW` (needed to send ICMP), and a single + small volume for its own state. +- It does **not** use host networking or open inbound ports. +- It does **not** phone home about your machine's identity beyond what's + inherent in making an outbound connection (your public IP, which the platform + geolocates to a region and source network — see below). + +### Your key stays yours + +The worker generates an **Ed25519** keypair on first boot and the **private key +never leaves your host** — not in logs, not to the coordinator, never. This is a +deliberate asymmetry: the platform can *verify* your measurements are yours, but +it **cannot impersonate you or forge measurements in your name**. If your key +were ever stolen, rotation and revocation limit the damage — see +[worker authentication](../walkthroughs/worker-authentication.md). + +### What the platform learns about you + +Only what's necessary to keep measurements fair and honest: + +- **Your public IP's region and source ASN** (via [GeoIP](../concepts/geoip.md)), + used to (a) spread each target's measurement across diverse networks and + (b) **stop you from measuring your own provider's network**. It's a coarse + network/region signal, not tracking. +- **Your operator account** (on VPS Advisor) — because a human vouches for each + worker. Per-operator weight caps mean one person can't dominate consensus. +- **Your worker's behavior** — trust events like invalid signatures or a wrong + clock, used to score [trust](../concepts/measurement-and-consensus.md#trust). + +### Data handling + +Raw measurements are **internal-only** and retained briefly (default ~14 days) +before being pruned; only aggregated, consensus-backed verdicts are ever +published, and those are about *providers*, not about you. See +[data retention](../architecture/03-database-design.md) and the +[security model](../architecture/05-security-trust-model.md). + +## The honest summary + +Running a worker costs you a few MB of RAM and a trickle of outbound bandwidth, +opens no ports, and lets a sandboxed container send pings to a fixed, signed +list of provider addresses — nothing more. Your private key and your machine's +contents stay yours. You can pause or remove it at any moment. If that trade +sounds fair, [run one](../getting-started/quickstart.md) — and thank you.