opdev · mrhillsman · May 22, 2026 · May 22, 2026 · May 22, 2026 · May 22, 2026
diff --git a/CLAUDE.md b/CLAUDE.md
@@ -6,6 +6,7 @@
 - Use conventional commit messages (feat:, fix:, test:, docs:, refactor:, chore:)
 - DO NOT commit secrets or other sensitive information
 - DO NOT commit engineering journals
+- Every commit must include a DCO `Signed-off-by` trailer (enforced by `commit-msg` hook); do NOT add a `Co-Authored-By` trailer
 
 ## Role Context
 - Expert Go developer
@@ -17,6 +18,7 @@
 - BDD: Use Ginkgo BDD framework
 - Use goroutines and channels for concurrency
 - Use mermaid for documentation diagrams
+- Use `internal/logging.NewLogger()` for structured logging in any code that performs I/O; do not use `fmt.Fprintf` or `log.Printf`. The pure data layer (`internal/workloads`, `internal/cloudinit`) should not log.
 
 ## Testing
 - Unit tests: `{source}_test.go` files alongside source
@@ -27,4 +29,11 @@
 ## Database
 - When a database is needed, use SQLite for local testing
 - Strictly adhere to PostgreSQL syntax standards (e.g., use standard SQL for dates, avoid loose typing)
-- Assume the production DB is strict.
+- Assume the production DB is strict.
+
+## Documentation Conventions
+- `docs/README.md` is the index/TOC for the `docs/` directory; link new docs from there.
+- Use mermaid for architecture, flow, sequence, ERD, and topology diagrams.
+- Do NOT reference bare issue numbers (`#123`) in published docs — if context from an issue matters, fold the relevant information into the doc prose so readers don't need to chase the tracker.
+- Two docs are intentionally frozen as historical snapshots and should not be updated piecemeal: `docs/implementation-plan.md` and `docs/openshift-virtualization-workload-automation.md`. Add new content to the live docs instead (architecture.md, development.md, configuration.md, etc.).
+- Engineering journals are not committed to this repo — that is the policy, not a missing directory.
diff --git a/README.md b/README.md
@@ -100,13 +100,31 @@ virtwork cleanup --run-id <uuid>
 
 ## Workloads
 
+virtwork ships nine built-in workloads, grouped by purpose. With `--vm-count 1` and no `--workloads` filter, a full run creates **11 VMs**: seven single-VM workloads plus two server/client pairs (network and tps).
+
+**Core load generators** — saturate one resource per VM:
+
 | Workload | VMs | Description | Tools |
 |----------|-----|-------------|-------|
-| **cpu** | N (configurable) | Continuous CPU stress | `stress-ng --cpu 0 --cpu-method all` |
-| **memory** | N (configurable) | Memory pressure at 80% | `stress-ng --vm 1 --vm-bytes 80%` |
-| **database** | N (configurable) | PostgreSQL with pgbench loop | `pgbench -c 10 -j 2 -T 300` |
-| **network** | N×2 (server + client pairs) | Bidirectional throughput | `iperf3 --bidir` |
-| **disk** | N (configurable) | Mixed random and sequential I/O | `fio` with multiple profiles |
+| **cpu** | N | Continuous CPU stress | `stress-ng --cpu 0 --cpu-method all` |
+| **memory** | N | Memory pressure at 80% | `stress-ng --vm 1 --vm-bytes 80%` |
+| **disk** | N | Mixed random and sequential I/O on a data disk | `fio` with multiple profiles |
+| **database** | N | PostgreSQL with pgbench loop on a data disk | `pgbench -c 10 -j 2 -T 300` |
+
+**Multi-VM benchmarks** — server/client pairs coordinated via a ClusterIP Service:
+
+| Workload | VMs | Description | Tools |
+|----------|-----|-------------|-------|
+| **network** | N × 2 | Bidirectional throughput | `iperf3 --bidir` on port 5201 |
+| **tps** | N × 2 | TCP request/response + HTTP file transfer | `netperf` (12865/12866) + Python HTTP server (8080) |
+
+**Chaos engineering** — inject failures inside the VM to test resilience. ⚠️ See [docs/chaos-workloads.md](docs/chaos-workloads.md) before deploying:
+
+| Workload | VMs | Description | Tools |
+|----------|-----|-------------|-------|
+| **chaos-disk** | N | Fill the data disk to a target percentage, release, repeat | `fallocate`, `dd` |
+| **chaos-network** | N | Inject latency and packet loss on egress | `tc` + `netem` |
+| **chaos-process** | N | Randomly send signals to non-essential processes | shell + `ps`/`kill` |
 
 All workloads run as systemd services inside the VMs, surviving reboots and auto-restarting on failure.
 
@@ -118,7 +136,7 @@ Deploy VMs with workloads.
 
 ```
 Flags:
-      --workloads strings          Workloads to deploy (default [cpu,database,disk,memory,network])
+      --workloads strings          Workloads to deploy (default: all nine — chaos-disk, chaos-network, chaos-process, cpu, database, disk, memory, network, tps)
       --vm-count int               Number of VMs per workload (default 1)
       --cpu-cores int              CPU cores per VM
       --memory string              Memory per VM (e.g., 2Gi)
@@ -163,6 +181,8 @@ virtwork uses a priority chain for configuration (highest to lowest):
 3. YAML config file (`--config`)
 4. Defaults
 
+The tables below cover the common surface. For a complete reference of every flag, environment variable, YAML key, and per-workload parameter, see [docs/configuration.md](docs/configuration.md).
+
 ### Environment Variables
 
 | Variable | Description |
@@ -205,6 +225,8 @@ The audit database records execution parameters, timestamps, workload details, V
 
 No SSH credentials are stored — only a boolean indicating whether SSH authentication was configured.
 
+For the full schema (five tables with relationships and column descriptions) and common query patterns, see [docs/audit-schema.md](docs/audit-schema.md).
+
 ```bash
 # Disable audit tracking
 virtwork run --no-audit
@@ -246,6 +268,8 @@ When no SSH flags are provided, no user account is configured in the VMs.
 
 virtwork can run as a pod on the cluster using the provided Kustomize manifests in `deploy/`. The deployment manifest uses a semantic version tag (e.g., `v0.0.1`) to pin the image for reproducible deployments. Update the version tag in `deploy/deployment.yaml` when upgrading. For development or testing, you can use `quay.io/opdev/virtwork:latest`.
 
+The section below is a quick reference. For a manifest-by-manifest deep-dive — resource topology, RBAC scope, image pinning policy, sizing, audit-DB persistence — see [docs/deployment.md](docs/deployment.md).
+
 ### Deploy with Kustomize
 
 ```bash
@@ -293,9 +317,11 @@ The codebase follows a strict layered architecture where each layer depends only
 
 ```
 Layer 4 — Orchestration     cmd/virtwork, cleanup, audit
-Layer 3 — Workload Defs     workloads (interface, cpu, memory, database, network, disk, registry)
+Layer 3 — Workload Defs     workloads (Workload + MultiVMWorkload interfaces, registry,
+                            cpu, memory, disk, database, network, tps, chaos-disk,
+                            chaos-network, chaos-process)
 Layer 2 — K8s Abstractions  vm, resources, wait
-Layer 1 — Infrastructure    config, cluster, cloudinit
+Layer 1 — Infrastructure    config, cluster, cloudinit, logging
 Layer 0 — Definitions       constants
 ```
 
@@ -313,15 +339,18 @@ virtwork/
 │   ├── config/                    # Viper-based config priority chain
 │   ├── cluster/                   # controller-runtime client init
 │   ├── cloudinit/                 # Cloud-config YAML builder
+│   ├── logging/                   # Structured logger (log/slog wrapper)
 │   ├── vm/                        # VM spec construction + CRUD + retry
 │   ├── resources/                 # Namespace + Service + Secret helpers
 │   ├── wait/                      # VMI readiness polling
 │   ├── cleanup/                   # Label-based teardown (VMs, Services, Secrets)
 │   ├── audit/                     # SQLite audit tracking (Auditor interface, schema, records)
-│   ├── workloads/                 # Workload interface + 5 implementations + registry
+│   ├── workloads/                 # Workload + MultiVMWorkload interfaces, 9 implementations, registry
 │   └── testutil/                  # Shared test helpers for integration + E2E
 ├── tests/
 │   └── e2e/                       # E2E acceptance tests (//go:build e2e)
+├── build/
+│   └── golden-image/              # Optional Fedora container disk with pre-installed tools
 ├── deploy/                        # Kustomize manifests for OpenShift deployment
 │   ├── kustomization.yaml
 │   ├── namespace.yaml
@@ -332,11 +361,22 @@ virtwork/
 │   ├── pvc.yaml
 │   └── deployment.yaml
 ├── Dockerfile                     # Multi-stage build (Debian builder + UBI9 runtime)
+├── Dockerfile.ci                  # CI variant of the runtime image
 ├── entrypoint.sh                  # Container entrypoint (auto-run or sleep)
+├── Makefile                       # CI targets (test, vet, lint, build, ci, verify)
 ├── docs/
+│   ├── README.md                  # Documentation index
 │   ├── architecture.md            # Layered architecture and diagrams
 │   ├── development.md             # Developer guide
-│   └── implementation-plan.md     # Phased build plan
+│   ├── configuration.md           # Complete config reference (flags, env, YAML)
+│   ├── deployment.md              # OpenShift deployment deep-dive
+│   ├── audit-schema.md            # SQLite audit schema reference
+│   ├── chaos-workloads.md         # Chaos engineering workload guide
+│   ├── virtwork-vs-kube-burner.md # Positioning vs kube-burner
+│   ├── guide/                     # Hands-on guides (overview, deploying, adding workloads)
+│   ├── implementation-plan.md     # Historical: original phased build plan
+│   └── openshift-virtualization-workload-automation.md  # Historical: original design rationale
+├── OWNERS
 ├── go.mod
 └── go.sum
 ```
@@ -371,6 +411,17 @@ go test -tags "integration e2e" ./...
 go build -o virtwork ./cmd/virtwork
 ```
 
+### Makefile shortcuts
+
+```bash
+make help          # list all targets
+make test          # unit tests with race detector and coverage
+make ci            # vet + test + build (no cluster required)
+make verify        # fmt + vet + lint + test (full pre-commit)
+make build         # build binary to bin/virtwork
+make container-build  # build the OCI image locally
+```
+
 See [docs/development.md](docs/development.md) for the full developer guide, including instructions for adding new workloads.
 
 ## License

diff --git a/build/golden-image/README.md b/build/golden-image/README.md
@@ -11,11 +11,16 @@ The golden image is based on `quay.io/containerdisks/fedora:41` and includes all
 - **stress-ng** — CPU and memory stress testing (cpu, memory workloads)
 - **fio** — Flexible I/O tester for disk benchmarking (disk workload)
 - **iperf3** — Network performance testing (network workload)
+- **netperf** — TCP_RR transaction performance (tps workload)
+- **python3** — HTTP file server for the tps workload's application-layer transfers
 - **postgresql-server** — PostgreSQL database with pgbench (database workload)
-- **iproute-tc** — Traffic control for network chaos engineering (future chaos workloads)
-- **iptables-nft** — Firewall rules for network partition simulation (future chaos workloads)
+- **procps-ng** — `ps`, `pkill`, `kill` for chaos-process
+- **iproute-tc** — Traffic control (`tc`) and the `sch_netem` kernel module hooks used by the chaos-network workload to inject latency and packet loss
+- **iptables-nft** — Firewall rules; reserved for future network partition / blackhole scenarios
 
-Additional tools like `fallocate`, `dd`, `kill`, and `pkill` are already present in the base Fedora image.
+Additional tools like `fallocate` and `dd` (used by chaos-disk) are already present in the base Fedora image.
+
+Workloads that need persistent storage (disk, database, chaos-disk) discover their data volume through `/dev/disk/by-id/virtio-<serial>` using the shared `diskSetupScript` helper — they do not depend on any tools beyond the standard userspace utilities already in the base image (`blkid`, `mkfs.xfs`, `mount`, `readlink`).
 
 ## Building
 
@@ -131,11 +136,17 @@ The current approach is simpler and maintains backward compatibility.
 ## Future Enhancements
 
 1. **Multi-architecture support**: Build for arm64 in addition to amd64
-2. **Automated builds**: GitHub Actions workflow on schedule
+2. **Automated builds**: CI workflow on schedule
 3. **Image scanning**: Add Trivy security scanning
 4. **Semantic versioning**: Pin specific package versions for reproducibility
 5. **Image variants**: Create minimal/full variants for different use cases
 
+## Related Docs
+
+- [docs/chaos-workloads.md](../../docs/chaos-workloads.md) — operator guide for the chaos workloads that use `iproute-tc`, `procps-ng`, and `fallocate`
+- [docs/deployment.md](../../docs/deployment.md) — how to set the golden image as the default container disk in your deployment
+- [docs/configuration.md](../../docs/configuration.md) — `--container-disk-image` flag, `VIRTWORK_CONTAINER_DISK_IMAGE` env var, and `container_disk_image` YAML key
+
 ## License
 
 Apache License 2.0. See [LICENSE](../../LICENSE).
diff --git a/docs/README.md b/docs/README.md
@@ -0,0 +1,46 @@
+# Virtwork Documentation
+
+This is the documentation index for virtwork. Start with the top-level [README](../README.md) for project orientation and quick start; come here when you need a specific reference.
+
+## Start Here
+
+- [../README.md](../README.md) — Project overview, motivation, installation, quick start, CLI reference, configuration summary, OpenShift deployment overview, architecture summary
+- [architecture.md](architecture.md) — Layered architecture, mermaid diagrams, concurrency model, workload class diagram, key design decisions
+
+## Guides
+
+Hands-on, narrative walkthroughs for first-time users and new contributors.
+
+- [guide/01-overview.md](guide/01-overview.md) — Mental model: the deploy-and-exit lifecycle, the `Workload` and `MultiVMWorkload` interfaces, where to find things in the code
+- [guide/02-deploying-workloads.md](guide/02-deploying-workloads.md) — Nine scenarios from dry-run to chaos to cluster-side deployment, with copy-pasteable commands and expected output
+- [guide/03-adding-a-workload.md](guide/03-adding-a-workload.md) — End-to-end TDD walkthrough that builds a new workload from scratch; covers simple, storage-backed, and multi-VM patterns
+
+## Reference
+
+Targeted references — read when you need a specific fact.
+
+- [configuration.md](configuration.md) — Complete reference for every CLI flag, environment variable, YAML key, and ConfigMap key (including per-workload parameters)
+- [chaos-workloads.md](chaos-workloads.md) — Operator guide for the three chaos engineering workloads (chaos-disk, chaos-network, chaos-process), including destructive-behavior warnings
+- [audit-schema.md](audit-schema.md) — SQLite audit database: five-table ERD, column-by-column reference, common queries, record lifecycle
+- [deployment.md](deployment.md) — OpenShift Kustomize deployment deep-dive: resource topology, RBAC scope, ConfigMap/Secret keys, image pinning, sizing, audit-DB persistence
+- [virtwork-vs-kube-burner.md](virtwork-vs-kube-burner.md) — Positioning compared to kube-burner; what each tool measures and where they complement each other
+
+## Contributor
+
+- [development.md](development.md) — Environment setup, building, running unit/integration/E2E tests, cluster prerequisites, Makefile targets, adding a new workload, SSH/audit configuration, testing patterns, commit conventions
+- [../build/golden-image/README.md](../build/golden-image/README.md) — Building and using the optional Fedora-based golden container disk image with pre-installed workload tools
+
+## Historical
+
+These documents capture earlier project state and are intentionally not updated. They are preserved as context.
+
+- [implementation-plan.md](implementation-plan.md) — Original phased build plan (phases 0–12), pre-chaos / pre-tps / pre-logging
+- [openshift-virtualization-workload-automation.md](openshift-virtualization-workload-automation.md) — Original design rationale, written before any application code existed
+
+---
+
+> **Conventions for this directory**
+>
+> - Diagrams are written in mermaid so they render natively on GitHub. Update them in place when the code they describe changes.
+> - Bare issue numbers (`#123`) are not used in published docs — any context that matters is folded into the prose.
+> - The historical snapshots above are frozen on purpose. New content goes into the live docs, not into them.