feat(evals): multi-turn conversation category + Personal posture for eval container

[Aaronontheweb]

Summary

Adds a dedicated Multi-Turn Conversation category to the eval suite that exercises chat -p --resume across 2-5 turn scripted sessions and captures per-turn KV cache + timing metrics. Also fixes a latent eval container misconfiguration that was silently degrading shell-using cases to tool-call-marker-only checks.

What's new

Multi-turn eval infrastructure

• eval_metrics table gains a turn_number column (primary key now includes it)
• run_prompt_resume — new helper that invokes chat -p --resume <session> against a named session and appends output to a shared per-case STDOUT_FILE so existing assertion helpers still work against the full concatenated transcript
• run_multi_turn_case — runs N scripted prompts through one session per run, captures per-turn metrics via a LAST_TURN_USAGE_LINE hand-off
• print_metrics_summary — new "Multi-Turn Cache Evolution" section shows cached vs uncached tokens and prompt_ms trending per turn so cache growth/decay is visible at a glance

New Category 8: Multi-Turn Conversation (5 cases)

Case	Turns	What it exercises
multi_turn_text_recall	3	Pure text recall across a distractor turn
multi_turn_text_growth	5	Short chit-chat (cache growth probe)
multi_turn_tool_carryover	2	Tool result → text recall without re-calling
multi_turn_tool_repeat	3	Two distinct file reads + recall by filename
multi_turn_python_app	4	Iteratively build + modify a Python script — write `greet()`, add `main`, recall signature from memory, add `style` parameter and run twice

Eval container posture fix (the latent bug)

Previously the eval container inherited strict defaults:

• `StrictDefaults=true` → `DeploymentPosture=Public` → `Audience=Public` → `ShellExecutionMode=Off`
• The `Public` audience only allowlists `file_read`, `file_write`, `attach_file`
• So `shell_execute` was silently denied on every eval prompt

Existing cases like `complex_diagnose_self` and `tool_shell` were "passing" because their assertions only checked for the `[tool:call] shell_execute` marker in stdout — which fires whether execution succeeds or is denied. The model was requesting shell, getting policy-denied, and the assertion was happy.

Fix: eval container now runs as:
```
NETCLAW_Security__DeploymentPosture=Personal
NETCLAW_Security__ShellExecutionMode=HostAllowed
NETCLAW_Security__StrictDefaults=false
NETCLAW_Tools__ShellMode=HostAllowed
```
SignalR/headless sessions already resolve to `TrustAudience.Personal`, and the Personal profile's default `ApprovalPolicy=null` means all tools are Auto-approved. Now shell_execute runs for real.

HeadlessChannel plain-text usage line fix

When a turn streams text deltas (Console.Write without newline), the subsequent [usage] line landed at the tail of the same line instead of on its own line, breaking ^[usage] anchors. Now we emit a newline before the usage line when deltas happened during the turn.

Verified results

1-iteration eval run on this branch (c5ca08ff-...):

• Overall: 23/28 cases passed (82.1%) — up from 78.6% (the 5 pre-existing failures are in Skills Discovery and flaky at 1-iteration)
• Multi-Turn Conversation: 5/5 green — all cases pass, including the 4-turn Python app
• Cache evolution table populated — per-turn metrics visible for all 17 multi-turn prompts

Performance baseline observation (saved to memorizer)

The Cache Evolution table reveals that conversation history is NOT being cached across turns in the current testlab setup. cached_tokens stays flat at ~4781 (system prompt only) while uncached grows linearly with conversation length. This is because Caddy round-robins same-session requests between the two llama-server GPU processes, so follow-up turns hit a cold cache 50% of the time.

Example from multi_turn_python_app:

Turn	Input	Cached	Uncached	prompt_ms
1	5194	4781	413	896ms
2	5897	4781	1116	1815ms
3	5901	4781	1120	1719ms
4	6237	4781	1456	1959ms

This is exactly the baseline measurement needed to validate session-sticky routing once it lands. Stored in memorizer under Local LLM Strategy project.

Test plan

• `bash -n evals/run-evals.sh` — syntax clean
• Docker image builds from source
• 1-iteration eval run: 5/5 multi-turn cases pass
• Multi-Turn Cache Evolution table populates with all 17 turns of data
• Existing single-turn cases continue to pass under Personal posture
• `complex_diagnose_self` actually runs shell now (previously silently denied)

Out of scope (follow-ups)

• Caddy session affinity — needed to make the multi-turn cache metrics meaningful as a regression signal. Currently measured baseline shows no history caching.
• Per-turn behavioral assertions — multi-turn cases currently run one assertion against concatenated stdout. A stricter variant would assert per-turn (e.g., "T3 must NOT call any tools" for the text-only recall turn).