NAIS-357: add the nemo-iron-swarm plugin (agent red-teaming and hardening) - #1037
NAIS-357: add the nemo-iron-swarm plugin (agent red-teaming and hardening)#1037koralchapnik wants to merge 28 commits into
Conversation
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Red-team and harden deployed NAT agents via iron-swarm, which runs in its own venv and is invoked by subprocess (never imported). Adds the CLI, v2 API, war-game and synth jobs, entity model, and unit tests. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Registers the plugin as a workspace member, adds the generated SDK client, and adds the Studio Iron Swarm surface (run list/detail with the swarm graph, manifest CRUD, harden flow) behind VITE_FF_IRON_SWARM_ENABLED. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Architecture and code review of the plugin before commit: severity-ranked findings with file:line and failure scenarios, and the triage outcome for each. Also ignores the run artifacts the CLI writes into the working dir. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
An agent-source manifest is re-resolved from the agent ref on every run, so any setting the entity does not carry is silently re-derived. egress had no field at all and secrets were never handed back, so both were dropped each run — a Studio user's egress entry only ever affected the initial display snapshot, and the victim's outbound calls were blocked. Store egress on IronSwarmManifest, hand both back to resolve_agent_to_manifest when re-materialising, and let PATCH edit egress. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
`init` took only a deployed agent, so a local project could only be war-gamed through Studio's upload wizard. Accept `--project-dir` as the second source: run iron-swarm's own interactive `init` at the operator's terminal — it already owns those questions, and the server path cannot ask them because `init --yes` runs behind an HTTP request — then upload the project and post the manifest it produced. `POST /manifests` accepts that pre-built `manifest_yaml` and skips its own subprocess, and now derives the entity's workflow/port/secrets/egress from the manifest rather than the request, so the two cannot disagree. `init --agent` stops resolving locally and delegates to the same endpoint Studio uses. Uploads defer to `git ls-files --exclude-standard` in a repo and always drop credentials (reusing nemo-agents' DOCKERIGNORE_TEMPLATE): a bundled dotenv is unreadable by the run, which repoints secrets_file at the platform secret store, so uploading one only persists a key in a fileset. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
The victim sandbox installs only nvidia-nat[langchain], not the platform's NAT plugins, so an agent config carrying general.telemetry (nemo_files) fails validation with union_tag_invalid and the victim never serves. Drop the block when scaffolding from an agent ref; project sources keep it, since the user's own pyproject supplies the dependency. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
iron-swarm is not on public PyPI yet, so `setup` needs to resolve it from an extra index. Add index_url/index_strategy config, passed as uv's --index on that one install so the platform's own dependencies are never resolved against it. Credentials come from ~/.netrc or UV_INDEX_<NAME>_*, and doctor redacts any embedded in the URL — Artifactory's "Set Me Up" hands out URLs with the token inline, which would otherwise reach terminal scrollback. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
FastAPI substitutes these markers at runtime from env_mappings.py; without the entry the flag resolves to nothing and Studio's Iron Swarm tab can never render, whatever the platform config says. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
The skill's flow went init -> run, which always fails: run is a pure consumer of the benign suite and synth-benign is mandatory between them. It also predated --manifest-id, --egress and --project-dir, and claimed the agent must be deployed rather than registered. Document the egress trap while here — a blocked host hangs rather than erroring, so the model answers from its own knowledge and the run reads as passing while the tool path was never exercised. The pyproject comment justifying the no-import seam was falsifiable: iron-swarm is >=3.11,<3.14, wants httpx>=0.27, and does not depend on garak. The real reasons are that importing removes none of the boundaries (garak's own venv, the Docker sandbox) while fusing both dependency graphs permanently. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
The UI section told the reader to export STUDIO_UI_VITE_FF_IRON_SWARM_ENABLED, which does nothing: the STUDIO_UI_* names are markers the studio service substitutes into the JS bundle, never read from the environment, and StudioConfig has no feature_flags field for the pydantic fallback to resolve. The flag only comes from studio.feature_flags.iron_swarm_enabled in the platform config file — and --config replaces the bundled local.yaml rather than merging, so the copy step matters. Troubleshooting and the env-var table repeated the same wrong advice. Also: the intro promised a *deployed* agent while the body said registered was enough and a local project needs neither; "How it works" blamed iron-swarm's own pins for the venv isolation when the conflicting closure is garak's, claimed the boundary is purely files (there is an HTTP event sink and the HITL bridge), and described a local scaffold `init` no longer writes. Add a quickstart covering the whole path — platform, provider, agent, war-game — and move the project-source section after the mandatory synth-benign explanation. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Agent-source manifests were re-resolved from the agent ref on every run. The reason was
mechanical, not principled: resolution *writes* a scaffold (scaffold_project +
materialize_workflow), and the API did that in a TemporaryDirectory, so the stored
manifest_yaml's project_dir pointed at a path that never existed at run time.
Two things fell out of that. Anything not re-derivable was silently lost — egress had no field
at all, so an operator's allow-list vanished each run, and because a blocked host hangs rather
than errors, the run still reported success while tool-using attacks quietly no-op'd. And the
two sources needed separate materializers, which is why PATCH egress worked on one and was
accepted-then-ignored on the other.
Store the resolved scaffold as a fileset, as project source already did. Both then materialize
through one _materialize_from_bundle: restore the bundle, use the stored manifest as-is, and
rewrite only what describes this host rather than the target — project_dir, secrets_file, and
the current Inference-Gateway route.
A manifest is now a frozen target, so two runs are comparable. Agent edits reach it via
POST /manifests/{name}/refresh (`nemo iron-swarm refresh`), which apply-mitigation calls itself
so harden -> apply -> re-run still measures what was just applied. Manifests saved before this
re-resolve once, store a bundle and freeze themselves — no user action. Deleting a manifest now
deletes its bundle; project bundles already leaked before this.
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
iron-swarm's AgentSpec already carries `env`, but the platform had no way to set it — which is why a host-backend URL needed special handling. Expose it as a manifest field, settable at `init --env KEY=VALUE` or by PATCH, merged over whatever iron-swarm baked in at init. Applied in _materialize_from_bundle rather than threaded through the resolver, so it reaches agent-source and project-source manifests by construction — the split that let PATCH egress work on one and be silently ignored on the other is what freezing removed. Kept deliberately non-secret: values sit in plaintext on the entity, so credentials stay in `secrets`, which stores only names and resolves them from the Secrets store at run time. Said so in the field description, the CLI help and the skill, because the two fields look interchangeable. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Freezing manifests made materialization re-apply the Inference-Gateway route, since that route
belongs to the platform we run on rather than to the frozen target. It was written to
`agent.backends`, but `backends` is a top-level field of iron-swarm's AgentManifest, a sibling of
`agent` — AgentSpec has no such field and the model forbids extras.
Every materialized manifest therefore failed validation before the victim started:
1 validation error for AgentManifest
agent.backends — Extra inputs are not permitted [extra_forbidden]
Both sources, and both `run` and `synth-benign`, died about two seconds in. The test missed it by
asserting the shape I had assumed instead of the schema; it now asserts backends is absent from
`agent`, and the fix was verified by loading a materialized manifest through
iron_swarm.manifest.load_manifest in the iron-swarm venv — the only authority on this contract.
Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
Only `port` was written back into manifest_yaml, so egress, secrets and env set after `init` left no trace in it. The run applied them at materialization regardless, but the YAML we show — and that `init -o` writes to disk — was the frozen base rather than what would actually run, so an operator who set `env` saw nothing and reasonably concluded it had been lost. Generalize _yaml_with_port into _yaml_with_agent_settings and apply it wherever the manifest is written: create, PATCH and refresh, for both sources. Unparseable YAML is returned untouched — a display concern must not cost someone their manifest. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
…erview Studio had no view of the target at all: the manifest detail page rendered the benign suite and defenders, while manifest_yaml, egress, secrets and env appeared nowhere. The create form collected egress and never showed it back — a write-only setting, which is how a silently dropped egress value went unnoticed for so long. Freezing is what makes fixing this worthwhile: the stored YAML is now what actually runs, so displaying it is truthful rather than a preview of something that would be regenerated anyway. Add a Target panel (source, port, egress, secrets, env, and the manifest itself), a Refresh action behind a confirm, and an env editor on both the create form and the detail page. The YAML stays read-only: overrides.defenders[].implementation is a dynamic import executed in the job process, so a free-text editor would turn manifest-write into code execution on the platform host. Also re-attach to a generation already in flight. The HITL prompt lives on the job's status_details, but the job name lived only in component state, so a reload, a navigation or a second tab left the job waiting for an answer with no UI asking for it — the interview simply vanished, and the job held its sandbox until the HITL timeout, which then collided with the next attempt. On mount we now look for a running synth job for the manifest and adopt it, letting the existing job poll decide whether it is really still live. Signed-off-by: Koral Chapnik Verbun <kchapnikverb@nvidia.com>
📝 WalkthroughWalkthroughIron Swarm adds a NeMo Platform plugin with REST APIs, jobs, SDK and CLI support, agent and project manifests, durable events, benign-suite synthesis, defense validation, and feature-gated Studio pages for runs, manifests, HITL workflows, and hardening. ChangesIron Swarm plugin foundation
Studio integration
Sequence Diagram(s)sequenceDiagram
participant Studio
participant IronSwarmAPI
participant IronSwarmJob
participant EventRelay
Studio->>IronSwarmAPI: submit manifest or job request
IronSwarmAPI->>IronSwarmJob: compile and run job
IronSwarmJob->>EventRelay: append run events
Studio->>EventRelay: poll events by cursor
EventRelay-->>Studio: return swarm events and HITL status
Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 2📝 Generate docstrings 💡
⚔️ Resolve merge conflicts 💡
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
Note
Due to the large number of review comments, Critical severity comments were prioritized as inline comments.
🟠 Major comments (19)
plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/provisioning.py-71-73 (1)
71-73: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winRedact the index URL before echoing it.
config.index_urlcan embed credentials (https://user:AKCp8token@host/simple).setupprints it verbatim into terminal scrollback and CI logs.doctoralready avoids this throughchecks.redact_index_url, andtest_doctor_never_prints_an_embedded_tokenlocks that behavior in. Use the same helper here.🔒️ Proposed fix
+from nemo_iron_swarm_plugin.cli.checks import redact_index_url ... if config.index_url: - typer.echo(f" using extra index {config.index_url}") + typer.echo(f" using extra index {redact_index_url(config.index_url)}") install_cmd += ["--index", config.index_url]🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/provisioning.py` around lines 71 - 73, Update the index URL output in the provisioning setup flow to pass config.index_url through the existing checks.redact_index_url helper before typer.echo, while continuing to use the original URL in install_cmd for installation.web/packages/studio/src/routes/NewIronSwarmManifestRoute/index.tsx-104-115 (1)
104-115: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winThe inspect effect overwrites operator edits and can apply a stale response.
The effect re-runs on every change of
source,selectedAgent, orworkspace. Two problems follow:
- If the operator edits
portorsecretsand then toggles the source control back toagent, the effect re-fires with the same agent and overwrites those edits.- If the operator switches agents quickly, an earlier in-flight response can resolve last and write stale values.
Track the agent the response belongs to and skip writes when it no longer matches.
🐛 Proposed fix
const selectedAgent = watch('agent'); const inspectAgent = useInspectAgent(); const { mutate: runInspectAgent } = inspectAgent; + const inspectedAgent = useRef<string | undefined>(undefined); useEffect(() => { if (source !== 'agent' || !selectedAgent) return; + if (inspectedAgent.current === selectedAgent) return; + inspectedAgent.current = selectedAgent; runInspectAgent( { workspace, agent: selectedAgent }, { onSuccess: (facts) => { + if (inspectedAgent.current !== selectedAgent) return; setValue('port', String(facts.port)); setValue('secrets', facts.secrets.join(', ')); }, } ); }, [source, selectedAgent, workspace, runInspectAgent, setValue]);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/routes/NewIronSwarmManifestRoute/index.tsx` around lines 104 - 115, Update the inspect effect around runInspectAgent so it does not overwrite operator edits when source toggles back to the same agent, and ignores responses for agents that are no longer selected. Track the agent associated with each inspection request, only apply setValue updates when that agent still matches the current selectedAgent, and preserve the existing source, workspace, and request behavior.web/packages/studio/src/components/ironSwarm/useSanityCheck.ts-112-122 (1)
112-122: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winUnvalidated
JSON.parsecan crash the report view.Line 117 asserts the parsed blob is a
ValidationReport.SanityCheckReportdestructuresreport.summaryand readssummary.attacks_blocked. If the result file is truncated, empty, or shaped differently, the render throws instead of showing an error.Validate the parsed payload before returning it, or default the missing fields.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/useSanityCheck.ts` around lines 112 - 122, Update the queryFn in useSanityCheck to safely handle malformed, empty, or differently shaped validation-result JSON before returning a ValidationReport. Validate the parsed payload’s required report and summary fields, or supply defaults for missing fields, so SanityCheckReport can access summary.attacks_blocked without throwing during render.web/packages/studio/src/components/ironSwarm/useRunWarGame.ts-20-31 (1)
20-31: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winOne failed poll aborts the flow silently.
ironSwarmListRunsis not wrapped in try/catch. A single transient failure rejects the promise, skips the fallbacknavigate, and produces an unhandled rejection because the caller usesvoid. The user sees the success toast and stays on the manifest page.Also add an abort guard so the loop stops when the component unmounts.
Proposed fix
const openRunForJob = async (jobName: string): Promise<void> => { for (let attempt = 0; attempt < 60; attempt++) { - const { data } = await ironSwarmListRuns(workspace, { sort: '-created_at', page_size: 20 }); - const run = (data as IronSwarmRun[] | undefined)?.find((r) => r.job_id === jobName); - if (run?.name) { - navigate(getIronSwarmRunDetailsRoute(workspace, run.name)); - return; - } + try { + const { data } = await ironSwarmListRuns(workspace, { sort: '-created_at', page_size: 20 }); + const run = (data as IronSwarmRun[] | undefined)?.find((r) => r.job_id === jobName); + if (run?.name) { + navigate(getIronSwarmRunDetailsRoute(workspace, run.name)); + return; + } + } catch { + // transient failure — keep polling until the window expires + } await new Promise((resolve) => setTimeout(resolve, 500)); } navigate(getIronSwarmRunListRoute(workspace)); };🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/useRunWarGame.ts` around lines 20 - 31, Update openRunForJob to catch transient errors from ironSwarmListRuns and continue polling so failures do not bypass the final fallback navigation. Add an unmount abort guard shared with the component lifecycle, checking it during each polling iteration and before navigation, and ensure the guard is released on cleanup.web/packages/studio/src/routes/IronSwarmManifestDetailRoute/index.tsx-86-93 (1)
86-93: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winCSV formula injection in the exported requests.csv.
escapeCsvonly quotes",,and\n. A value that begins with=,+,-,@, tab, or CR is written raw. The suite content is LLM-generated and operator-editable. When the downloaded file is opened in Excel or Sheets, such a value is evaluated as a formula.Prefix those values with a single quote or
\t, and quote the field.Proposed fix
-const escapeCsv = (value: string): string => - /[",\n]/.test(value) ? `"${value.replace(/"/g, '""')}"` : value; +const FORMULA_PREFIX = /^[=+\-@\t\r]/; +const escapeCsv = (value: string): string => { + const safe = FORMULA_PREFIX.test(value) ? `'${value}` : value; + return /["',\r\n]/.test(safe) ? `"${safe.replace(/"/g, '""')}"` : safe; +};🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/routes/IronSwarmManifestDetailRoute/index.tsx` around lines 86 - 93, Update escapeCsv to prevent formula injection in exported requests.csv: detect values beginning with =, +, -, @, tab, or carriage return, prefix them with a single quote (or tab), and ensure the resulting field is quoted. Preserve existing escaping of quotes, commas, and newlines, and keep toRequestsCsv using escapeCsv for every field.web/packages/studio/src/components/ironSwarm/useSanityCheck.ts-163-176 (1)
163-176: 🚀 Performance & Scalability | 🟠 Major | ⚡ Quick win
useLatestSanityCheckJobpolls forever.
refetchIntervalis a constant. The hook keeps listing 50 runs on every interval for as long as the Harden tab is open, including after the job is found and after it completes. Stop the interval once ajob_idis returned.Proposed fix
- refetchInterval: JOB_POLLING_INTERVAL_MS, + refetchInterval: (query) => (query.state.data ? false : JOB_POLLING_INTERVAL_MS),🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/useSanityCheck.ts` around lines 163 - 176, Update useLatestSanityCheckJob so refetchInterval becomes conditional on the query result: continue polling while no job_id has been found, and disable polling once a job_id is returned. Preserve the existing query key, enabled condition, and latest-run lookup behavior.web/packages/studio/src/components/ironSwarm/useMitigations.ts-221-229 (1)
221-229: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winUnmemoized
defensesarray drives a downstream render loop.useMitigationsallocates a new array wheneverquery.datais undefined, andHardenPaneluses that array as an effect dependency that callssetSelectedwith a newSet.
web/packages/studio/src/components/ironSwarm/useMitigations.ts#L221-L229: wrapdefensesandrecommendationsinuseMemokeyed onquery.data.web/packages/studio/src/components/ironSwarm/HardenPanel.tsx#L162-L165: key the selection-reset effect on the joined defense ids instead of the array identity.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/useMitigations.ts` around lines 221 - 229, Memoize the defenses and recommendations values in useMitigations, keyed by query.data, so unchanged data preserves array identity. In web/packages/studio/src/components/ironSwarm/HardenPanel.tsx lines 162-165, update the selection-reset effect dependency to use joined defense IDs rather than the defenses array identity.web/packages/studio/src/components/ironSwarm/useGenerateBenignSuite.ts-59-69 (1)
59-69: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winBoth run-resolution paths call
ironSwarmListRunswithout error handling or cleanup. A rejection becomes an unhandled promise rejection, and state setters can fire after unmount.
web/packages/studio/src/components/ironSwarm/useGenerateBenignSuite.ts#L59-L69: wrap the request intry/catch, and toast when the run never resolves sostartingdoes not staytrue.web/packages/studio/src/components/ironSwarm/useGenerateBenignSuite.ts#L78-L94: wrap the async IIFE intry/catchand add an ignore flag in the effect cleanup before callingsetRunName/setJobName.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/useGenerateBenignSuite.ts` around lines 59 - 69, Update web/packages/studio/src/components/ironSwarm/useGenerateBenignSuite.ts lines 59-69 in resolveRun to catch ironSwarmListRuns failures and toast when all attempts expire, ensuring starting is reset instead of remaining true; update lines 78-94 in the async IIFE to catch request failures and add effect-cleanup ignore guards before setRunName and setJobName.web/packages/studio/src/routes/IronSwarmRunDetailsRoute/index.tsx-122-130 (1)
122-130: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winTab value becomes orphaned when the HITL prompt clears.
The effect switches to
interview, but nothing switches back. After the operator submits an answer,hitlPendingturns false, and both theinterviewtrigger and itsTabsContentunmount whiletabstill equals'interview'. The panel then renders no content until the user clicks another tab.Return to
swarmwhen the prompt clears.🐛 Proposed fix
useEffect(() => { - if (hitlPending) setTab('interview'); + setTab((current) => { + if (hitlPending) return 'interview'; + return current === 'interview' ? 'swarm' : current; + }); }, [hitlPending]);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/routes/IronSwarmRunDetailsRoute/index.tsx` around lines 122 - 130, Update the tab-selection effect near hitlPending and setTab so it selects interview while HITL is pending and returns to swarm when hitlPending becomes false, preventing the tab value from referencing the unmounted interview content.web/packages/studio/src/components/ironSwarm/swarm/SwarmGraph.tsx-196-201 (1)
196-201: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick winNodes are selectable only by pointer.
The
<g>element handlesonPointerDownonly. Keyboard users cannot select a node, soNodeDetailstays empty for them. Addrole="button",tabIndex={0}, an accessible name, and anonKeyDownhandler that callsonSelect(n.id)on Enter and Space.♿ Proposed fix
<g key={n.id} + role="button" + tabIndex={0} + aria-label={`${n.title} (${status})`} onPointerDown={(e) => onNodePointerDown(e, n)} + onKeyDown={(e) => { + if (e.key === 'Enter' || e.key === ' ') { + e.preventDefault(); + onSelect(n.id); + } + }} className="cursor-grab active:cursor-grabbing" >🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/swarm/SwarmGraph.tsx` around lines 196 - 201, Update the node `<g>` element in the SwarmGraph rendering to be keyboard-accessible by adding button semantics, keyboard focus via tabIndex={0}, and an accessible name. Add an onKeyDown handler that calls onSelect(n.id) when the key is Enter or Space, while preserving the existing pointer selection behavior.web/packages/studio/src/components/ironSwarm/ReviewPanel.tsx-17-19 (1)
17-19: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winLocal rows never resync with a new
suiteprop.
useStateusessuiteonly for the first render. The parent route keeps this component mounted while it pollsstatus_details, so a second review round (or a late-arriving suite) leaves the operator editing and approving the previous suite.Reset rows when
suitechanges, or key the component by round in the parent.🔁 Proposed fix
-import { FC, useState } from 'react'; +import { FC, useEffect, useState } from 'react'; @@ const [rows, setRows] = useState<SuiteRow[]>(suite); + useEffect(() => { + setRows(suite); + }, [suite]);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/ReviewPanel.tsx` around lines 17 - 19, Update ReviewPanel’s local rows state so it resynchronizes with the incoming suite prop whenever suite changes, preserving user edits between suite updates while replacing stale rows for new review rounds or late-arriving data.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/sdk.py-280-308 (1)
280-308: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winPreserve the async client's authentication.
make_sdk(str(self._platform.base_url))creates a direct-modeNeMoPlatformwithout auth headers. Both async methods therefore drop the caller's credentials and can fail on authenticated deployments. Preserve the async client's auth configuration when creating the sync client.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/sdk.py` around lines 280 - 308, Update the sync client creation in the async methods wrapping _run_war_game and _run_synth_benign to preserve the caller’s authentication configuration from self._platform. Ensure the client passed to asyncio.to_thread retains the async client’s auth headers while using the existing base URL.plugins/nemo-iron-swarm/tests/unit/test_events.py-88-120 (1)
88-120: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winThis test cannot detect a filename mismatch in the fallback.
_events_pathis patched to.../missing/events.jsonl, andfake_downloadwritesevents.jsonlinto the destination directory. The two names match only because of that patch. Production names the local log<safe-run-name>.jsonl, so a fileset that storesevents.jsonlwould produce an empty response while this test still passes. Patch_events_pathto a run-named file, for examplemy-run.jsonl, and keep the download writing the name the real fileset contains.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/tests/unit/test_events.py` around lines 88 - 120, Update test_get_events_falls_back_to_fileset_when_local_missing so the patched _events_path uses a run-named file such as my-run.jsonl, while fake_download continues writing events.jsonl from the fileset. Keep the existing fallback assertions and setup unchanged so the test detects mismatches between the local log filename and the downloaded fileset filename.plugins/nemo-iron-swarm/README.md-14-18 (1)
14-18: 📐 Maintainability & Code Quality | 🟠 Major | 🏗️ Heavy liftSplit this README by Diataxis type.
This page combines tutorial, how-to, troubleshooting, reference, and explanation content. Split these into separate pages. Put prerequisites before task steps. Add Next Steps links. Provide verified Python SDK and CLI alternatives in tab sets where the SDK supports the task.
As per coding guidelines, each documentation page must fit one Diataxis quadrant and task pages must list prerequisites first.
Also applies to: 287-294, 412-447
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/README.md` around lines 14 - 18, Restructure the README into separate Diataxis-focused pages, separating tutorial, how-to, troubleshooting, reference, and explanatory content. Ensure each task page begins with prerequisites, add Next Steps links, and provide verified Python SDK and CLI alternatives in tab sets wherever both support the task. Apply the same restructuring to the sections corresponding to the referenced later ranges, preserving the existing instructions and examples.Source: Coding guidelines
plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/manifests.py-555-566 (1)
555-566: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winRefresh leaks the new fileset when
entity_client.updatefails.If
updateraises, the freshly uploadedfilesetis never referenced by any entity and is never deleted. Wrap the update and delete the new fileset on failure. Also mapNemoEntityConflictError/NemoEntityNotFoundErrorto 409/404 asupdate_manifestdoes; today they surface as 500.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/manifests.py` around lines 555 - 566, Update the refresh flow around entity_client.update to clean up the newly assigned fileset when the update fails, deleting fileset rather than the stale fileset in the exception path. Add the same NemoEntityConflictError and NemoEntityNotFoundError mappings used by update_manifest so failures return 409 and 404 respectively, while preserving stale-fileset cleanup only after a successful update.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/manifest.py-236-242 (1)
236-242: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winReject an absolute or escaping
agent["workflow"].
manifest_dir / project_dir / workflowwritesdefense_workflowwherever the joined path resolves. An absoluteworkflowvalue, or one containing.., writes outside the job storage directory. The values come from a stored manifest that an API client controls at create time. Resolve the path and confirm it stays undermanifest_dir.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/manifest.py` around lines 236 - 242, Validate the resolved workflow path in the manifest-writing flow before creating directories or writing defense_workflow. Using workflow_file and manifest_dir, reject absolute agent["workflow"] values and any resolved path that escapes manifest_dir, including traversal through project_dir; only proceed with mkdir and write_text when the resolved path remains under manifest_dir.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/synth_benign.py-108-137 (1)
108-137: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winA failure before
_run_serviceleaves a pre-created run record stuck atrunning.
compilecan hand arun_namein the step config. If_materialize_manifest,require_provisioned, orcheck_victim_secretsraises,run()returns a failed result but never touches that record. Studio then shows the run asrunningforever. Finalize the record in theexceptbranch whenconfig.get("run_name")is set, as the war-game job does.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/synth_benign.py` around lines 108 - 137, Update synth-benign’s run exception path to finalize the pre-created run record when config.get("run_name") is set, including failures from _materialize_manifest, require_provisioned, or check_victim_secrets before _run_service executes. Mirror the existing war-game job’s record-finalization behavior, then preserve the current classified failure result.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/manifests.py-585-590 (1)
585-590: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winGuard the fileset cleanup so a storage error does not turn a successful delete into a 500.
The entity is already deleted at this point. If
delete_filesetraises, the endpoint returns 500 and the caller assumes the manifest still exists.🛡️ Proposed fix
for ref in (existing.agent_fileset, existing.project_fileset): if ref: - await run_in_threadpool(delete_fileset, sdk, ref) + try: + await run_in_threadpool(delete_fileset, sdk, ref) + except Exception: # the entity is gone; an orphan bundle must not fail the delete + logger.warning("failed to delete fileset '%s' for manifest '%s'", ref, name, exc_info=True)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/manifests.py` around lines 585 - 590, Guard the post-deletion fileset cleanup loop in the manifest deletion flow so exceptions from delete_fileset do not propagate as endpoint failures. Keep the entity deletion successful and handle or log cleanup errors around each fileset independently, preserving cleanup attempts for both existing.agent_fileset and existing.project_fileset.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/execution.py-278-283 (1)
278-283: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winTear down the sandbox when
upfails.
upcan fail after it started containers, for example when the health wait times out. This early return skips thefinallyblock, so the victim container and its port forward stay allocated. Move the teardown to cover this path.🛡️ Proposed fix
if up_failure is not None: + _teardown_sandbox(bin_path, manifest, env, ctx) return RunOutcome("failed", up_done.returncode, record_name=record_name, failure=up_failure)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/execution.py` around lines 278 - 283, Ensure the sandbox teardown/finally scope also covers the _run_iron_swarm invocation for the “up” command. Restructure the up_failure handling so it records or propagates the failure only after cleanup runs, while preserving the existing failed RunOutcome values and releasing the victim container and port forward.
🟡 Minor comments (19)
plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/main.py-418-424 (1)
418-424: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winHandle a malformed
mitigations.json.
json.loadsraisesJSONDecodeErrorand prints a traceback. The adjacent missing-file case gets a clean message. Also guarddefense_idsagainst a payload that is not the expected shape.🐛 Proposed fix
- mitigations = json.loads(path.read_text(encoding="utf-8")) + try: + mitigations = json.loads(path.read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError) as exc: + typer.secho(f"Could not read mitigations file {mitigations_file}: {exc}", fg="red") + raise typer.Exit(code=1) from exc🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/main.py` around lines 418 - 424, Update the mitigations-loading flow in the CLI around path.read_text, json.loads, and defense_ids to catch malformed JSON and report a concise red error before exiting with code 1, matching the existing missing-file handling. Validate that the decoded payload has the expected shape before passing it to defense_ids, and handle invalid payloads with the same clean failure behavior.web/packages/studio/src/routes/NewIronSwarmManifestRoute/index.tsx-147-151 (1)
147-151: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winNeither manifest path bounds the victim port. Both forms accept a port outside
1-65535and write it into the manifest.
web/packages/studio/src/routes/NewIronSwarmManifestRoute/index.tsx#L147-L151: extend the check to rejectport < 1andport > 65535.web/packages/studio/src/components/ironSwarm/ProjectManifestWizard.tsx#L131-L131: replacepositive()with.min(1).max(65535)and attach a message to the type check so non-numeric input reports a clear error.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/routes/NewIronSwarmManifestRoute/index.tsx` around lines 147 - 151, Bound port validation in NewIronSwarmManifestRoute/index.tsx lines 147-151 to reject values below 1 or above 65535 while preserving the whole-number check. In ProjectManifestWizard.tsx line 131, replace positive() with min/max validation for 1–65535 and provide a clear message for non-numeric input.web/packages/studio/src/components/ironSwarm/ModelGroupFields.tsx-186-208 (1)
186-208: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winClear
testResultwhen the model or endpoint changes.The verdict text stays on screen after the operator edits
model,base_url, or the secret. The stale "Connection OK." then describes a configuration that no longer exists. Reset the result in eachonChange.🐛 Proposed fix
+ const update = (patch: Partial<ModelChoice>) => { + setTestResult(null); + onChange(withGroup(value, group, patch)); + };Then call
update({ model: e.target.value || undefined })and the equivalent forbase_urlandapi_key_secret.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/ModelGroupFields.tsx` around lines 186 - 208, Update the model, base_url, and api_key_secret change handlers in ModelGroupFields to clear testResult whenever any of those values changes. Preserve the existing normalized values and onChange/update behavior while resetting the stale connection verdict in each handler.web/packages/studio/src/constants/routes.ts-113-117 (1)
113-117: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winNamespace resource detail routes.
React Router resolves
/iron-swarm/manifeststoironSwarmManifestList, so a run namedmanifestsis unreachable. TheNAME_PATTERNacceptsnew, so a manifest namednewis also unreachable because/iron-swarm/manifests/newresolves toironSwarmManifestNew. Namespace runs under/iron-swarm/runs/:ironSwarmRunNameand reserve or namespace the manifestnewroute.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/constants/routes.ts` around lines 113 - 117, Update the ironSwarm route definitions so run details use the `/iron-swarm/runs/:ironSwarmRunName` namespace, preventing the `manifests` run name from colliding with manifest routes. Also adjust the manifest detail/new route structure to reserve or namespace the `new` endpoint, while preserving manifest list and creation navigation.web/packages/studio/src/components/ironSwarm/eventTypes.ts-4-6 (1)
4-6: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winStale transport reference: SSE was replaced by JSON polling.
Lines 4-6 and 91-92 describe an "SSE relay" and "plugin SSE endpoint". The PR changed event delivery to JSON polling. Update both comments so readers do not look for an SSE endpoint.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/eventTypes.ts` around lines 4 - 6, Update the comments near the event catalog and the plugin endpoint reference in eventTypes.ts to describe JSON polling instead of SSE, including removing references to the SSE relay and plugin SSE endpoint while preserving the existing event-source and rendering context.web/packages/studio/src/routes/IronSwarmManifestDetailRoute/index.tsx-270-286 (1)
270-286: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick winEnvironment values containing a comma are silently truncated.
saveEnvsplits the draft on,before it finds=. A value such asHOSTS=a,bis parsed asHOSTS=aplus a discarded fragment. The user gets a success toast and loses data.Reject entries that fail to parse, or switch the dialog to newline-separated pairs.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/routes/IronSwarmManifestDetailRoute/index.tsx` around lines 270 - 286, Update saveEnv so environment values containing commas are not silently truncated: either parse a format that preserves commas, such as newline-separated key/value pairs, or validate each comma-separated entry and reject malformed fragments before calling clearManifest.mutateAsync. Only show the success toast after all entries are valid and saved; retain the existing error-toast path for rejected input or mutation failures.web/packages/studio/src/routes/IronSwarmManifestDetailRoute/index.tsx-303-311 (1)
303-311: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winRevoking the object URL synchronously can cancel the download.
URL.revokeObjectURLruns in the same tick asanchor.click(). Some browsers have not started reading the blob yet, so the download fails. Defer the revoke.Proposed fix
anchor.click(); - URL.revokeObjectURL(url); + setTimeout(() => URL.revokeObjectURL(url), 0);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/routes/IronSwarmManifestDetailRoute/index.tsx` around lines 303 - 311, Update downloadCsv to defer URL.revokeObjectURL until after the browser has initiated the anchor download, rather than revoking it synchronously after anchor.click(). Keep the existing blob creation, filename, and click behavior unchanged.web/packages/studio/src/components/ironSwarm/InterviewPanel.tsx-33-36 (1)
33-36: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winDefaults do not re-seed when
promptchanges.
useStateruns its initializer once. If the parent renders a second prompt at the same position, the component stays mounted andanswerskeeps the previous gaps. Questions then show no selection, and submit sends empty answers.Give the panel a key derived from the prompt at the call site, or re-seed on prompt change.
Proposed fix at the call site
- <InterviewPanel + <InterviewPanel + key={gen.interview.questions.map((q) => q.gap).join('|')} prompt={gen.interview}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/InterviewPanel.tsx` around lines 33 - 36, Update InterviewPanel so its answers state is re-seeded whenever prompt changes, preserving defaults for the new prompt’s questions; alternatively, at the component’s call site provide a key derived from the prompt so React remounts it. Use the existing defaultAnswer initialization path and ensure submissions reflect the current prompt rather than prior gaps.web/packages/studio/src/components/ironSwarm/hitlTypes.ts-46-64 (1)
46-64: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winPrompts are cast without validating their array payloads.
pendingInterviewandpendingReviewcheck onlyround.questionsandsuiteare asserted, not verified. If the job writes a prompt without those keys, consumers that map over them (for exampleReviewPanelrenderingreview.suite) throw at render.Guard the arrays and normalize to
[].🛡️ Proposed fix
if (!interview || typeof interview.round !== 'number') return null; if (response?.round === interview.round) return null; - return interview; + return { round: interview.round, questions: Array.isArray(interview.questions) ? interview.questions : [] }; @@ if (!review || typeof review.round !== 'number') return null; if (response?.round === review.round) return null; - return review; + return { round: review.round, suite: Array.isArray(review.suite) ? review.suite : [] };🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/ironSwarm/hitlTypes.ts` around lines 46 - 64, Update pendingInterview and pendingReview to validate their prompt array payloads before returning them, requiring interview.questions and review.suite to be arrays alongside the existing round checks. Normalize missing or invalid arrays to [] as appropriate so consumers can safely iterate them, while preserving the existing null response-round behavior.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/events.py-109-119 (1)
109-119: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winRouter description still advertises SSE.
This module now implements polling only.
service.pyregisters this router with the description "Live run-event ingest (from the run) + SSE stream (to Studio)". That text reaches the generated OpenAPI tag description, so API consumers see a transport that no longer exists. Update the description inservice.pyto describe the poll endpoint.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/events.py` around lines 109 - 119, Update the router description supplied by service.py when registering this events router so it describes the polling endpoint and no longer mentions an SSE stream. Keep the existing ingest-run context and ensure the generated OpenAPI tag description matches the module’s polling-only behavior.plugins/nemo-iron-swarm/tests/unit/test_compose_defense.py-97-106 (1)
97-106: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winDo not preserve success for a nonexistent run.
This test creates no
run-1entity but asserts200. It locks the route contract where/runs/{name}/compose-defenseignores{name}. Look up the run in the handler, return404when absent, and change this test to assert that result.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/tests/unit/test_compose_defense.py` around lines 97 - 106, Update the compose-defense handler to resolve the run identified by the route’s name parameter and return 404 when it does not exist, rather than composing a successful response unconditionally. Revise test_compose_defense_endpoint_composes_selection to create or reference an existing run for the success case, and add or update the nonexistent run assertion to expect 404.plugins/nemo-iron-swarm/README.md-300-312 (1)
300-312: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winUse one manifest lifecycle contract.
Lines 300-312 say a manifest freezes its resolved target until
refresh. Lines 441-447 say an agent-source manifest re-resolves on every run. These rules give opposite results for run comparability and refresh behavior. Document the implemented contract consistently.As per PR objectives, manifests freeze resolved targets and explicit refresh incorporates agent changes.
Also applies to: 441-447
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/README.md` around lines 300 - 312, Update the manifest lifecycle documentation around the frozen-target guidance and the agent-source manifest section to consistently state that manifests preserve their resolved targets across runs. Document explicit refresh as the mechanism that incorporates agent changes, and remove or revise the claim that agent-source manifests re-resolve on every run.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/model_preflight.py-72-76 (1)
72-76: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winHandle non-object JSON responses.
If
resp.json()returns an array or scalar,.get()raises an uncaughtAttributeError. Validate the decoded object before accessing"data"and return the existing soft-pass result.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/model_preflight.py` around lines 72 - 76, Update the JSON handling in the model preflight probe to validate that resp.json() returns an object with mapping behavior before calling .get("data"). For arrays or scalar JSON responses, return the existing reachable/authenticated soft-pass result with list_supported=False and the current detail, while preserving normal processing for valid objects.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/synth_client.py-60-70 (1)
60-70: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winClassify malformed synth responses as synth-service failures.
Catch
ValueErrorfrom invalid JSON and reject non-object JSON before returning from_post.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/synth_client.py` around lines 60 - 70, Update _post to catch ValueError from resp.json() and classify it as CATEGORY_SYNTH_SERVICE via IronSwarmRunError. Validate that the decoded JSON is an object/dict before returning it, and raise the same synth-service failure for any other JSON shape while preserving existing HTTPError handling.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/service.py-69-74 (1)
69-74: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winStale "SSE stream" wording; events are polled JSON.
Event delivery moved from SSE to cursor-based JSON polling.
GET /runs/{name}/eventsreturnsEventsResponsewith anaftercursor. ThisRouterSpecdescription surfaces in the generated OpenAPI tag, so the wrong wording reaches API consumers. The module docstring at Line 9 has the same problem.📝 Proposed fix
RouterSpec( router=events.router, tag="Iron Swarm Events", - description="Live run-event ingest (from the run) + SSE stream (to Studio).", + description="Run-event ingest (from the run) + polled event reads (to Studio).", prefix="/v2/workspaces/{workspace}", ),🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/service.py` around lines 69 - 74, Update the events RouterSpec description to describe cursor-based JSON polling rather than an SSE stream, and revise the module docstring to remove the stale SSE wording. Keep the description accurate for GET /runs/{name}/events returning EventsResponse with an after cursor.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/manifests.py-114-132 (1)
114-132: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winReturn the serialized filter, not the model.
Line 114 computes
filter_dict, but line 131 returns the rawfiltermodel. The response then includes every unset field asnull, which disagrees with the filter actually applied.♻️ Proposed fix
- "filter": filter or None, + "filter": filter_dict or None,🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/manifests.py` around lines 114 - 132, Update the response construction in the manifest-listing function to return the computed serialized filter_dict instead of the raw filter model. Preserve the existing filter or None behavior when no filter values are provided, while ensuring unset fields remain excluded consistently with the filter passed to entity_client.list.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/manifest.py-165-168 (1)
165-168: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winGuard against non-mapping
backendsentries.
manifest.get("backends")comes from stored YAML. If any entry is not a mapping,b.get("name")raisesAttributeErrorand the run fails with an unclassified error instead of a manifest error.🛡️ Proposed fix
- others = [b for b in (manifest.get("backends") or []) if b.get("name") != gw_backend.get("name")] + others = [ + b + for b in (manifest.get("backends") or []) + if isinstance(b, dict) and b.get("name") != gw_backend.get("name") + ]🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/manifest.py` around lines 165 - 168, Update the backends filtering logic in the manifest handling block to validate each entry is a mapping before calling b.get("name"). Preserve valid mappings, exclude or reject non-mapping entries using the existing manifest-error handling path, and ensure malformed YAML produces a classified manifest error instead of an AttributeError.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/runs.py-142-158 (1)
142-158: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick winReturn 409 for optimistic-lock conflicts.
NemoEntitiesClient.update()sendsexpected_db_versionand prevents lost updates, but this handler mapsNemoEntityConflictErrorto 500. Catch it explicitly and return 409.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/runs.py` around lines 142 - 158, Update the exception handling around entity_client.update(agent) to catch NemoEntityConflictError explicitly and raise an HTTPException with status 409, preserving the conflict as the cause; keep the existing generic exception path returning 500 for other update failures.plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/tasks/war_game/__main__.py-25-38 (1)
25-38: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winMove sandbox teardown to cover
up. If SIGTERM arrives afterupstarts the sandbox but before it returns,_shutdown_handlerexits before_teardown_sandboxis registered.subprocess.runalso does not clean up descendant processes. Wrapupin thetry/finallyblock and terminate the subprocess group during cancellation.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/tasks/war_game/__main__.py` around lines 25 - 38, Update the war-game task’s up/teardown flow so sandbox cleanup is registered before invoking `up`, ensuring SIGTERM during `up` still reaches teardown. In the teardown logic, terminate the subprocess process group rather than only the direct process, while preserving normal cleanup and cancellation behavior.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: d58ccc27-2434-466b-b2b6-71af8e07923b
⛔ Files ignored due to path filters (3)
uv.lockis excluded by!**/*.lockweb/packages/sdk/generated/iron-swarm/api.tsis excluded by!**/generated/**web/pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (116)
.gitignoredocs/iron-swarm-review/findings.mdplugins/nemo-iron-swarm/.gitignoreplugins/nemo-iron-swarm/README.mdplugins/nemo-iron-swarm/openapi/openapi.yamlplugins/nemo-iron-swarm/pyproject.tomlplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/_perms.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/agent_resolver.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/_filters.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/events.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/jobs.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/manifests.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/runs.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/schemas.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/authz.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/checks.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/client.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/credentials.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/main.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/cli/provisioning.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/config.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/entities.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/filesets.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/_common.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/artifacts.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/benign_suite.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/defenses.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/errors.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/execution.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/hitl.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/manifest.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/records.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/run.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/spec.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/synth_benign.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/jobs/synth_client.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/model_config.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/model_preflight.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/sdk.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/service.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/skills.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/skills/iron-swarm/SKILL.mdplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/tasks/synth_benign/__main__.pyplugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/tasks/war_game/__main__.pyplugins/nemo-iron-swarm/tests/unit/_doubles.pyplugins/nemo-iron-swarm/tests/unit/test_agent_resolver.pyplugins/nemo-iron-swarm/tests/unit/test_api_manifests.pyplugins/nemo-iron-swarm/tests/unit/test_api_runs.pyplugins/nemo-iron-swarm/tests/unit/test_apply_mitigation.pyplugins/nemo-iron-swarm/tests/unit/test_artifacts.pyplugins/nemo-iron-swarm/tests/unit/test_benign_suite.pyplugins/nemo-iron-swarm/tests/unit/test_compose_defense.pyplugins/nemo-iron-swarm/tests/unit/test_errors.pyplugins/nemo-iron-swarm/tests/unit/test_events.pyplugins/nemo-iron-swarm/tests/unit/test_filesets.pyplugins/nemo-iron-swarm/tests/unit/test_garak_provision.pyplugins/nemo-iron-swarm/tests/unit/test_model_config.pyplugins/nemo-iron-swarm/tests/unit/test_model_preflight.pyplugins/nemo-iron-swarm/tests/unit/test_operator_env.pyplugins/nemo-iron-swarm/tests/unit/test_preflight.pyplugins/nemo-iron-swarm/tests/unit/test_run_cli.pyplugins/nemo-iron-swarm/tests/unit/test_run_record.pyplugins/nemo-iron-swarm/tests/unit/test_run_service.pyplugins/nemo-iron-swarm/tests/unit/test_sanity_check_cli.pyplugins/nemo-iron-swarm/tests/unit/test_sdk_resources.pyplugins/nemo-iron-swarm/tests/unit/test_service.pyplugins/nemo-iron-swarm/tests/unit/test_synth_benign.pyplugins/nemo-iron-swarm/tests/unit/test_synth_hitl.pypyproject.tomlservices/studio/src/nmp/studio/env_mappings.pyweb/packages/sdk/orval/constants.tsweb/packages/sdk/package.jsonweb/packages/studio/env/.env.fastapiweb/packages/studio/package.jsonweb/packages/studio/src/api/ironSwarm.tsweb/packages/studio/src/components/dataViews/IronSwarmManifestsDataView/index.tsxweb/packages/studio/src/components/dataViews/IronSwarmRunsDataView/index.tsxweb/packages/studio/src/components/ironSwarm/BenignInterviewCard.tsxweb/packages/studio/src/components/ironSwarm/BenignSuiteEditor.tsxweb/packages/studio/src/components/ironSwarm/BenignSuiteTable.tsxweb/packages/studio/src/components/ironSwarm/HardenPanel.tsxweb/packages/studio/src/components/ironSwarm/InterviewPanel.tsxweb/packages/studio/src/components/ironSwarm/ModelGroupFields.tsxweb/packages/studio/src/components/ironSwarm/ProjectManifestWizard.tsxweb/packages/studio/src/components/ironSwarm/ReconChecklist.tsxweb/packages/studio/src/components/ironSwarm/ReviewPanel.tsxweb/packages/studio/src/components/ironSwarm/SanityCheckReport.tsxweb/packages/studio/src/components/ironSwarm/TargetPanel.tsxweb/packages/studio/src/components/ironSwarm/YamlDiff.tsxweb/packages/studio/src/components/ironSwarm/eventTypes.tsweb/packages/studio/src/components/ironSwarm/hitlTypes.tsweb/packages/studio/src/components/ironSwarm/swarm/MessageFeed.tsxweb/packages/studio/src/components/ironSwarm/swarm/NodeDetail.tsxweb/packages/studio/src/components/ironSwarm/swarm/SwarmGraph.tsxweb/packages/studio/src/components/ironSwarm/swarm/swarmModel.test.tsweb/packages/studio/src/components/ironSwarm/swarm/swarmModel.tsweb/packages/studio/src/components/ironSwarm/swarm/useSwarmEvents.tsweb/packages/studio/src/components/ironSwarm/useGenerateBenignSuite.tsweb/packages/studio/src/components/ironSwarm/useMitigations.test.tsweb/packages/studio/src/components/ironSwarm/useMitigations.tsweb/packages/studio/src/components/ironSwarm/useRunWarGame.tsweb/packages/studio/src/components/ironSwarm/useSanityCheck.tsweb/packages/studio/src/constants/environment.tsweb/packages/studio/src/constants/featureFlags/featureFlags.tsweb/packages/studio/src/constants/routes.tsweb/packages/studio/src/routes/IronSwarmManifestDetailRoute/index.tsxweb/packages/studio/src/routes/IronSwarmManifestListRoute/index.tsxweb/packages/studio/src/routes/IronSwarmRunDetailsRoute/index.tsxweb/packages/studio/src/routes/IronSwarmRunListRoute/index.tsxweb/packages/studio/src/routes/NewIronSwarmManifestRoute/index.tsxweb/packages/studio/src/routes/WorkspaceLayout/WorkspaceSideNav.tsxweb/packages/studio/src/routes/groups/index.tsweb/packages/studio/src/routes/groups/ironSwarmRoutes.tsxweb/packages/studio/src/routes/index.tsxweb/packages/studio/src/routes/utils.tsweb/packages/studio/src/tests/title-change.test.tsx
| if body.secrets_file: | ||
| cmd += ["--secrets-file", body.secrets_file] |
There was a problem hiding this comment.
🔒 Security & Privacy | 🔴 Critical | ⚡ Quick win
Validate body.secrets_file before passing it to the subprocess.
secrets_file is a client-supplied path that the service passes to iron-swarm init --secrets-file. The subprocess runs on the platform host, so a caller can point it at any readable host file (for example /proc/self/environ or another tenant's dotenv) and have its contents folded into the manifest. Constrain the value to a path inside the extracted project directory, or drop the field from the API and require secrets to come from the platform Secrets store.
🛡️ Proposed containment
if body.secrets_file:
- cmd += ["--secrets-file", body.secrets_file]
+ candidate = (project_dir / body.secrets_file).resolve()
+ if not candidate.is_relative_to(project_dir.resolve()):
+ raise ValueError("secrets_file must be inside the uploaded project.")
+ cmd += ["--secrets-file", str(candidate)]📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if body.secrets_file: | |
| cmd += ["--secrets-file", body.secrets_file] | |
| if body.secrets_file: | |
| candidate = (project_dir / body.secrets_file).resolve() | |
| if not candidate.is_relative_to(project_dir.resolve()): | |
| raise ValueError("secrets_file must be inside the uploaded project.") | |
| cmd += ["--secrets-file", str(candidate)] |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@plugins/nemo-iron-swarm/src/nemo_iron_swarm_plugin/api/v2/manifests.py`
around lines 383 - 384, Validate body.secrets_file before appending it in the
manifest initialization command: resolve the supplied path and require it to
remain within the extracted project directory, rejecting traversal and
host-absolute paths; otherwise remove this API field and rely on the platform
Secrets store. Ensure only the validated path is passed to the subprocess.
Source: Linters/SAST tools
| useEffect(() => { | ||
| if (!data?.events?.length) return; | ||
| const next: SwarmEvent[] = data.events.map((e: Record<string, unknown>) => ({ | ||
| id: typeof e['id'] === 'number' ? e['id'] : Date.now(), | ||
| event: typeof e['event'] === 'string' ? e['event'] : '', | ||
| payload: (e['payload'] ?? {}) as Record<string, unknown>, | ||
| ts: Date.now(), | ||
| })); | ||
| setAllEvents((prev) => [...prev, ...next]); | ||
| setAfterId(next[next.length - 1].id); | ||
| }, [data]); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🔴 Critical | ⚡ Quick win
Date.now() id fallback poisons the cursor.
Line 39 substitutes Date.now() when an event has no numeric id. Line 45 then sets afterId from that value. An epoch-millisecond cursor is far above any real event id, so the server returns nothing afterwards and the feed stops permanently. The same fallback also creates duplicate React keys in MessageFeed.
Skip events without a numeric id, and advance the cursor from the maximum real id only.
🐛 Proposed fix
- const next: SwarmEvent[] = data.events.map((e: Record<string, unknown>) => ({
- id: typeof e['id'] === 'number' ? e['id'] : Date.now(),
- event: typeof e['event'] === 'string' ? e['event'] : '',
- payload: (e['payload'] ?? {}) as Record<string, unknown>,
- ts: Date.now(),
- }));
+ const next: SwarmEvent[] = data.events
+ .filter((e: Record<string, unknown>) => typeof e['id'] === 'number')
+ .map((e: Record<string, unknown>) => ({
+ id: e['id'] as number,
+ event: typeof e['event'] === 'string' ? e['event'] : '',
+ payload: (e['payload'] ?? {}) as Record<string, unknown>,
+ ts: Date.now(),
+ }));
+ if (next.length === 0) return;
setAllEvents((prev) => [...prev, ...next]);
- setAfterId(next[next.length - 1].id);
+ setAfterId((prev) => Math.max(prev, ...next.map((e) => e.id)));📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| useEffect(() => { | |
| if (!data?.events?.length) return; | |
| const next: SwarmEvent[] = data.events.map((e: Record<string, unknown>) => ({ | |
| id: typeof e['id'] === 'number' ? e['id'] : Date.now(), | |
| event: typeof e['event'] === 'string' ? e['event'] : '', | |
| payload: (e['payload'] ?? {}) as Record<string, unknown>, | |
| ts: Date.now(), | |
| })); | |
| setAllEvents((prev) => [...prev, ...next]); | |
| setAfterId(next[next.length - 1].id); | |
| }, [data]); | |
| useEffect(() => { | |
| if (!data?.events?.length) return; | |
| const next: SwarmEvent[] = data.events | |
| .filter((e: Record<string, unknown>) => typeof e['id'] === 'number') | |
| .map((e: Record<string, unknown>) => ({ | |
| id: e['id'] as number, | |
| event: typeof e['event'] === 'string' ? e['event'] : '', | |
| payload: (e['payload'] ?? {}) as Record<string, unknown>, | |
| ts: Date.now(), | |
| })); | |
| if (next.length === 0) return; | |
| setAllEvents((prev) => [...prev, ...next]); | |
| setAfterId((prev) => Math.max(prev, ...next.map((e) => e.id))); | |
| }, [data]); |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@web/packages/studio/src/components/ironSwarm/swarm/useSwarmEvents.ts` around
lines 36 - 46, Update the event mapping in the useSwarmEvents effect to discard
events whose id is not numeric instead of assigning Date.now(). Append only
valid events, and advance afterId using the maximum real event id from the
accepted events, preserving the existing behavior when no valid events are
returned.
1 participant
Adds the
nemo-iron-swarmplugin: a security war-game that red-teams and hardens NAT agents. garakattackers probe a sandboxed copy of the agent, defenders generate guardrails and OpenShell network
policy, and validators replay the attacks plus a benign suite to confirm the fix blocked the
attack without breaking ordinary behaviour.
The plugin contributes a service, a job, a CLI, an SDK namespace, two entities and an agent skill.
It never imports iron-swarm: garak pulls
litellm → httpx>=0.28andtorch, which conflict withnvidia-nat'shttpx~=0.27, so iron-swarm is provisioned into its own venv bynemo iron-swarm setupand driven by subprocess. Nothing breaks when it isn't installed, and theStudio tab is behind
studio.feature_flags.iron_swarm_enabled, off by default.What's here
init --agent <name>for any registered agent (it need not bedeployed), or
init --project-dir <path>for a local NAT project with nothing registered. Bothsave the same
IronSwarmManifest, so--manifest-idis the single handle afterwards. CLI andStudio go through the same
POST /manifests.runis a pure consumer of the benign suite. It never synthesises;synth-benignis aseparate command and job, and the reviewed suite is cached on the manifest. A missing suite fails
fast rather than doing something surprising.
initresolves once and stores the resulting scaffold as afileset; runs download it instead of re-resolving, so two runs are comparable — which is what a
"did the hardening help?" answer depends on. Agent edits land via an explicit
POST /manifests/{name}/refresh, whichapply-mitigationcalls automatically.git ls-files --exclude-standardand drop.env*/*.pem/*.key, reusingnemo-agents'DOCKERIGNORE_TEMPLATE. Victim secrets come from the platform Secrets store by name.refresh and an editor for non-secret env vars.
Verification
280 unit tests, plus real war-games against the bundled
react-agentand theresearchagent fromagents-labuploaded as a project. The second is the interesting result — attacks 11/11 blocked,benign 8/12 passed (3 refused, 1 error): the defenses stopped everything but broke a third of
normal functionality, which is exactly what the benign suite exists to catch.
Studio flows were driven end-to-end with Playwright: manifest creation, the target view, refresh,
the env editor, and a full benign-suite generation including the HITL interview.
Security notes for review
overrides.defenders[].implementationis aPython import path that iron-swarm loads in the job process (it logs
importing user-configured module … ensure the source is trusted), so it executes on the platform host with the job'senvironment and Docker access — not inside the victim sandbox. A free-text YAML editor would turn
manifest-write permission into code execution.
envon a manifest is stored in plaintext and documented as non-secret; credentials usesecrets,which stores only names and resolves values from the Secrets store at run time.
Correct for local/dev, explicitly not a production deployment — containerising the orchestrator
and decoupling the sandbox from a host Docker daemon is the Phase-2 item.
Known issues, logged rather than hidden
then fails on a name collision, so one fault presents as a different fault each time. Cancellation
tears down correctly — the failure paths do not.
completedwhile persisting nothing, and a cachedbenign_interviewis not cleared when a new suite lands without one.Not in this PR
Routing attack and detection through the NeMo Auditor (NAIS-356/357) is prototyped on a separate
branch and deliberately excluded here.
Summary by CodeRabbit
New Features
Documentation
Bug Fixes