Skip to content

docs(weather-agent): UI-first rewrite of advanced demo + LLM setup#466

Merged
mrsabath merged 4 commits into
rossoctl:mainfrom
mrsabath:docs/weather-advanced-llm-config
Jun 3, 2026
Merged

docs(weather-agent): UI-first rewrite of advanced demo + LLM setup#466
mrsabath merged 4 commits into
rossoctl:mainfrom
mrsabath:docs/weather-advanced-llm-config

Conversation

@mrsabath

@mrsabath mrsabath commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Restructure authbridge/demos/weather-agent/demo-ui-advanced.md as a UI-first walkthrough. The previous version was a kubectl-heavy procedure with the UI flow buried in "Step 4 (Optional)"; reviewers (and a real user run-through) found it too verbose for what is meant to be a UI demo.

  • Lead with what changes vs. the standard demo (single comparison table — AuthBridge on the tool + RFC 8693 token exchange + Keycloak audience scopes).
  • Primary flow is now Kagenti UI, mirroring authbridge/demos/github-issue/demo-ui.md:
    • Step 1: one Keycloak setup script invocation.
    • Step 2: import the tool via the UI (with AuthBridge + SPIRE checked — the differentiator vs. standard).
    • Step 3: import the agent via the UI. LLM env vars loaded via Import from URL using the existing beginner .env.openai / .env.ollama files, with one MCP_URL edit to retarget the advanced tool. Outbound Routing Rules expander replaces the manual configmaps-advanced.yaml apply for backends with kagenti#1194; the kubectl fallback is a single line for older builds.
    • Step 4: chat in the UI.
    • Step 5 (optional): deploy_and_verify_advanced.sh for CLI / CI users.
  • Demote the kubectl flow (manifest applies, AgentRuntime gotchas, raw OpenAI patches) to a clearly-marked Appendix: kubectl-only path so it's still discoverable but no longer dominates.
  • Keep the LLM troubleshooting rows added in the first revision of this PR — the underlying gap (deploy_and_verify_advanced.sh doesn't exercise the LLM path, so the script can succeed while UI chat fails with Connection error or No LLM API key configured) still exists and is the demo's most common surprise.

Coordination with #439

#439 updates the standard demo's Installer-Provided Resources section to reflect that keycloak-admin-secret lives in kagenti-system (operator 0.2+), not team1. This PR mirrors that exact phrasing in the advanced demo's Prerequisites so both files stay consistent. Different files; no merge conflict.

Test plan

  • Walked the new Step 1–4 UI flow against a Kind cluster; UI chat returns weather successfully with OpenAI configured via the .env.openai import + openai-secret.
  • Reproduced both Error: LLM execution failed: Connection error. (Ollama not running) and Error: No LLM API key configured. (empty openai-secret) and confirmed the new troubleshooting rows resolve them.
  • Confirmed the kubectl appendix still contains the AgentRuntime + spec.type: agent + client-registration-label gotchas that were in the original doc.
  • DCO sign-off + Assisted-By trailer on both commits; commit-msg hook passes.

Assisted-By: Claude Code

Summary by CodeRabbit

  • Documentation
    • Clarified that chat-style UI verification requires a configured LLM provider, while the alternate CLI verification path does not call the LLM and can succeed without it.
    • Added explicit guidance for OpenAI secret handling and a separate note for an Ollama outbound-port requirement.
    • Documented a UI routing rule for RFC 8693 token exchange.
    • Expanded troubleshooting with UI-specific LLM error mappings, import guidance for empty secrets, and concrete remediation (refresh secrets, redeploy/restart).

deploy_and_verify_advanced.sh exercises only the AuthBridge / MCP
token-exchange path, so it can succeed while the Kagenti UI still
returns "LLM execution failed: Connection error." or "No LLM API key
configured." — both happen when the agent cannot reach the configured
LLM.

Restructure the LLM section into two explicit options (Ollama default
vs. OpenAI) with the kubectl secret + env-patch recipe, note the
duplicate-LLM_API_KEY behavior of the patch flow, and add troubleshooting
rows tying each UI error message to its fix. Also flag in Prerequisites
that the verify script does not exercise the LLM path.

Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Mariusz Sabath <mrsabath@gmail.com>
@coderabbitai

coderabbitai Bot commented Jun 3, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e67691e7-e315-4a0a-b0f7-0df5302a9be4

📥 Commits

Reviewing files that changed from the base of the PR and between 20759bd and 70d17da.

📒 Files selected for processing (1)
  • authbridge/demos/weather-agent/demo-ui-advanced.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • authbridge/demos/weather-agent/demo-ui-advanced.md

📝 Walkthrough

Walkthrough

Docs now state that chat-style UI verification requires a configured LLM (Ollama or OpenAI), specify using a team1 openai-secret for LLM_API_KEY and OPENAI_API_KEY, add RFC 8693 routing guidance, note Ollama port exclusion, clarify CLI verification does not call the LLM, add kubectl secret workflows, extend troubleshooting, and link the related script.

Changes

LLM Provider Setup Documentation

Layer / File(s) Summary
Top-level prerequisites update
authbridge/demos/weather-agent/demo-ui-advanced.md
Rewrites intro/prereqs to state chat-style verification requires an LLM provider and add requirement for a team1 openai-secret Secret.
UI Import and routing rules
authbridge/demos/weather-agent/demo-ui-advanced.md
Import Agent step now requires openai-secret for both LLM_API_KEY and OPENAI_API_KEY; documents RFC 8693 token exchange routing rule and separates Ollama-only outbound port 11434 exclusion from OpenAI guidance.
UI chat JWT/token-exchange note
authbridge/demos/weather-agent/demo-ui-advanced.md
Documents expected JWT/token-exchange behavior and clarifies UI error messages are LLM-side failures routed to troubleshooting.
CLI verification behavior
authbridge/demos/weather-agent/demo-ui-advanced.md
Clarifies deploy_and_verify_advanced.sh does not call the LLM so its success/failure is independent of LLM connectivity or API keys.
Troubleshooting table additions
authbridge/demos/weather-agent/demo-ui-advanced.md
Adds two UI-specific rows mapping "Connection error" and "No LLM API key configured" to concrete causes and remediation (start/pull Ollama model or re-import OpenAI .env; recreate openai-secret and restart deployment).
kubectl OpenAI env/secret workflow
authbridge/demos/weather-agent/demo-ui-advanced.md
Adds kubectl set env/kubectl patch instructions to clear a literal LLM_API_KEY- and re-add LLM_API_KEY and OPENAI_API_KEY as secret-backed env entries.
Related files update
authbridge/demos/weather-agent/demo-ui-advanced.md
Adds deploy_and_verify_advanced.sh to the Related Files table.

Estimated Code Review Effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Suggested reviewers

  • pdettori

Poem

🐰 A rabbit reads the guide tonight,
Keys in place, the docs alight,
Ollama watches its own port,
CLI runs its separate court,
Hopped-through fixes—what a sight!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: a UI-first rewrite of the advanced weather agent demo documentation with LLM setup guidance, which directly matches the PR's primary objective of restructuring the demo into a UI-first walkthrough.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@authbridge/demos/weather-agent/demo-ui-advanced.md`:
- Line 390: In the troubleshooting command in demo-ui-advanced.md where the
secret verification uses escaped pipes (`\| base64 -d \| wc -c`), remove the
backslashes so the shell pipes work when copied; update the string to use plain
pipes (`| base64 -d | wc -c`) in the table row that mentions `kubectl get secret
openai-secret -o jsonpath='{.data.apikey}' \| base64 -d \| wc -c` so the command
executes correctly when run, and run the markdown spellcheck/linting afterward.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: fc217751-4782-4a99-88ad-c15a958d183d

📥 Commits

Reviewing files that changed from the base of the PR and between 87b529b and d2a3b3b.

📒 Files selected for processing (1)
  • authbridge/demos/weather-agent/demo-ui-advanced.md

Comment thread authbridge/demos/weather-agent/demo-ui-advanced.md Outdated
Address review feedback that the previous structure was too verbose and
too kubectl-heavy for a "UI demo." Rewrite the file end-to-end so the
primary flow runs in the Kagenti UI, mirroring the github-issue UI
demo's pattern:

- Open with a "How this differs from the standard demo" comparison table
  so users know up-front what changes vs. demo-ui.md.
- Step 1: one Keycloak script invocation.
- Step 2: import the tool via the UI with AuthBridge + SPIRE checked.
- Step 3: import the agent via the UI; LLM env loaded via Import from
  URL using the existing beginner .env.openai / .env.ollama files, with
  one MCP_URL edit to point at the advanced tool. Outbound Routing Rules
  expander replaces the manual configmaps-advanced.yaml apply for
  backends that support it (kagenti#1194); fallback to kubectl apply
  called out as a one-liner.
- Step 4: chat in the UI.
- Step 5 (optional): deploy_and_verify_advanced.sh for CLI / CI users.

Move the existing kubectl + manifest flow + AgentRuntime gotchas to a
clearly-marked "Appendix: kubectl-only path" so users who prefer the
manifest path still have it, without it dominating the doc.

Mirror PR rossoctl#439's "keycloak-admin-secret is not in team1; operator 0.2+
keeps it in kagenti-system" phrasing in Prerequisites for consistency
across the two weather demos.

Keep the LLM troubleshooting rows from the previous commit. The
underlying gap (deploy_and_verify_advanced.sh does not exercise the LLM
path) still exists and remains the demo's most common surprise.

Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Mariusz Sabath <mrsabath@gmail.com>
@mrsabath mrsabath changed the title docs(weather-agent): document OpenAI LLM setup for advanced demo docs(weather-agent): UI-first rewrite of advanced demo + LLM setup Jun 3, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@authbridge/demos/weather-agent/demo-ui-advanced.md`:
- Around line 137-139: The fenced code block containing the MCP_URL environment
line is missing a language tag (triggers MD040); update the opening fence for
the block that contains "MCP_URL=http://weather-tool-advanced-mcp:8000/mcp" to
include a language identifier such as bash (e.g., change ``` to ```bash) so the
markdown linter recognizes the language.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: ddba4ce9-5616-401e-a2e9-c92c7b836049

📥 Commits

Reviewing files that changed from the base of the PR and between d2a3b3b and 2e96753.

📒 Files selected for processing (1)
  • authbridge/demos/weather-agent/demo-ui-advanced.md

Comment thread authbridge/demos/weather-agent/demo-ui-advanced.md Outdated
Round of fixes from a real run-through of the UI flow:

- Step 2: button label is "Build & Deploy Tool" (not "Deploy Tool"); add
  explicit Target Port: 8000.
- Step 2 / Step 3: prepend a callout that the tool/agent names MUST be
  the -advanced variants. Importing as plain weather-tool /
  weather-service silently breaks token exchange because the Service
  ends up as weather-tool-mcp instead of weather-tool-advanced-mcp,
  MCP_URL stops resolving, and the Keycloak audience scopes registered
  by Step 1 don't match the workload SPIFFE.
- Step 3: reorder Outbound Routing Rules to come right after SPIRE,
  before Service Port — matches the actual import-form layout. Replace
  the labeled-column table with a positional 3-field list because the
  UI form fields are currently unlabeled.
- Step 4: note that the Agent Catalog shows BOTH workloads (the
  advanced demo labels the tool kagenti.io/type=agent so AuthBridge
  gets injected; injectTools feature gate is off by default), and the
  Tool Catalog is empty. Without the note this looks broken.
- Cleanup: also delete Shipwright Build + BuildRun. The Kagenti backend
  reconciliation service treats orphaned builds as "needs deployment"
  and re-creates the Deployment + Service + ServiceAccount within
  seconds, making it impossible to delete via kubectl alone.
- Troubleshooting: add rows for the recreated-on-delete and
  wrong-MCP-URL symptoms above.

Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Mariusz Sabath <mrsabath@gmail.com>

@pdettori pdettori left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean docs PR — UI-first rewrite is well-structured, all 3 commits properly signed-off, CI green. CodeRabbit's earlier findings (escaped pipes, missing lang tag) are resolved in the final commit.

Comment thread authbridge/demos/weather-agent/demo-ui-advanced.md
Address review nit on PR rossoctl#466: the env-var snippet inside the agent
import step had a bare ``` opener; markdownlint MD040 wants a language
tag. Tagging it `text` since it's a literal env var assignment, not a
shell command.

Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Mariusz Sabath <mrsabath@gmail.com>
@mrsabath
mrsabath merged commit abd2856 into rossoctl:main Jun 3, 2026
19 checks passed
@github-project-automation github-project-automation Bot moved this from New /:ToDo to Done in Kagenti Issue Prioritization Jun 3, 2026
@mrsabath
mrsabath deleted the docs/weather-advanced-llm-config branch June 3, 2026 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants