-
Notifications
You must be signed in to change notification settings - Fork 378
feat: Add outage-detector kit #263
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
amanbud4530
wants to merge
25
commits into
Lamatic:main
Choose a base branch
from
amanbud4530:add-outage-detector
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
25 commits
Select commit
Hold shift + click to select a range
30202b4
feat: Add outage-detector kit
amanbud4530 e910d72
Fix lamatic dependency version and add gitignore
amanbud4530 9ddcc39
Trigger Vercel redeploy
amanbud4530 746278b
Update kits/outage-detector/lamatic.config.ts
amanbud4530 9eae653
refactor: move config/workflowId resolution to orchestrate.ts
amanbud4530 d115aa8
refactor: move config/workflowId resolution to orchestrate.ts
amanbud4530 047e586
refactor: extract colors into CSS custom properties
amanbud4530 fdd54ab
refactor: migrate to Tailwind CSS v4 utilities and shadcn/ui Button
amanbud4530 9d4b36e
fix: remove duplicate synthetic_tickets.json, reference canonical pub…
amanbud4530 cf0cc45
chore: add mandated dependencies (tailwindcss, shadcn support, react-…
amanbud4530 85451f5
security: treat ticket content as untrusted data in default constitution
amanbud4530 256623c
fix: ground drafting agent's impacted accounts in real candidate records
amanbud4530 c6c22b4
docs: clarify shared vector store in agent.md
amanbud4530 eb84e12
docs: add language identifier to flow diagram code block
amanbud4530 f95d7ab
fix: correctly highlight active ticket in queue, including index=0
amanbud4530 9414341
fix: revert unrelated root .gitignore change, add kit-root .env.example
amanbud4530 5afd29f
Trigger fresh Vercel deployment
amanbud4530 1a062de
docs: add live demo link to lamatic.config.ts
amanbud4530 bdb0427
fix: add config_json export for Studio Phase 2 validator
amanbud4530 e8bcc52
fix: correct lamatic.config.ts path in kit-root .env.example comment
amanbud4530 f8013dd
fix: normalize array-wrapped status field from flow response
amanbud4530 c03c2dd
fix: pin lamatic dependency to tested version, regenerate lockfile
amanbud4530 0813a64
fix: include config_json in default export alongside named export
amanbud4530 5603919
fix: use bare closing code fences in README.md
amanbud4530 45f9c8c
fix: remove duplicate edge from addNode_601 to responseNode_triggerNo…
amanbud4530 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| # Copy this file to .env.local and fill in real values from studio.lamatic.ai | ||
| # (Settings > API Keys for LAMATIC_API_KEY, Settings > Project for LAMATIC_PROJECT_ID, | ||
| # Settings > API Docs > Endpoint for LAMATIC_API_URL) | ||
|
|
||
| LAMATIC_API_KEY= | ||
| LAMATIC_PROJECT_ID= | ||
| LAMATIC_API_URL= | ||
|
|
||
| # Deployed Flow ID for the Outage Detector flow (matches the envKey | ||
| # declared in lamatic.config.ts) | ||
| OUTAGE_DETECTOR= | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| apps/node_modules/ | ||
| apps/.next/ | ||
| apps/.env | ||
| apps/.env.local | ||
| apps/.env*.local | ||
| .DS_Store |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| # Outage Detector | ||
|
|
||
| A Lamatic **kit**: a flow plus a runnable Next.js app that correlates a new | ||
| support ticket against ticket history to catch a shared outage before it | ||
| looks like a pattern to a human — verifying genuine root-cause correlation | ||
| rather than surface wording similarity. | ||
|
|
||
| ## What it does | ||
|
|
||
| 1. **Retrieves** semantically similar historical tickets via vector search. | ||
| 2. **Verifies** whether those candidates share a genuine root cause with the | ||
| new ticket — same technical component, same failure mode, a plausible | ||
| single upstream cause, and consistent timing — rather than just similar | ||
| wording. Tickets with a self-inflicted cause on the customer's own side | ||
| (e.g. an expired API key) are explicitly rejected even if the wording is | ||
| superficially similar to a real cluster. | ||
| 3. **Routes** on a confidence threshold: only genuinely correlated tickets | ||
| get flagged. | ||
| 4. **Drafts** an internal note and a customer-facing message, grounded in | ||
| the actual correlated tickets — not generic boilerplate. | ||
|
|
||
| ## Flow structure | ||
|
|
||
| ```text | ||
| API Trigger | ||
| → Vector Search (top 8, certainty >= 0.7) | ||
| → Vectorize → VectorDB write (indexes the new ticket for future searches) | ||
| → Correlation Verification Agent (JSON Agent) | ||
| → Condition (confidence >= 0.75) | ||
| ├── "Condition 1" → Drafting Agent (internal_note + customer_message) | ||
| └── "Else" → passthrough (fields stay empty) | ||
| → API Response | ||
| ``` | ||
|
|
||
| Full node-level detail, error scenarios, and design notes are documented | ||
| in the docblock at the top of `flows/outage-detector.ts`. | ||
|
|
||
| ## Running the app locally | ||
|
|
||
| ```bash | ||
| cd apps | ||
| cp .env.example .env.local # fill in real values — see below | ||
| npm install | ||
| npm run dev | ||
| ``` | ||
|
|
||
| Then open `http://localhost:3000`. The demo steps through a queue of | ||
| synthetic tickets (`apps/public/data/synthetic_tickets.json`) one at a time, | ||
| submitting each to the deployed flow and showing the response live. A | ||
| genuine cluster (T-1005, T-1007, T-1011) and two decoys (T-1009, T-1017) | ||
| are seeded into the queue — watch the right panel for when the flow | ||
| catches the pattern on T-1013. | ||
|
|
||
| ## Required credentials | ||
|
|
||
| | Env var | Where to find it | | ||
| |---|---| | ||
| | `LAMATIC_API_KEY` | Studio: Settings → API Keys | | ||
| | `LAMATIC_PROJECT_ID` | Studio: Settings → Project → Project ID | | ||
| | `LAMATIC_API_URL` | Studio: Settings → API Docs → API → Endpoint | | ||
| | `OUTAGE_DETECTOR` | The deployed flow's ID — open the flow, check the URL or the details panel (⋮ menu) | | ||
|
|
||
| You'll also need, inside the flow itself (configured in Studio, not via env | ||
| vars — see `flows/outage-detector.ts`'s `inputs` export for the exact | ||
| fields): | ||
| - A Vector Store (developed against one named `support-tickets`) | ||
| - An embedding model credential (developed against Cohere `embed-english-v3.0`) | ||
| - An LLM credential for both JSON Agent nodes (developed against a | ||
| Groq-hosted model, e.g. Llama 3.3 70B) | ||
|
|
||
| ## Known caveats | ||
|
|
||
| - There are two independent thresholds in this flow: Vector Search's own | ||
| `certainty >= 0.7` (which tickets even become candidates) and the | ||
| Condition node's `confidence >= 0.75` (whether a verified match gets | ||
| flagged). Don't conflate them when tuning. | ||
| - `internal_note`/`customer_message` are legitimately empty strings on the | ||
| "Else" branch — that's expected, not a bug. | ||
| - The demo app sends only the current ticket per request — no client-side | ||
| batching. All correlation happens server-side inside the flow itself, so | ||
| the vector store must build up across the sequence of submissions. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| # Outage Detector | ||
|
|
||
| ## Identity | ||
|
|
||
| Outage Detector is a correlation-verification agent for customer support | ||
| teams. Given a new support ticket, it determines whether the ticket is part | ||
| of a genuine, shared technical outage already visible in ticket history — | ||
| as opposed to a ticket that merely uses similar words to describe an | ||
| unrelated or self-inflicted problem. | ||
|
|
||
| ## Capabilities | ||
|
|
||
| - Retrieves semantically similar historical tickets via vector search. | ||
| - Verifies whether retrieved candidates share a genuine root cause with the | ||
| new ticket: same technical component, same failure mode, a plausible | ||
| single upstream cause (a cert rotation, a deploy, a third-party outage), | ||
| and timing consistent with one shared triggering event. | ||
| - Explicitly rejects candidates that are surface-similar but not a real | ||
| match — most commonly, tickets describing a self-inflicted cause on the | ||
| customer's own side (an expired credential, their own misconfiguration). | ||
| - On a confirmed match above a confidence threshold, drafts a grounded | ||
| internal note (impacted accounts, suspected component, recommended next | ||
| step) and a customer-facing message — both referencing the actual | ||
| correlated tickets rather than generic boilerplate. | ||
| - Indexes every incoming ticket into the shared `support-tickets` vector | ||
| store configured by the outage-detector flow, so correlation quality | ||
| improves as more tickets are processed. | ||
|
|
||
| ## Non-goals | ||
|
|
||
| - Does not resolve the underlying technical issue — it identifies and | ||
| drafts, a human still acts. | ||
| - Does not deduplicate or merge tickets in an external ticketing system; | ||
| it returns a structured recommendation for the calling application to | ||
| act on. | ||
| - Does not perform multi-hop reasoning across unrelated ticket categories | ||
| (e.g. billing vs. infrastructure) — correlation is scoped to genuinely | ||
| plausible shared technical root causes. | ||
|
|
||
| ## Intended callers | ||
|
|
||
| Support ticketing systems or their middleware, submitting one new ticket | ||
| at a time via the flow's API trigger, and consuming the structured | ||
| response (`status`, `confidence`, `matched_ticket_ids`, `suspected_component`, | ||
| `reasoning`, `internal_note`, `customer_message`) to decide whether to | ||
| surface an outage alert to a human. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| # Copy this file to .env.local and fill in real values from studio.lamatic.ai | ||
| # (Settings > API Keys for LAMATIC_API_KEY, Settings > Project for LAMATIC_PROJECT_ID, | ||
| # Settings > API Docs > Endpoint for LAMATIC_API_URL) | ||
|
|
||
| LAMATIC_API_KEY= | ||
| LAMATIC_PROJECT_ID= | ||
| LAMATIC_API_URL= | ||
|
|
||
| # Deployed Flow ID for the Outage Detector flow (matches the envKey | ||
| # declared in ../lamatic.config.ts) | ||
| OUTAGE_DETECTOR= |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| "use server"; | ||
|
|
||
| import { submitTicket, TicketPayload, FlowResult } from "@/lib/lamatic-client"; | ||
| import lamaticConfig from "../../lamatic.config"; | ||
|
|
||
| export async function processTicket(ticket: TicketPayload): Promise<FlowResult> { | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
| try { | ||
| const step = lamaticConfig.steps.find((s) => s.id === "outage-detector"); | ||
| const workflowEnvKey = step?.envKey ?? "OUTAGE_DETECTOR"; | ||
| const workflowId = process.env[workflowEnvKey]; | ||
|
|
||
| if (!workflowId) { | ||
| throw new Error( | ||
| `Workflow ID environment variable "${workflowEnvKey}" is not set. Please add it to your .env.local file.` | ||
| ); | ||
| } | ||
|
|
||
| return await submitTicket(ticket, workflowId); | ||
| } catch (err) { | ||
| console.error("Flow execution failed:", err); | ||
| return { | ||
| status: "Else", | ||
| confidence: 0, | ||
| matched_ticket_ids: [], | ||
| suspected_component: "unknown", | ||
| reasoning: | ||
| "Flow call failed — check OUTAGE_DETECTOR, LAMATIC_API_KEY, LAMATIC_PROJECT_ID, and LAMATIC_API_URL in .env.local", | ||
| internal_note: "", | ||
| customer_message: "", | ||
| }; | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| @import "tailwindcss"; | ||
|
|
||
| @theme { | ||
| --color-background: #0b0d12; | ||
| --color-foreground: #e6e8ec; | ||
| --color-muted: #9aa1ac; | ||
| --color-dim: #6b7280; | ||
| --color-subtle: #b7bcc5; | ||
|
|
||
| --color-accent: #4f7cff; | ||
| --color-disabled: #2a2f3a; | ||
|
|
||
| --color-panel: #12151c; | ||
| --color-panel-border: #232733; | ||
|
|
||
| --color-flagged: #4a1d1d; | ||
| --color-flagged-foreground: #ff8a8a; | ||
| --color-normal: #1d2a1d; | ||
| --color-normal-foreground: #8dff8a; | ||
| } | ||
|
|
||
| body { | ||
| margin: 0; | ||
| font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; | ||
| background: var(--color-background); | ||
| color: var(--color-foreground); | ||
| } | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| import "./globals.css"; | ||
|
|
||
| export const metadata = { | ||
| title: "Outage Detector", | ||
| description: | ||
| "Cross-ticket root-cause correlation demo built on Lamatic — catches a shared outage before it looks like a pattern to a human.", | ||
| }; | ||
|
|
||
| export default function RootLayout({ children }: { children: React.ReactNode }) { | ||
| return ( | ||
| <html lang="en"> | ||
| <body>{children}</body> | ||
| </html> | ||
| ); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,131 @@ | ||
| "use client"; | ||
|
|
||
| import { useState, useEffect } from "react"; | ||
| import { processTicket } from "@/actions/orchestrate"; | ||
| import type { TicketPayload, FlowResult } from "@/lib/lamatic-client"; | ||
| import { Button } from "@/components/ui/button"; | ||
| import { cn } from "@/lib/utils"; | ||
|
|
||
| type LogEntry = { ticket: TicketPayload; result: FlowResult }; | ||
|
|
||
| export default function Page() { | ||
| const [tickets, setTickets] = useState<TicketPayload[]>([]); | ||
| const [index, setIndex] = useState(0); | ||
| const [log, setLog] = useState<LogEntry[]>([]); | ||
| const [loading, setLoading] = useState(false); | ||
|
|
||
| useEffect(() => { | ||
| fetch("/data/synthetic_tickets.json") | ||
| .then((r) => r.json()) | ||
| .then((data) => setTickets(data.tickets)); | ||
| }, []); | ||
|
|
||
| async function stepForward() { | ||
| if (index >= tickets.length) return; | ||
| setLoading(true); | ||
| const ticket = tickets[index]; | ||
| const result = await processTicket(ticket); | ||
| setLog((prev) => [{ ticket, result }, ...prev]); | ||
| setIndex((i) => i + 1); | ||
| setLoading(false); | ||
| } | ||
|
|
||
| return ( | ||
| <div className="mx-auto max-w-[880px] px-6 pb-20 pt-10"> | ||
| <h1 className="text-2xl font-semibold mb-1">Outage Detector</h1> | ||
| <p className="text-muted text-[0.95rem] leading-relaxed mb-7"> | ||
| Steps through {tickets.length || "..."} synthetic tickets one at a | ||
| time. Each submission both queries and writes to the flow's | ||
| vector store, so the store builds up as you go — a hidden cluster | ||
| (T-1005, T-1007, T-1011) and two decoys (T-1009, T-1017) are in | ||
| there. Watch the right panel for when it catches on. | ||
| </p> | ||
|
|
||
| <Button onClick={stepForward} disabled={loading || index >= tickets.length}> | ||
| {loading ? "Processing…" : `Submit next ticket (${index}/${tickets.length})`} | ||
| </Button> | ||
|
|
||
| <div className="grid grid-cols-1 md:grid-cols-2 gap-5 mt-6"> | ||
| <div className="bg-panel border border-panel-border rounded-[10px] p-4"> | ||
| <h3 className="mt-0 mb-3 text-sm uppercase tracking-wide text-muted"> | ||
| Ticket queue | ||
| </h3> | ||
| {(() => { | ||
| const sliceStart = Math.max(0, index - 1); | ||
| return tickets.slice(sliceStart, index + 3).map((t, i) => ( | ||
| <div | ||
| key={t.ticket_id} | ||
| className={cn( | ||
| "py-2.5 text-[0.9rem] border-b border-panel-border last:border-none", | ||
| sliceStart + i === index && "text-foreground font-semibold" | ||
| )} | ||
| > | ||
| <strong>{t.ticket_id}</strong> · {t.account_name} ({t.account_tier}) | ||
| <div>{t.subject}</div> | ||
| </div> | ||
| )); | ||
| })()} | ||
| </div> | ||
|
|
||
| <div className="bg-panel border border-panel-border rounded-[10px] p-4"> | ||
| <h3 className="mt-0 mb-3 text-sm uppercase tracking-wide text-muted"> | ||
| Flow output | ||
| </h3> | ||
| {log.length === 0 && <p>No tickets submitted yet.</p>} | ||
| {log.map((entry, i) => { | ||
| const flagged = entry.result.status === "Condition 1"; | ||
| return ( | ||
| <div | ||
| key={i} | ||
| className="py-2.5 text-[0.9rem] border-b border-panel-border last:border-none" | ||
| > | ||
| <span | ||
| className={cn( | ||
| "inline-block px-2 py-0.5 rounded-full text-xs font-semibold mr-2", | ||
| flagged | ||
| ? "bg-flagged text-flagged-foreground" | ||
| : "bg-normal text-normal-foreground" | ||
| )} | ||
| > | ||
| {flagged ? "flagged" : "normal"} | ||
|
amanbud4530 marked this conversation as resolved.
|
||
| </span> | ||
| <strong>{entry.ticket.ticket_id}</strong> — {entry.ticket.subject} | ||
| <div className="mt-1 text-[0.85rem] text-subtle"> | ||
| <span className="text-dim">Confidence:</span> {entry.result.confidence} | ||
| </div> | ||
| {entry.result.matched_ticket_ids.length > 0 && ( | ||
| <div className="mt-1 text-[0.85rem] text-subtle"> | ||
| <span className="text-dim">Matched:</span>{" "} | ||
| {entry.result.matched_ticket_ids.join(", ")} | ||
| </div> | ||
| )} | ||
| {entry.result.suspected_component && ( | ||
| <div className="mt-1 text-[0.85rem] text-subtle"> | ||
| <span className="text-dim">Suspected component:</span>{" "} | ||
| {entry.result.suspected_component} | ||
| </div> | ||
| )} | ||
| {entry.result.reasoning && ( | ||
| <div className="mt-1 text-[0.85rem] text-subtle">{entry.result.reasoning}</div> | ||
| )} | ||
| {flagged && entry.result.internal_note && ( | ||
| <div className="mt-1 text-[0.85rem] text-subtle"> | ||
| <span className="text-dim">Internal note:</span>{" "} | ||
| {entry.result.internal_note} | ||
| </div> | ||
| )} | ||
| {flagged && entry.result.customer_message && ( | ||
| <div className="mt-1 text-[0.85rem] text-subtle"> | ||
| <span className="text-dim">Customer message:</span>{" "} | ||
| {entry.result.customer_message} | ||
| </div> | ||
| )} | ||
| </div> | ||
| ); | ||
| })} | ||
| </div> | ||
| </div> | ||
| </div> | ||
| ); | ||
| } | ||
|
|
||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.