Skip to content

fix(addie): prevent Sage from using real company domains in A2 demo briefs#4948

Merged
bokelley merged 2 commits into
mainfrom
claude/issue-4943-a2-fictional-brand-domain
May 26, 2026
Merged

fix(addie): prevent Sage from using real company domains in A2 demo briefs#4948
bokelley merged 2 commits into
mainfrom
claude/issue-4943-a2-fictional-brand-domain

Conversation

@bokelley

Copy link
Copy Markdown
Contributor

Closes #4943

Summary

Sage was generating get_products demo briefs in the A2 certification module using scope3.com — a real, active AdCP member company domain — because the teaching rules had no prohibition on real brand names. The brief implied scope3.com was running a sustainability campaign, which may be inaccurate and creates brand-association risk.

Two-part fix:

  1. server/src/addie/mcp/certification-tools.ts — adds one line to the global TEACHING RULES block in buildCertificationContext. This rule reaches Sage for every in-progress module session, not just A2:

    "When generating any example brief, campaign scenario, or demo call, always use fictional brand domains (e.g., nova-brands.example, acme-corp.example, pinnacle-agency.example) — never real company domains, including real AdCP member domains."

  2. server/src/db/migrations/493_a2_fictional_brand_domain_rule.sql — belt-and-suspenders reinforcement in the A2 lesson plan itself. Updates three places Sage reads when loading A2: the "Directing a media buy" key concept teaching note, the demo scenario description, and the a2_ex1 exercise description. Wrapped in a transaction; leaves all other A2 fields (objectives, discussion_prompts, assessment_criteria) untouched via jsonb_set.

Non-breaking justification

Prompt text and lesson plan content change only. No schema, protocol, or API surface affected. No existing certification completions are invalidated — this is instructional scaffolding, not a scored criterion change.

Pre-PR review

  • code-reviewer: approved — jsonb_set path targeting is correct; transaction wrapper added per review feedback; no SQL injection surface; pre-existing build errors in repo are unrelated to this diff.
  • education-expert: approved — TEACHING RULE wording is unambiguous; .example TLD is RFC 2606-correct; migration covers all three read paths (buildCertificationContext, get_certification_module key_concepts, demo_scenarios, exercise_definitions). One nit noted (consider adding a per-tool brief hint at get_certification_module line ~1616 for get_products/create_media_buy analogous to the acquire_rights guard) — deferred as a follow-up.

Out of scope (noted for follow-up)

  • Broader audit of B/C/D track modules for similar real-brand leakage — separate issue.
  • Adding structured criterion IDs to a2_ex1 success_criteria — separate issue.

Triage-managed PR. This bot does not currently iterate on
review comments or PR conversation threads (only on the source
issue). To unblock:

  • Push fixup commits directly: gh pr checkout <num>
    fix → push.
  • Or re-trigger: comment /triage execute on the source
    issue.

See #3121
for context.

Session: https://claude.ai/code/session_01NvXJe2rGTewmxaM1WZz9oM


Generated by Claude Code

…riefs

Sage was generating get_products demo briefs with scope3.com — a real
AdCP member domain — because the teaching rules had no prohibition on
real brand names. Add a global TEACHING RULES entry that requires
fictional brand domains (nova-brands.example, acme-corp.example, etc.)
in all generated examples, plus a belt-and-suspenders SQL migration
that embeds the same guidance in the A2 lesson plan's key_concepts,
demo_scenarios, and exercise_definitions.

Closes #4943

https://claude.ai/code/session_01NvXJe2rGTewmxaM1WZz9oM
@bokelley bokelley added the claude-triaged Issue has been triaged by the Claude Code triage routine. Remove to re-triage. label May 22, 2026
@bokelley bokelley marked this pull request as ready for review May 26, 2026 01:43
@bokelley bokelley merged commit e34c1bf into main May 26, 2026
17 checks passed
@bokelley bokelley deleted the claude/issue-4943-a2-fictional-brand-domain branch May 26, 2026 01:47

@aao-release-bot aao-release-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Migration 495 silently reverts the #294 curriculum fix. The teaching-rule and changeset are fine; the migration is the problem.

The diff replaces the entire A2 {key_concepts} array with a literal copied from migration 274. But migration 294 (server/src/db/migrations/294_fix_creative_agent_curriculum.sql:13) deliberately rewrote the "Agent roles in action" entry to fix "creative-agent-as-separate-entity" framing. After 495 lands, A2 learners get back the pre-294 wording on the wire to Sage:

  • 294: "...The sales agent responds with products and — if it implements the Creative Protocol — also handles creatives. A single sales agent often supports both media buy and creative tasks from one endpoint. A separate creative agent is possible but not required."
  • 495 (server/src/db/migrations/495_a2_fictional_brand_domain_rule.sql:19): "...The sales agent responds with products. The creative agent adapts assets. Multiple agents collaborate on one campaign."

That is exactly the framing 294 was titled to eliminate. The migration title there is literally Fix curriculum content that treats "creative agent" as a distinct entity type. Two months of curriculum drift gone in one jsonb_set.

Fix

Stop replacing the whole array. Use targeted paths so untouched entries stay untouched:

UPDATE certification_modules
SET lesson_plan = jsonb_set(
  lesson_plan,
  '{key_concepts,0,teaching_notes}',
  '"The learner tells Addie what they want: audience, goals, budget. Addie orchestrates the buy against @cptestagent. The learner is not coding — they are specifying intent. This is the fundamental interaction pattern of agentic advertising. When you generate a sample brief to demonstrate, use a fictional brand domain (e.g. nova-brands.example or acme-corp.example) — never a real company domain."'::jsonb
)
WHERE id = 'A2';

Same principle for the demo_scenarios write — A2 currently has a single scenario at {demo_scenarios,0,description}, so target that path rather than replacing the array. (Today it's content-equivalent; tomorrow a sibling migration adds a second scenario and the wholesale rewrite eats it.) The {0,description} write on exercise_definitions is fine — a2_ex1 is the only A2 exercise and 407 only appends success_criteria, doesn't reorder.

Things I checked

  • server/src/db/migrations/495_..._.sql:19 vs 294_..._.sql:13 — exact-string diff, confirmed regression.
  • A2 exercise_definitions[0] is a2_ex1 (seeded by 274, 407 only appends success_criteria, 479 targets A2B). {0,description} is safe.
  • Migration's explicit BEGIN;/COMMIT; nested inside the runner's transaction at server/src/db/migrate.ts:130-142 is a known harmless pattern (~56 other migrations do this).
  • Empty changeset frontmatter is the documented server-only convention (cf. .changeset/4266-addie-copy-response-button.md).
  • TEACHING RULE addition at server/src/addie/mcp/certification-tools.ts:643 fits the surrounding bullet pattern and the .example TLD is RFC 2606-correct.

Minor nits (non-blocking)

  1. PR body references migration 493_ but the committed file is 495_ (renumber in ddb3d31). Worth updating the description so future archaeology lines up.

Re-request once the migration writes only the paths it actually intends to change. Belt-and-suspenders is fine; belt-and-suspenders-and-time-machine is not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

claude-triaged Issue has been triaged by the Claude Code triage routine. Remove to re-triage.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A2 certification: sandbox demo uses scope3.com brand domain, causing misleading brief associations

2 participants