From 30f32a331d1c1342402c115ee4b2e994155e8472 Mon Sep 17 00:00:00 2001 From: samvaity Date: Thu, 16 Apr 2026 11:08:33 -0700 Subject: [PATCH 1/4] Add service-specific skills infrastructure for Azure SDK Adds the infrastructure for service teams to create, validate, and maintain package-specific Copilot skills -- the tribal knowledge that helps AI agents work correctly on their SDK packages. ## What's included ### 1. find-package-skill (discovery registry) Routes agents to package-specific skills before modifying any SDK package. Matches the pattern from JS, Python, and .NET repos. ### 2. create-package-skill (bootstrap wizard) Interactive wizard that walks service teams through creating a skill: - Phase 0: Scan package (detect architecture, customizations, key files) - Phase 1: Scaffold SKILL.md (common pitfalls, error categorization, patterns) - Phase 2: Generate references (architecture.md, customizations.md) - Phase 3: Validate (vally lint) - Phase 4: Register in find-package-skill table ### 3. azure-search-documents (first package skill) Created using the wizard as a dogfood. Covers: - Error categorization table (generated/@Generated vs hand-written) - SearchCustomizations.java fragility patterns - ServiceVersion dedup guards - Breaking change detection - Clear MCP tool handoff (activates after generate-sdk-locally shared skill) ## Key design decisions - Skill name = Maven artifactId (azure-search-documents) - No version numbers in skills -- keep content static - TypeSpec-first customization guardrail - Skills activate after MCP tools have been tried -- no duplication - Based on eval data: 1/3 -> 4/4 correct fix locations with skill ## Related - JS: https://github.com/Azure/azure-sdk-for-js/pull/37484 - Python: https://github.com/Azure/azure-sdk-for-python/pull/45972 - .NET: https://github.com/Azure/azure-sdk-for-net/pull/57580 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/create-package-skill/SKILL.md | 76 +++++++++++++++ .../phases/00-scan-package.md | 63 ++++++++++++ .../phases/01-scaffold-skill.md | 95 +++++++++++++++++++ .../phases/02-generate-references.md | 49 ++++++++++ .../phases/03-validate.md | 46 +++++++++ .../phases/04-register.md | 41 ++++++++ .../references/skill-template.md | 52 ++++++++++ .../references/validation-tools.md | 24 +++++ .github/skills/find-package-skill/SKILL.md | 25 +++++ .../skills/azure-search-documents/SKILL.md | 58 +++++++++++ .../references/architecture.md | 84 ++++++++++++++++ .../references/customizations.md | 78 +++++++++++++++ 12 files changed, 691 insertions(+) create mode 100644 .github/skills/create-package-skill/SKILL.md create mode 100644 .github/skills/create-package-skill/phases/00-scan-package.md create mode 100644 .github/skills/create-package-skill/phases/01-scaffold-skill.md create mode 100644 .github/skills/create-package-skill/phases/02-generate-references.md create mode 100644 .github/skills/create-package-skill/phases/03-validate.md create mode 100644 .github/skills/create-package-skill/phases/04-register.md create mode 100644 .github/skills/create-package-skill/references/skill-template.md create mode 100644 .github/skills/create-package-skill/references/validation-tools.md create mode 100644 .github/skills/find-package-skill/SKILL.md create mode 100644 sdk/search/azure-search-documents/.github/skills/azure-search-documents/SKILL.md create mode 100644 sdk/search/azure-search-documents/.github/skills/azure-search-documents/references/architecture.md create mode 100644 sdk/search/azure-search-documents/.github/skills/azure-search-documents/references/customizations.md diff --git a/.github/skills/create-package-skill/SKILL.md b/.github/skills/create-package-skill/SKILL.md new file mode 100644 index 000000000000..151e5f197b8b --- /dev/null +++ b/.github/skills/create-package-skill/SKILL.md @@ -0,0 +1,76 @@ +--- +name: create-package-skill +description: 'Interactive wizard that walks service teams through creating a package-specific skill for their Azure SDK package. Scans the package, detects customization patterns, scaffolds a SKILL.md with references, validates with vally lint, and registers in find-package-skill. WHEN: create package skill; add service skill; bootstrap skill for package; new package skill; skill for my SDK package; write skill for search; write skill for cosmos.' +--- + +# Create Package Skill Wizard + +> **Minimal beats comprehensive. Human-written beats auto-generated. Scaffold and iterate.** + +> Skills encode tribal knowledge — the "I wish someone had told me" stuff that's hard to learn from just reading code. Focus on what's non-obvious and package-specific. + +## Interaction Protocols + +**CONFIRM Protocol** (asset-producing steps — creating files): +1. PRESENT the proposed assets and explain why. +2. ASK exactly one question: "Create now (recommended), edit first, or skip?" +3. ACT immediately. Create → write files this turn. Edit → refine, re-ask. Skip → move on. + +**DECIDE Protocol** (informational/correction steps — no files created): +1. PRESENT the information or findings. +2. ASK one specific question appropriate to the decision. +3. PROCEED based on the answer. + +One question at a time. Respect "skip" — never re-ask or defer. + +## Wizard Flow + +Run each phase in order. **Progressive loading:** Read only the current phase file. + +| Phase | Description | Instructions | +|---|---|---| +| **Phase 0** | 🧭 Scan Package — detect architecture, customizations, key files | [phases/00-scan-package.md](phases/00-scan-package.md) | +| **Phase 1** | 📝 Scaffold SKILL.md — generate skill with common pitfalls, architecture, workflow | [phases/01-scaffold-skill.md](phases/01-scaffold-skill.md) | +| **Phase 2** | 📚 Generate References — create architecture.md and customizations.md | [phases/02-generate-references.md](phases/02-generate-references.md) | +| **Phase 3** | Validate -- run vally lint | [phases/03-validate.md](phases/03-validate.md) | +| **Phase 4** | 📋 Register — add to find-package-skill table | [phases/04-register.md](phases/04-register.md) | + +## Guardrails + +**Content:** +- Every line must be non-obvious and package-specific. No generic Java/SDK patterns. +- SKILL.md should be under 500 tokens (soft limit). Move details to references/. +- References under 1000 tokens each. Split if larger. +- Never duplicate what's already in `.github/copilot-instructions.md` or shared skills. + +**Relationship to existing SDK tools:** +- Package skills **complement** the Azure SDK MCP tools (`azsdk_package_generate_code`, `azsdk_package_build_code`, etc.) and the `generate-sdk-locally` shared skill — they do NOT replace them. +- MCP tools handle deterministic operations (generate, build, test). Package skills provide the reasoning context an agent needs to use those tools correctly for a specific package. +- Never redefine how generation, building, or testing works — reference the existing tools instead (e.g., "Run `tsp-client update`", not custom generation steps). +- If a workflow step is already handled by an MCP tool or shared skill, just reference it — don't re-document it. + +**Structure:** +- Skill directory: `sdk///.github/skills//` +- Directory name MUST match `name` field in frontmatter (vally lint enforces this). +- Use semicolons to separate trigger phrases in description (YAML-safe). + +**Security:** +- Never embed secrets or credentials in skill content. +- Never instruct agents to bypass CI, linters, or checkstyle rules. +- Never instruct agents to edit generated files directly — always route through customizations. + +## Key Principles (from eval data) + +Our eval showed that skill **structure** matters more than **volume**: + +| Pattern | Impact | +|---|---| +| "Common Pitfalls" section at the TOP | Agent reads pitfalls before analyzing errors → correct diagnosis | +| "Check X FIRST" directives | Changes agent default from "fix the error location" to "check the customization layer" | +| Error categorization tables | Gives agent a decision framework, not just procedures | +| `@Generated` annotation guidance | Agent distinguishes auto-updated vs hand-written methods in generated files | + +## References (load on demand) + +- [references/skill-template.md](references/skill-template.md) -- SKILL.md template with required sections +- [references/validation-tools.md](references/validation-tools.md) -- vally lint, CI workflow setup diff --git a/.github/skills/create-package-skill/phases/00-scan-package.md b/.github/skills/create-package-skill/phases/00-scan-package.md new file mode 100644 index 000000000000..8408c18cddca --- /dev/null +++ b/.github/skills/create-package-skill/phases/00-scan-package.md @@ -0,0 +1,63 @@ +# Phase 0: Scan Package 🧭 + +> 📍 **Phase 0 — Scan Package** | Detect the package's architecture, customization patterns, and key files. + +## Step 1 — Identify the Package + +Ask the user: "Which SDK package should I create a skill for?" + +The user should provide either: +- A package name (e.g., `azure-search-documents`, `azure-cosmos`) +- A path (e.g., `sdk/search/azure-search-documents`) + +Resolve to the package root directory. Verify it exists and contains `pom.xml` or `tsp-location.yaml`. + +## Step 2 — Scan the Package + +Scan the package using the checklist below. Use glob/grep/view tools. + +### Scan Checklist + +1. **Code generation**: Check for `tsp-location.yaml` → TypeSpec-generated package. If present, note the spec directory and commit SHA. + +2. **Customizations**: Check for `customizations/src/main/java/*.java` → Post-generation AST customizations. Read the file and list all customization methods. + +3. **Source layout**: Glob `src/main/java/**/*.java` → count files. Identify packages (documents, indexes, models, implementation, etc.). + +4. **Hand-written vs generated**: Check first few `.java` files for `// Code generated by Microsoft (R) TypeSpec Code Generator.` header. Note which directories are generated vs hand-written. + +5. **`@Generated` annotation**: Check if generated files contain methods WITHOUT `@Generated` (hand-written convenience wrappers mixed into generated files). + +6. **Service version**: Check for `*ServiceVersion.java` → how service versions are managed. + +7. **Key hand-written files**: Look for non-generated utility files in the scanned source. + +8. **Tests**: Check `src/test/java/` structure. Look for `assets.json` (recorded tests), `*TestBase.java`, live test configuration. + +## Step 3 — Present Package Profile + +Print a concise summary: + +📋 **Package Profile** + +| Field | Value | +|---|---| +| Package | `com.azure:` | +| Path | `sdk//` | +| Generated | Yes (TypeSpec) | +| Customizations | N methods in customizations file | +| Source files | N generated, M hand-written | +| Mixed files | Files with both @Generated and non-@Generated methods | +| Service versions | N versions | +| Tests | Unit: Y, Live: Y/N, Recorded: Y/N | +| Key hand-written files | List discovered files | + +## Step 4 — DECIDE + +Question: "Does this profile look right? Anything to add or correct?" + +📍 **Phase 0 complete** | Package scanned | Next: Phase 1 + +--- +## → Next: Phase 1 — Scaffold SKILL.md +Read [phases/01-scaffold-skill.md](phases/01-scaffold-skill.md) and begin immediately. diff --git a/.github/skills/create-package-skill/phases/01-scaffold-skill.md b/.github/skills/create-package-skill/phases/01-scaffold-skill.md new file mode 100644 index 000000000000..372ad4a325a4 --- /dev/null +++ b/.github/skills/create-package-skill/phases/01-scaffold-skill.md @@ -0,0 +1,95 @@ +# Phase 1: Scaffold SKILL.md 📝 + +> 📍 **Phase 1 — Scaffold SKILL.md** | Generate the main skill file with key sections. + +> 📖 Read `references/skill-template.md` for the full template. + +Using the package profile from Phase 0, generate a `SKILL.md` at: +``` +sdk///.github/skills//SKILL.md +``` + +The skill directory name MUST match the Maven package identifier (e.g., `com.azure:azure-search-documents`). This is both the `name` in frontmatter and the directory name (vally lint enforces the match). + +## Content Principles + +- **Keep it static.** Document architecture, design patterns, and convenience layer patterns -- things that rarely change. Do NOT include version numbers, current API versions, or anything that changes every release. The skill should be valid for years, not months. +- **Point to source code, not hardcoded lists.** When referencing things that change (version lists, method names, enum values), point the agent to the authoritative source file instead of enumerating values in the skill. For example: "See `includeOldApiVersions()` in `SearchCustomizations.java` for the version list" -- NOT a hardcoded list of versions. +- **Prefer TypeSpec customizations over Java customizations.** When documenting customization patterns, always note: "Use Java customizations when TypeSpec cannot express the desired behavior, or when the behavior is Java-specific. For TypeSpec-level customizations (preferred when possible), see [TypeSpec Client Customizations Reference](https://github.com/Azure/azure-sdk-for-java/blob/main/eng/common/knowledge/customizing-client-tsp.md)." +- **Don't re-document MCP tools.** The `generate-sdk-locally` shared skill and `azsdk_customized_code_update` MCP tool already handle generation, build, customization application, and testing workflows. The package skill adds only what those tools don't know. +- **Focus on the convenience layer.** The highest-value content is: how is the package designed, what convenience patterns exist, what does the agent need to know to write/maintain the convenience layer correctly. + +## Required Sections + +### 1. Frontmatter + +```yaml +--- +name: +description: '. WHEN: regenerate ; modify ; fix bug; add feature; tsp-client update.' +--- +``` + +Use semicolons for trigger phrases (YAML-safe). Include package name in every trigger. + +### 2. Common Pitfalls (MUST be first section after heading) + +List the most dangerous mistakes: + +- **Never hand-edit generated files** -- files with `// Code generated by Microsoft (R) TypeSpec Code Generator.` are overwritten on `tsp-client update`. All modifications go through the customizations file or TypeSpec decorators. +- If the package has customizations: **"Check `.java` FIRST when generated files have errors"** +- If generated files have mixed `@Generated`/non-`@Generated` methods: **"Methods without `@Generated` are hand-written convenience wrappers that the generator does NOT update. After regeneration, update them to match the new generated signatures."** Note: generated classes CAN be modified directly if `partial-update` is enabled in `tsp-config.yaml` -- check that setting first. +- Any package-specific gotchas found during scanning + +### 3. Architecture + +- Source layout with generated vs hand-written distinction +- Customization mechanism (if any) +- Key packages and their purpose + +### 4. After Regeneration + +**Do NOT re-document the generation/build/test workflow.**The `generate-sdk-locally` shared skill and MCP tools (`azsdk_package_generate_code`, `azsdk_customized_code_update`) handle that. The package skill adds only: + +- **Error categorization table** -- which file to fix based on error type (generated/@Generated, generated/non-@Generated, hand-written) +- **Package-specific customization patterns** -- what each customization method does and when it needs updating +- **Service version management** -- how the version enum works with the customization layer +- **Breaking change detection** -- what to look for after spec changes (our eval found 29 removed types that were not caught without this guidance) + +Note: The `azsdk_customized_code_update` MCP tool handles many customization fixes automatically (TypeSpec decorators first, then code patches). The package skill documents what that tool does NOT know -- package-specific patterns like `hideWithResponseBinaryDataApis` fragility or `includeOldApiVersions` dedup guards. + +### 5. Post-Regeneration Customizations (if customizations exist) + +Add the guardrail: "Use Java customizations when TypeSpec cannot express the desired behavior, or when the behavior is Java-specific. For TypeSpec-level customizations (preferred when possible), see [TypeSpec Client Customizations Reference](https://github.com/Azure/azure-sdk-for-java/blob/main/eng/common/knowledge/customizing-client-tsp.md)." + +For each customization method, document: +- What it does +- When to update it +- What breaks if the generated code changes under it +- Whether it could be replaced by a TypeSpec-level customization + +**Do NOT include "how to add a new customization" steps** (e.g., `mvn clean compile`, `tsp-client update`, write-compile-verify loops). The `azsdk_customized_code_update` MCP tool and `generate-sdk-locally` shared skill already handle the customization workflow. Only document what is unique to this package's customization layer. + +### 6. Testing Notes + +How to run tests, recorded test setup, environment requirements. + +### 7. References table + +Link to `references/architecture.md` and `references/customizations.md`. + +## Step 1 — Present + +Generate the full SKILL.md content and print it. Use the package profile to fill in package-specific details. Mark sections where domain expertise is needed with ``. + +## Step 2 — CONFIRM + +Question: "Create this SKILL.md now (recommended), edit first, or skip?" + +If confirmed, create the skill directory and SKILL.md file. + +📍 **Phase 1 complete** | Created: SKILL.md | Next: Phase 2 + +--- +## → Next: Phase 2 — Generate References +Read [phases/02-generate-references.md](phases/02-generate-references.md) and begin immediately. diff --git a/.github/skills/create-package-skill/phases/02-generate-references.md b/.github/skills/create-package-skill/phases/02-generate-references.md new file mode 100644 index 000000000000..a1bdbc7a0553 --- /dev/null +++ b/.github/skills/create-package-skill/phases/02-generate-references.md @@ -0,0 +1,49 @@ +# Phase 2: Generate References 📚 + +> 📍 **Phase 2 — Generate References** | Create supporting reference documents. + +## references/architecture.md + +Generate from the package scan. Include: + +- **Repository layout** — directory tree with generated/hand-written annotations +- **Source layout** — package structure under `src/main/java/` +- **Code generation** — toolchain (`TypeSpec → emitter → src/`), `tsp-location.yaml` format +- **Generated vs custom** — table showing mechanism, location, when to use +- **Public client types** — all sync/async client classes and their purpose +- **Service version management** — how the version enum works, customization interaction +- **Key supporting files** — table of important files and their purpose +- **Dependencies** — compile and test dependencies +- **Build and test commands** — exact Maven commands + +**Important**: Only include information that is accurate based on scanning the actual code. Mark anything uncertain with ``. + +## references/customizations.md (if customizations exist) + +Generate from reading the actual customization file. For each method: + +- **Problem**: What issue does this customization solve? +- **Solution**: What AST manipulation does it perform? +- **When to update**: What changes in the generated code would break this? +- **Code example**: Show the key JavaParser calls + +Also include: +- **Common customization patterns** — table of JavaParser operations (rename, hide, remove, add) +- **Adding a new customization** — step-by-step +- **Removing a customization** — step-by-step +- **Troubleshooting** — silent failures, syntax errors, stale references +- **Quick-reference checklist** — post-regeneration verification steps + +## Step 1 — Present + +Print the proposed reference files content. + +## Step 2 — CONFIRM + +Question: "Create these reference files now (recommended), edit first, or skip?" + +📍 **Phase 2 complete** | Created: references/ | Next: Phase 3 + +--- +## → Next: Phase 3 — Validate +Read [phases/03-validate.md](phases/03-validate.md) and begin immediately. diff --git a/.github/skills/create-package-skill/phases/03-validate.md b/.github/skills/create-package-skill/phases/03-validate.md new file mode 100644 index 000000000000..a6a1401d385f --- /dev/null +++ b/.github/skills/create-package-skill/phases/03-validate.md @@ -0,0 +1,46 @@ +# Phase 3: Validate + +> 📍 **Phase 3 — Validate** | Run structural validation on the new skill. + +> 📖 Read `references/validation-tools.md` for tool details. + +## Step 1 — Structural Validation (vally lint) + +Run vally lint against the skill: + +```powershell +# If vally is installed (npm package from microsoft/evaluate) +vally lint sdk///.github/skills/ + +# If vally is not installed, check manually: +# - SKILL.md exists with valid YAML frontmatter +# - name field matches directory name +# - All markdown links in SKILL.md resolve to existing files +# - No orphaned files in references/ (every file linked from SKILL.md) +``` + +Expected: 3/3 checks pass (spec-compliance, valid-refs, orphan-files). + +**Common failure**: `name-directory-mismatch` -- the `name` in frontmatter doesn't match the directory name. Fix by renaming the directory to match the Maven artifactId. + +## Step 2 — Token Budget Check + +Verify token budgets: +- SKILL.md: under 500 tokens (soft), under 5000 (hard) +- Each reference file: under 1000 tokens + +If over budget, split content into additional reference files. + +## Step 3 — DECIDE + +Present validation results. If all pass: +Question: "Validation passed. Proceed to register the skill?" + +If failures exist, present them and ask: +Question: "These issues need fixing. Fix now, or skip validation?" + +📍 **Phase 3 complete** | Validation: pass/fail | Next: Phase 4 + +--- +## → Next: Phase 4 — Register +Read [phases/04-register.md](phases/04-register.md) and begin immediately. diff --git a/.github/skills/create-package-skill/phases/04-register.md b/.github/skills/create-package-skill/phases/04-register.md new file mode 100644 index 000000000000..485ac8dce234 --- /dev/null +++ b/.github/skills/create-package-skill/phases/04-register.md @@ -0,0 +1,41 @@ +# Phase 4: Register 📋 + +> 📍 **Phase 4 — Register** | Add the skill to the find-package-skill discovery table. + +## Step 1 — Update find-package-skill + +Add a row to `.github/skills/find-package-skill/SKILL.md` in the **Package Skills** table: + +```markdown +| `` | `sdk///.github/skills//SKILL.md` | +``` + +## Step 2 — Summary + +Print a summary of everything created: + +📋 **Package Skill Created** + +| Item | Path | Status | +|---|---|---| +| SKILL.md | `sdk///.github/skills//SKILL.md` | Created | +| architecture.md | `...references/architecture.md` | Created/Skipped | +| customizations.md | `...references/customizations.md` | Created/Skipped | +| find-package-skill | `.github/skills/find-package-skill/SKILL.md` | Updated | +| vally lint | 3/3 checks | Passed | + +**Next steps for the service team:** +1. Fill in any `` sections with domain-specific knowledge +2. Test the skill by asking an agent to regenerate your package +3. Iterate: agent gets something wrong -> update skill -> test again +4. Submit a PR + +**Maintaining your skill:** +- When your package's customizations change, update the skill +- Keep content static -- no version numbers or release-specific info + +## Step 3 — CONFIRM + +Question: "Register the skill in find-package-skill now (recommended), or skip?" + +📍 **Phase 4 complete** | Skill registered | Wizard done 🎉 diff --git a/.github/skills/create-package-skill/references/skill-template.md b/.github/skills/create-package-skill/references/skill-template.md new file mode 100644 index 000000000000..da9c5fe54c30 --- /dev/null +++ b/.github/skills/create-package-skill/references/skill-template.md @@ -0,0 +1,52 @@ +# SKILL.md Template for Azure SDK Package Skills + +Use this template when creating a new package skill. Replace placeholders with package-specific content. + +```yaml +--- +name: : +description: '. WHEN: regenerate ; modify ; fix bug; add feature; tsp-client update.' +--- +``` + +The `name` field and directory name MUST match the Maven package identifier (e.g., `com.azure:azure-search-documents`). + +## Content Principles + +- **Keep it static** -- no version numbers, no current API versions. Document design and patterns, not release state. +- **Prefer TypeSpec over Java customizations** -- always note when a customization could be a TypeSpec decorator instead. +- **Don't re-document MCP tools** -- the `generate-sdk-locally` skill and `azsdk_customized_code_update` handle generation workflows. +- **Focus on the convenience layer** -- what does the agent need to know to write/maintain convenience patterns correctly. + +## Required Sections (in order) + +### Common Pitfalls +List 3-5 most dangerous mistakes. This section MUST come first -- agents read it before analyzing errors. + +### Architecture +Source layout, generated vs hand-written code, customization mechanism. No version numbers. + +### After Regeneration +Error categorization table, service version management, breaking change detection. Do NOT re-document the generation/build/test steps -- those are in the shared skill. + +### Post-Regeneration Customizations (if customizations exist) +Per-method documentation with "when to update" guidance and whether TypeSpec-level customization is possible instead. + +### Testing Notes +Commands, recorded test setup, environment requirements. + +### References +Table linking to references/*.md files. + +## Structural Rules + +| Rule | Enforced By | +|---|---| +| `name` matches directory name (= Maven package identifier) | `vally lint` | +| All markdown links resolve | `vally lint` | +| No orphaned reference files | `vally lint` | +| Code references still exist in codebase | Manual review | +| SKILL.md under 5000 tokens | `vally lint` | +| No version numbers or release-specific info | Manual review | +| Trigger phrases include package name | Manual review | +| No cross-language content | Manual review | diff --git a/.github/skills/create-package-skill/references/validation-tools.md b/.github/skills/create-package-skill/references/validation-tools.md new file mode 100644 index 000000000000..2243448bc930 --- /dev/null +++ b/.github/skills/create-package-skill/references/validation-tools.md @@ -0,0 +1,24 @@ +# Validation Tools for Package Skills + +## vally lint (Structural Validation) + +**Source**: `microsoft/evaluate` (currently private, will be public) + +```bash +# Install (once published) +npm install -g @microsoft/vally-cli + +# Lint a skill +vally lint sdk///.github/skills/ + +# Lint all skills in repo +vally lint .github/skills +``` + +### What it checks + +| Check | What it does | Common failure | +|---|---|---| +| `spec-compliance` | Validates frontmatter (name, description), name-directory match | Directory name doesn't match `name` field | +| `valid-refs` | All markdown links resolve to existing files | Broken link to reference file | +| `orphan-files` | No unreferenced files in references/ | File in references/ not linked from SKILL.md | diff --git a/.github/skills/find-package-skill/SKILL.md b/.github/skills/find-package-skill/SKILL.md new file mode 100644 index 000000000000..0306b1b7be5b --- /dev/null +++ b/.github/skills/find-package-skill/SKILL.md @@ -0,0 +1,25 @@ +--- +name: find-package-skill +description: '**UTILITY SKILL** — Must be consulted before making any changes to packages under sdk/. Provides package-specific domain knowledge that prevents common mistakes. WHEN: add feature to package; fix bug in package; modify package code; regenerate from typespec; change client library.' +--- + +# Find Package Skill + +Before modifying any SDK package in this monorepo, check this registry to see +if a package-specific skill exists. Package skills contain tribal knowledge +(architecture, data flows, type mappings, pitfalls) that prevents common mistakes. + +Always check this registry before modifying any SDK package — even if you think +you already know the package well. + +## How to Use + +1. Find the package you're modifying in the table below. +2. Read the SKILL.md at the listed path using the Read tool. Then read all files under the `references/` directory next to it for additional context. +3. If the package isn't listed, no package-specific skill exists yet — proceed normally. + +## Package Skills + +| Package | Path | +| -------------------------- | ------------------------------------------------------------------------------------------------- | +| `azure-search-documents` | `sdk/search/azure-search-documents/.github/skills/azure-search-documents/SKILL.md` | diff --git a/sdk/search/azure-search-documents/.github/skills/azure-search-documents/SKILL.md b/sdk/search/azure-search-documents/.github/skills/azure-search-documents/SKILL.md new file mode 100644 index 000000000000..d31f910eba7c --- /dev/null +++ b/sdk/search/azure-search-documents/.github/skills/azure-search-documents/SKILL.md @@ -0,0 +1,58 @@ +--- +name: azure-search-documents +description: 'Post-regeneration customization guide for azure-search-documents SDK. Covers error diagnosis, customization patterns, and search-specific fixes after tsp-client update. WHEN: regenerate azure-search-documents; fix azure-search-documents build errors; azure-search-documents customization errors; azure-search-documents API version update; update azure-search-documents service version.' +--- + +# azure-search-documents -- Package Skill + +> The `generate-sdk-locally` shared skill handles generation, build, test, and customization workflows -- including running `azsdk_customized_code_update` when builds fail. **This skill activates after those tools have been tried.** It covers what they can't solve: search-specific error diagnosis, customization fragility patterns, and convenience layer design. + +## Common Pitfalls + +- **Never hand-edit generated files.** Files with `// Code generated by Microsoft (R) TypeSpec Code Generator.` are overwritten on every `tsp-client update`. All modifications go through `SearchCustomizations.java`. +- **Check `SearchCustomizations.java` FIRST when generated files have errors.** The customizations run during generation and can produce broken output if the generated code structure changed. The most common culprits: + - `hideWithResponseBinaryDataApis()` -- rewires method bodies across packages, can create cross-package import mismatches + - `includeOldApiVersions()` -- can duplicate enum constants the generator now produces + - `addSearchAudienceScopeHandling()` -- may fail if builder structure changed +- **Methods without `@Generated` in generated files are hand-written.** These are convenience wrappers that the generator preserves but does NOT update. After regeneration, you must manually update them to match any changed generated signatures. Look at how the `@Generated` methods in the same file were updated as a pattern to follow. +- **`includeOldApiVersions` can create duplicate enum constants.** If the generator starts producing a version that was previously only in the customization list, you get a compilation error. Remove the version from the customization list. + +## After Regeneration -- Diagnose and Fix Build Errors + +Categorize each compilation error: + +| Error location | What it means | Where to fix | +|---|---|---| +| Generated file, `@Generated` method | Customization produced broken output | **Fix `SearchCustomizations.java`** -- update AST queries to match new generated code | +| Generated file, method WITHOUT `@Generated` | Hand-written wrapper references changed generated types | **Fix the hand-written method** -- match the pattern used by `@Generated` methods in the same file | +| Hand-written file (`SearchUtils.java`, `FieldBuilder.java`, batching, tests) | References removed/renamed generated types | **Fix the hand-written file** | + +## Verify Service Version + +`SearchServiceVersion.java` is generated but customized by `SearchCustomizations.java`. + +1. Check `getLatest()` returns the new version +2. In `includeOldApiVersions()`: + - **Remove** any version the generator now produces (prevents duplicates) + - **Add** the previous latest version if not already produced by the generator +3. Verify all expected old versions are present + +## Detect Breaking Changes + +```bash +# Removed/renamed public types +git diff --name-status HEAD -- sdk/search/azure-search-documents/src/main/java/ | grep "^D" +``` + +Watch for: removed types, renamed constants (need `@Deprecated` aliases), changed property types, removed method overloads. + +## Test Recording Notes + +If test recordings are stale, update `assets.json` and re-record. Remove tests for features that no longer exist in the API. + +## References + +| File | Contents | +|---|---| +| [references/architecture.md](references/architecture.md) | Source layout, generated vs custom split, key files, packages | +| [references/customizations.md](references/customizations.md) | Per-method customization guide, JavaParser patterns, update triggers | diff --git a/sdk/search/azure-search-documents/.github/skills/azure-search-documents/references/architecture.md b/sdk/search/azure-search-documents/.github/skills/azure-search-documents/references/architecture.md new file mode 100644 index 000000000000..4d18c7e1fec8 --- /dev/null +++ b/sdk/search/azure-search-documents/.github/skills/azure-search-documents/references/architecture.md @@ -0,0 +1,84 @@ +# azure-search-documents SDK Architecture + +## Overview + +- **Maven**: `com.azure:azure-search-documents` +- **Java target**: Java 8+ +- **Generation**: TypeSpec via `@azure-tools/typespec-java` emitter + +## Source Layout + +``` +sdk/search/azure-search-documents/ + tsp-location.yaml # TypeSpec spec pin (commit SHA) + pom.xml # Maven project + customizations/src/main/java/ + SearchCustomizations.java # All post-generation AST modifications + src/main/java/com/azure/search/documents/ + SearchClient.java # Sync client (GENERATED + hand-written methods) + SearchAsyncClient.java # Async client (GENERATED + hand-written methods) + SearchClientBuilder.java # Builder (GENERATED + customized) + SearchServiceVersion.java # Version enum (GENERATED + customized) + SearchIndexingBufferedSender.java # Batch upload (HAND-WRITTEN) + SearchIndexingBufferedAsyncSender.java # Async batch upload (HAND-WRITTEN) + implementation/ + SearchUtils.java # Request conversion helpers (HAND-WRITTEN) + FieldBuilder.java # Model reflection to SearchField (HAND-WRITTEN) + batching/ # Batch indexing internals (HAND-WRITTEN) + models/ # Wire-only models (GENERATED) + models/ # Public models (GENERATED) + SearchPagedFlux.java # Async paging (HAND-WRITTEN) + SearchPagedIterable.java # Sync paging (HAND-WRITTEN) + SearchPagedResponse.java # Page response (HAND-WRITTEN) + indexes/ # Index management clients (GENERATED) + models/ # Index/indexer/skillset models (GENERATED, ~150 files) + knowledgebases/ # Knowledge base clients (GENERATED) + models/ # KB models (GENERATED) + options/ # Buffered sender callbacks (HAND-WRITTEN) +``` + +## Generated vs Hand-Written + +All generated files have header: `// Code generated by Microsoft (R) TypeSpec Code Generator.` + +Generated methods have `@Generated` annotation. Some generated files contain BOTH: +- `@Generated` methods -- auto-updated by generator +- Methods WITHOUT `@Generated` -- hand-written wrappers the generator preserves but does NOT update + +**Key mixed files** (generated + hand-written methods): +- `SearchClient.java` -- `getEndpoint()`, `getIndexName()`, `getServiceVersion()` are hand-written +- `SearchAsyncClient.java` -- same accessors are hand-written +- Both contain `hiddenGenerated*` methods created by `hideWithResponseBinaryDataApis` customization + +## Service Version + +`SearchServiceVersion.java` is generated but customized. The generator produces only the latest version constant. Older versions are added by `includeOldApiVersions()` in `SearchCustomizations.java`. + +When a new API version is released, the generator produces a new constant and updates `getLatest()`. The previous latest must be added to the customization list -- but only if the generator does NOT also produce it (otherwise you get duplicate enum constants). + +## Key Hand-Written Files + +| File | Purpose | +|---|---| +| `SearchUtils.java` | Converts SearchOptions to SearchRequest, handles headers | +| `FieldBuilder.java` | Reflects on model types to build SearchField definitions | +| `SearchPagedResponse.java` | ContinuablePage for paged search results | +| `SearchPagedFlux.java` | Async reactive paging (ContinuablePagedFluxCore) | +| `SearchPagedIterable.java` | Sync paging (ContinuablePagedIterable) | +| `SearchIndexingBufferedSender.java` | Batch document upload with auto-flush, retry, splitting | +| `SearchIndexingPublisher.java` | Internal: batch publishing with configurable flush interval | + +## Dependencies + +| Dependency | Scope | +|---|---| +| `azure-core` | compile | +| `azure-json` | compile | +| `azure-core-http-netty` | compile | +| `azure-core-test` | test | +| `azure-identity` | test | + +## Tests + +Test files in `src/test/java/`. Base class: `SearchTestBase` extends `TestProxyTestBase`. +Test recordings via `assets.json` pointing to `Azure/azure-sdk-assets`. diff --git a/sdk/search/azure-search-documents/.github/skills/azure-search-documents/references/customizations.md b/sdk/search/azure-search-documents/.github/skills/azure-search-documents/references/customizations.md new file mode 100644 index 000000000000..53e8e21c0d06 --- /dev/null +++ b/sdk/search/azure-search-documents/.github/skills/azure-search-documents/references/customizations.md @@ -0,0 +1,78 @@ +# azure-search-documents -- Customization Guide + +All post-generation modifications live in `customizations/src/main/java/SearchCustomizations.java`. This file extends `Customization` and uses JavaParser AST manipulation at code-generation time. + +**Use Java customizations when TypeSpec cannot express the desired behavior, or when the behavior is Java-specific.** For TypeSpec-level customizations (preferred when possible), see [TypeSpec Client Customizations Reference](https://github.com/Azure/azure-sdk-for-java/blob/main/eng/common/knowledge/customizing-client-tsp.md). + +Note: The `azsdk_customized_code_update` MCP tool can apply many customization fixes automatically (TypeSpec decorators first, then code patches). The documentation below covers what that tool does NOT handle -- search-specific patterns that require manual understanding. + +## Customization Methods + +### 1. hideGeneratedSearchApis + +**What**: Strips public modifiers from `searchWithResponse`, `autocompleteWithResponse`, `suggestWithResponse` on SearchClient/SearchAsyncClient (makes them package-private). + +**Why**: The generator must expose these to infer SearchOptions, but they shouldn't be public API. + +**When to update**: If the generator changes how SearchOptions is inferred or renames these operations. + +### 2. addSearchAudienceScopeHandling + +**What**: Adds a mutable `private String[] scopes = DEFAULT_SCOPES` field to all builder classes, replaces `DEFAULT_SCOPES` with `scopes` in `createHttpPipeline()` method body. + +**Applied to**: SearchClientBuilder, SearchIndexClientBuilder, SearchIndexerClientBuilder, KnowledgeBaseRetrievalClientBuilder. + +**When to update**: When [typespec#9458](https://github.com/microsoft/typespec/issues/9458) is resolved. Also if `DEFAULT_SCOPES` field or `createHttpPipeline` method is renamed/restructured. + +### 3. includeOldApiVersions + +**What**: Prepends older version constants to the SearchServiceVersion enum. + +**Current list**: See `includeOldApiVersions()` in `SearchCustomizations.java` for the authoritative version list. When a new API version is released, add the previous latest to this list -- but only if the generator does not already produce it. + +**When to update**: When a new API version is released -- add the previous latest to this list. **Critical**: check if the generator now produces a version that's in this list. If so, remove it to avoid duplicate enum constants. + +### 4. removeGetApis + +**What**: Removes all methods with prefixes `searchGet`, `suggestGet`, `autocompleteGet` from SearchClient/SearchAsyncClient. + +**Why**: The spec defines GET and POST variants; Java SDK only exposes POST. + +**When to update**: If the spec changes operation names or adds new GET variants. + +### 5. hideWithResponseBinaryDataApis (MOST FRAGILE) + +**What**: For each public `@Generated` method using BinaryData in return type or parameters: +1. Renames to `hiddenGenerated` and strips modifiers (package-private) +2. Updates convenience methods to call the renamed version + +**Applied to**: All 8 client classes (sync + async for search, index, indexer, knowledge base). + +**Why it's fragile**: It does string replacement across method bodies, which can create cross-package import mismatches when operations are added/removed (type numbering changes). + +**When to update**: When the Java TypeSpec generator natively supports typed `WithResponse` APIs, or when operations are added/removed in the spec. + +## Post-Regeneration Checklist + +``` +[ ] Check if any customization methods reference names that no longer exist + (silently does nothing -- no error, no warning) +[ ] If new API version: + [ ] Check includeOldApiVersions() list for duplicates with generator + [ ] Verify getLatest() returns the new version +[ ] If new client classes generated: + [ ] Add hideWithResponseBinaryDataApis() call + [ ] Add addSearchAudienceScopeHandling() call for builder +[ ] If new public methods should be hidden: + [ ] Add AST manipulation in SearchCustomizations.java +``` + +> For adding new customizations or applying fixes, use the `azsdk_customized_code_update` MCP tool from the `generate-sdk-locally` shared skill. It handles the classify-apply-regenerate-build cycle automatically. + +## Troubleshooting + +**Build fails after customization**: The AST manipulation produced invalid Java. Check `StaticJavaParser.parseBlock()` calls for syntax errors. + +**Customization silently does nothing**: A method/class name changed upstream. The `getMethodsByName()` call returns empty. Verify string literals match current generated code. + +**Cross-package import errors after hideWithResponseBinaryDataApis**: The method body rewiring created imports referencing types in the wrong package. Check if operations were added/removed in the spec. From a75f48a814cc203c319939eb5e9ebaf89b89467c Mon Sep 17 00:00:00 2001 From: Sameeksha Vaity Date: Fri, 17 Apr 2026 10:50:50 -0700 Subject: [PATCH 2/4] Update .github/skills/create-package-skill/phases/00-scan-package.md Co-authored-by: Alan Zimmer <48699787+alzimmermsft@users.noreply.github.com> --- .github/skills/create-package-skill/phases/00-scan-package.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/skills/create-package-skill/phases/00-scan-package.md b/.github/skills/create-package-skill/phases/00-scan-package.md index 8408c18cddca..648045fc69d7 100644 --- a/.github/skills/create-package-skill/phases/00-scan-package.md +++ b/.github/skills/create-package-skill/phases/00-scan-package.md @@ -8,7 +8,7 @@ Ask the user: "Which SDK package should I create a skill for?" The user should provide either: - A package name (e.g., `azure-search-documents`, `azure-cosmos`) -- A path (e.g., `sdk/search/azure-search-documents`) +- A path (e.g., `sdk/search/azure-search-documents`, `sdk/cosmos/azure-cosmos`) Resolve to the package root directory. Verify it exists and contains `pom.xml` or `tsp-location.yaml`. From 6066c95e97c7b1cae9705d3173facfcaa0937f4c Mon Sep 17 00:00:00 2001 From: samvaity Date: Fri, 17 Apr 2026 12:43:21 -0700 Subject: [PATCH 3/4] add cspell exclusion for vally --- .../skills/create-package-skill/phases/01-scaffold-skill.md | 6 +++--- .vscode/cspell.json | 1 + 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/.github/skills/create-package-skill/phases/01-scaffold-skill.md b/.github/skills/create-package-skill/phases/01-scaffold-skill.md index 372ad4a325a4..587a775058ca 100644 --- a/.github/skills/create-package-skill/phases/01-scaffold-skill.md +++ b/.github/skills/create-package-skill/phases/01-scaffold-skill.md @@ -14,7 +14,7 @@ The skill directory name MUST match the Maven package identifier (e.g., `com.azu ## Content Principles - **Keep it static.** Document architecture, design patterns, and convenience layer patterns -- things that rarely change. Do NOT include version numbers, current API versions, or anything that changes every release. The skill should be valid for years, not months. -- **Point to source code, not hardcoded lists.** When referencing things that change (version lists, method names, enum values), point the agent to the authoritative source file instead of enumerating values in the skill. For example: "See `includeOldApiVersions()` in `SearchCustomizations.java` for the version list" -- NOT a hardcoded list of versions. +- **Point to source code, not hardcoded lists.** When referencing things that change (version lists, method names, enum values), point the agent to the authoritative source file instead of enumerating values in the skill. - **Prefer TypeSpec customizations over Java customizations.** When documenting customization patterns, always note: "Use Java customizations when TypeSpec cannot express the desired behavior, or when the behavior is Java-specific. For TypeSpec-level customizations (preferred when possible), see [TypeSpec Client Customizations Reference](https://github.com/Azure/azure-sdk-for-java/blob/main/eng/common/knowledge/customizing-client-tsp.md)." - **Don't re-document MCP tools.** The `generate-sdk-locally` shared skill and `azsdk_customized_code_update` MCP tool already handle generation, build, customization application, and testing workflows. The package skill adds only what those tools don't know. - **Focus on the convenience layer.** The highest-value content is: how is the package designed, what convenience patterns exist, what does the agent need to know to write/maintain the convenience layer correctly. @@ -54,9 +54,9 @@ List the most dangerous mistakes: - **Error categorization table** -- which file to fix based on error type (generated/@Generated, generated/non-@Generated, hand-written) - **Package-specific customization patterns** -- what each customization method does and when it needs updating - **Service version management** -- how the version enum works with the customization layer -- **Breaking change detection** -- what to look for after spec changes (our eval found 29 removed types that were not caught without this guidance) +- **Breaking change detection** -- what to look for after spec changes -Note: The `azsdk_customized_code_update` MCP tool handles many customization fixes automatically (TypeSpec decorators first, then code patches). The package skill documents what that tool does NOT know -- package-specific patterns like `hideWithResponseBinaryDataApis` fragility or `includeOldApiVersions` dedup guards. +Note: The `azsdk_customized_code_update` MCP tool handles many customization fixes automatically (TypeSpec decorators first, then code patches). The package skill documents what that tool does NOT know -- package-specific patterns. ### 5. Post-Regeneration Customizations (if customizations exist) diff --git a/.vscode/cspell.json b/.vscode/cspell.json index 270f9fae5fd3..8556080bd667 100644 --- a/.vscode/cspell.json +++ b/.vscode/cspell.json @@ -427,6 +427,7 @@ "undelete", "unmanaged", "unmutated", + "vally", "vectorizer", "versionid", "vertx", From 7e167b12e75e9cf8d2545e846129689d8149a194 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 17 Apr 2026 20:36:05 +0000 Subject: [PATCH 4/4] Fix broken relative links in create-package-skill phase files Agent-Logs-Url: https://github.com/Azure/azure-sdk-for-java/sessions/ad9469e5-6a57-4492-b387-17d1544a15b2 Co-authored-by: samvaity <16845631+samvaity@users.noreply.github.com> --- .github/skills/create-package-skill/phases/00-scan-package.md | 2 +- .github/skills/create-package-skill/phases/01-scaffold-skill.md | 2 +- .../create-package-skill/phases/02-generate-references.md | 2 +- .github/skills/create-package-skill/phases/03-validate.md | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/skills/create-package-skill/phases/00-scan-package.md b/.github/skills/create-package-skill/phases/00-scan-package.md index 648045fc69d7..4296a7a2ded7 100644 --- a/.github/skills/create-package-skill/phases/00-scan-package.md +++ b/.github/skills/create-package-skill/phases/00-scan-package.md @@ -60,4 +60,4 @@ Question: "Does this profile look right? Anything to add or correct?" --- ## → Next: Phase 1 — Scaffold SKILL.md -Read [phases/01-scaffold-skill.md](phases/01-scaffold-skill.md) and begin immediately. +Read [01-scaffold-skill.md](01-scaffold-skill.md) and begin immediately. diff --git a/.github/skills/create-package-skill/phases/01-scaffold-skill.md b/.github/skills/create-package-skill/phases/01-scaffold-skill.md index 587a775058ca..be2eea79fd7c 100644 --- a/.github/skills/create-package-skill/phases/01-scaffold-skill.md +++ b/.github/skills/create-package-skill/phases/01-scaffold-skill.md @@ -92,4 +92,4 @@ If confirmed, create the skill directory and SKILL.md file. --- ## → Next: Phase 2 — Generate References -Read [phases/02-generate-references.md](phases/02-generate-references.md) and begin immediately. +Read [02-generate-references.md](02-generate-references.md) and begin immediately. diff --git a/.github/skills/create-package-skill/phases/02-generate-references.md b/.github/skills/create-package-skill/phases/02-generate-references.md index a1bdbc7a0553..3c2c8f4fe05d 100644 --- a/.github/skills/create-package-skill/phases/02-generate-references.md +++ b/.github/skills/create-package-skill/phases/02-generate-references.md @@ -46,4 +46,4 @@ Question: "Create these reference files now (recommended), edit first, or skip?" --- ## → Next: Phase 3 — Validate -Read [phases/03-validate.md](phases/03-validate.md) and begin immediately. +Read [03-validate.md](03-validate.md) and begin immediately. diff --git a/.github/skills/create-package-skill/phases/03-validate.md b/.github/skills/create-package-skill/phases/03-validate.md index a6a1401d385f..82367fade642 100644 --- a/.github/skills/create-package-skill/phases/03-validate.md +++ b/.github/skills/create-package-skill/phases/03-validate.md @@ -43,4 +43,4 @@ Question: "These issues need fixing. Fix now, or skip validation?" --- ## → Next: Phase 4 — Register -Read [phases/04-register.md](phases/04-register.md) and begin immediately. +Read [04-register.md](04-register.md) and begin immediately.