generate integrations reference from catalog

[DyanGalih]

What changed

• Added a small generator for the integrations reference table, backed by INTEGRATION_REGISTRY plus per-key URL and notes maps in catalog_docs.py.
• Updated docs/reference/integrations.md so the supported agent table is generated from the integration registry (not hand-maintained).
• Added a regression test that checks the committed doc stays in sync with the rendered table output.

Why

The integrations reference had been hand-maintained, which made it easy for the docs to drift from the runtime registry. This change makes the doc a checked artifact and reduces maintenance overhead.

User impact

• Maintainers can update the integration registry and regenerate the docs with specify integration search --markdown.
• Readers get a more reliable integrations reference with less chance of stale entries.

Validation

• specify integration search --markdown
• pytest tests/test_catalog_docs.py -q

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds catalog-backed generation and validation for the integrations reference documentation, along with a CLI script and tests to keep the generated markdown in sync.

Changes:

• Introduce specify_cli.catalog_docs helpers to render/update the generated integrations table from integrations/catalog.json.
• Add scripts/generate_integrations_reference.py with --check/--write modes for CI and local updates.
• Regenerate docs/reference/integrations.md with generated-table markers and add tests to enforce consistency.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
tests/test_catalog_docs.py	Adds tests asserting the committed docs match the generator and that registry metadata is reflected.
src/specify_cli/catalog_docs.py	Implements catalog loading, table rendering, and marker-based replacement for the docs page.
scripts/generate_integrations_reference.py	Provides a CLI entrypoint to check or rewrite the generated integrations reference file.
docs/reference/integrations.md	Converts the integrations table into a generated block and updates surrounding instructions.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

[mnriem]

Love it, but can we integrate it into specify integration search --markdown? Specifically can we keep it simpler than this and just render out the table. On the project end we will take care of integrating it into the docs. I do not want to burden the CLI with that part of the process

[DyanGalih]

Thanks for the direction! I've refactored based on your feedback:

What changed:

• Removed scripts/generate_integrations_reference.py (standalone script gone)
• Stripped all doc-injection machinery from catalog_docs.py — it now only contains the table-rendering helpers (list_integrations_for_docs, render_integrations_table)
• Wired render_integrations_table() into the existing specify integration search --markdown flag — it now outputs the rich Agent/Key/Notes table (with proper column alignment and pipe/CR escaping) instead of the old simple Name/ID/Version/Description/Author table
• Removed the HTML marker comments from docs/reference/integrations.md and updated the note to reference specify integration search --markdown
• Simplified tests/test_catalog_docs.py to just verify table rendering and registry metadata — no subprocess or doc-path tests

Usage:

specify integration search --markdown

Prints the full integrations reference table to stdout. The docs team can paste it wherever needed.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

[Copilot]

Copilot's findings

• Files reviewed: 4/4 changed files
• Comments generated: 3

[mnriem]

Please address Copilot feedback. Please revert the change to integrations.md as we will setup a separate GitHub actions job for that. Thanks for the great work

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

[DyanGalih]

Done! Both tasks completed:\n\n- Copilot feedback addressed — all review threads from the latest rounds have been fixed and replied to (removed dead INTEGRATIONS_REFERENCE_PATH constant, dropped URL-length padding from table renderer, updated integration_search docstring to reflect dual behavior, removed unreachable FileNotFoundError from the except clause, fixed naming from "catalog-backed" to registry-backed, updated PR description validation steps).\n- docs/reference/integrations.md reverted — restored to the upstream/main state and the in-process sync test was removed from the pytest suite (the GH Actions job will handle that check separately)."

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

[Copilot]

Copilot's findings

• Files reviewed: 5/5 changed files
• Comments generated: 3

[mnriem]

Please address Copilot feedback

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

[DyanGalih]

Addressed the remaining actionable Copilot feedback in the latest commit:

• Centralized the shared repo root via _repo_root() in the docs helpers
• Escaped markdown link text so [ / ] don’t break table labels
• Trimmed whitespace-only repository values so they don’t become invalid links
• Narrowed the markdown rendering error handling to expected failures
• Updated integration search --markdown help/docstring text to match behavior
• Added skip guards and safer parsing in the docs tests for minimal/package layouts
• Kept the community docs table rendering and tests aligned with the current branch scope

The remaining older comments are stale/outdated relative to these changes.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.