Skip to content

refactor(workflow): re-home API-workflow validate under comfy workflow validate (BE-3000)#522

Open
mattmillerai wants to merge 2 commits into
mainfrom
matt/be-3000-workflow-validate
Open

refactor(workflow): re-home API-workflow validate under comfy workflow validate (BE-3000)#522
mattmillerai wants to merge 2 commits into
mainfrom
matt/be-3000-workflow-validate

Conversation

@mattmillerai

Copy link
Copy Markdown
Collaborator

ELI-5

comfy had three different validate commands that all read like duplicates but do totally different things: one checks an API-format workflow, one checks a custom node before you publish it, and one checks a workflow-fragment file. This PR gives the workflow one a proper home — comfy workflow validate — next to the other comfy workflow … commands, and rewrites the one-line help of the other two so it's obvious at a glance which noun each validates. The old comfy validate still works exactly as before; it just prints a gentle "use comfy workflow validate instead" note and is hidden from the help listing.

What changed

  • New canonical command: comfy workflow validate --workflow <f> — the API-format workflow validator now lives under the workflow group. The implementation was factored into a shared workflow.validate_api_workflow(...) so there is no logic duplication.
  • comfy validate → hidden deprecated alias. Still fully functional; it warns ([DEPRECATED — use 'comfy workflow validate'] in help, plus a runtime warning) and delegates to the shared implementation. The runtime warning is routed to stderr in JSON modes, so structured stdout envelopes stay clean.
  • Sharper help one-liners so the three validators read as distinct nouns:
    • comfy node validateValidate this custom node for registry publishing
    • comfy workflow fragment validateValidate that a workflow-fragment file is well-formed
    • comfy workflow validate → keeps the existing API-format description
  • The three commands are not merged — they operate on different objects.
  • Updated the discovery command→schema map (comfy workflow validate added; comfy validate kept for the alias) and the two bundled SKILL.md doc references to the canonical command.

Behavior parity

The envelope command label differs per entry point (workflow validate vs the alias's validate), but the validation payload (valid, error_count, warning_count, errors, warnings) and exit code are identical — asserted by a dedicated test.

Tests

New tests/comfy_cli/command/test_workflow_validate.py:

  • comfy workflow validate passes on a valid workflow (exit 0, envelope command == "workflow validate").
  • Invalid workflow exits non-zero with ok:false.
  • comfy validate still works and warns, labelling itself validate.
  • The alias's payload matches the canonical command's for identical inputs.

Full suite green: 2577 passed, 37 skipped. ruff format + ruff check clean.

Notes / judgment calls

  • The ticket referenced a "deprecation-alias helper from the model/models child." That sibling helper is not on main yet, so the deprecation is implemented self-contained (hidden command + renderer.warn + delegation) rather than blocking on it. Trivial to swap onto a shared helper later if one lands.
  • Step 5 asked to "update the help-json snapshot." comfy_cli/schemas/help.json is a JSON Schema validator (additionalProperties: true), not a captured snapshot — nothing to regenerate. The equivalent doc updates (discovery map + SKILL.md) were made instead.
  • Not a capability denial: the deprecation redirects to comfy workflow validate, which was verified to work end-to-end (tests + live --help/invocation), and the old command remains functional — no dead-end path is introduced.

…ow validate` (BE-3000)

Register the API-format workflow validator as the canonical
`comfy workflow validate`, factoring the shared logic into
`workflow.validate_api_workflow`. Keep top-level `comfy validate` as a
hidden, still-functional deprecated alias that warns and delegates.

Sharpen the sibling validators' help so the three read as distinct nouns:
node validate -> registry publishing; fragment validate -> workflow-fragment
file. Update the discovery schema map and bundled SKILL.md docs to the
canonical command.
@dosubot dosubot Bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Jul 14, 2026
@mattmillerai mattmillerai added cursor-review Request Cursor bot review agent-coded PR authored by the agent-work loop labels Jul 14, 2026
@dosubot dosubot Bot added the enhancement New feature or request label Jul 14, 2026
@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 11 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 40630ad5-4c3c-4ac2-b521-96cf1acf2d65

📥 Commits

Reviewing files that changed from the base of the PR and between a732f5c and e5c41e8.

📒 Files selected for processing (7)
  • comfy_cli/cmdline.py
  • comfy_cli/command/custom_nodes/command.py
  • comfy_cli/command/workflow.py
  • comfy_cli/command/workflow_fragments.py
  • comfy_cli/discovery.py
  • comfy_cli/skills/comfy/SKILL.md
  • tests/comfy_cli/command/test_workflow_validate.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch matt/be-3000-workflow-validate
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch matt/be-3000-workflow-validate

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Cursor Review — Consolidated panel

Triggered by @mattmillerai.

Found 3 finding(s).

Severity Count
🟡 Medium 2
🟢 Low 1

Panel: 6/8 reviewers contributed findings.

Reviewers that did not contribute: kimi-k2.5:adversarial (empty), kimi-k2.5:edge-case (empty)

Comment thread comfy_cli/command/workflow.py
Comment thread comfy_cli/command/workflow.py
Comment thread comfy_cli/command/workflow.py Outdated
…-3000)

Address Cursor review panel findings on `comfy workflow validate`:
- Catch OSError/UnicodeDecodeError when reading the workflow file so an
  unreadable/non-UTF-8 file emits a structured `workflow_read_error`
  envelope instead of crashing with a raw traceback.
- Validate `--where` up front; a typo now emits a `where_invalid`
  envelope rather than letting a bare ValueError escape from Graph.load.
- Escape Rich markup on workflow-supplied node ids / messages in pretty
  mode so a crafted file can't spoof or hide validation output.

Also fix the deprecated-alias test to read CliRunner's stderr: the
deprecation warning is routed to stderr by design, and newer click (>=8.2)
captures stderr separately from stdout instead of mixing it in, which broke
the assertion under CI's fresh dependency install.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@mattmillerai

Copy link
Copy Markdown
Collaborator Author

Automated reviews-resolver note: the red Windows Specific Commands check is a pre-existing repo-wide CI infra failure, not caused by this PR. It fails in Install Dependencies with error: failed to remove file ...\_pydantic_core.cp312-win_amd64.pyd: Access is denied (os error 5) — a Windows DLL file-lock during uv's install/reinstall. The same failure reproduces on main-targeted PRs across every open branch today (be-2999, be-2994, be-2982, be-3001, be-2975, …); last green Windows run was 2026-03-11. Re-ran twice, identical result. Needs a dedicated CI-infra fix to .github/workflows/test-windows.yml (out of scope for this refactor). Otherwise this PR is clean: no merge conflict, no unresolved review threads, local tests 7/7 green, ruff clean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent-coded PR authored by the agent-work loop cursor-review Request Cursor bot review enhancement New feature or request size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant