feat: Validate no custom columns in DD configs

[mikeknep]

The upstream Data Designer library allows users define custom column types via a decorated Python function. These don't work in the platform plugin because we don't have the user-created implementation server-side, nor can we serialize the function and send it over the wire. Today if a user builds a config client-side with a custom column and submits that config to the service, they get a raw error message when Pydantic tries and fails to deserialize the config. This PR ensures we provide a clearer error message that custom columns are not supported here.

Summary by CodeRabbit

• Bug Fixes

  • Added validation to reject unsupported custom columns during configuration, preview, creation, and remote execution.
  • Validation reports now clearly indicate when custom columns are not supported, without interrupting other validation checks.

• Tests

  • Added coverage for custom-column rejection in preview, creation, and SDK validation workflows.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f76dfd4d-c3de-44ba-8030-554c2655b72d

📥 Commits

Reviewing files that changed from the base of the PR and between 1063d06 and eb71946.

📒 Files selected for processing (6)

• packages/data_designer_nemo/src/data_designer_nemo/columns.py
• packages/data_designer_nemo/src/data_designer_nemo/context.py
• plugins/nemo-data-designer/src/nemo_data_designer_plugin/functions/_types.py
• plugins/nemo-data-designer/src/nemo_data_designer_plugin/jobs/spec.py
• plugins/nemo-data-designer/tests/integration/test_remote_validation_errors.py
• plugins/nemo-data-designer/tests/integration/test_validate_sdk.py

---

📝 Walkthrough

Walkthrough

Custom-column validation is added to raw request, parsed configuration, preview, job, local-context, remote-context, and SDK validation paths. Integration tests cover preview, create, and remote SDK validation errors.

Changes

Custom Column Validation

Layer / File(s)	Summary
Custom column detection and parsed-config validation packages/data_designer_nemo/src/data_designer_nemo/columns.py	Adds defensive raw-payload detection and parsed-config validation for columns using the custom discriminator.
Validation entrypoint integration plugins/nemo-data-designer/src/nemo_data_designer_plugin/functions/_types.py, plugins/nemo-data-designer/src/nemo_data_designer_plugin/jobs/spec.py, packages/data_designer_nemo/src/data_designer_nemo/context.py	Runs custom-column checks during preview, job configuration, and local or remote context validation while collecting context errors.
Custom-column integration coverage plugins/nemo-data-designer/tests/integration/test_remote_validation_errors.py, plugins/nemo-data-designer/tests/integration/test_validate_sdk.py	Verifies unsupported custom columns are reported by preview, create, and remote SDK validation.

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant PreviewSpec
  participant DataDesignerJobConfig
  participant RemoteDataDesignerContext
  participant ValidationHelpers
  Client->>PreviewSpec: Validate preview configuration
  PreviewSpec->>ValidationHelpers: Check raw columns
  Client->>DataDesignerJobConfig: Validate job configuration
  DataDesignerJobConfig->>ValidationHelpers: Check raw columns
  Client->>RemoteDataDesignerContext: Validate parsed configuration
  RemoteDataDesignerContext->>ValidationHelpers: Check parsed columns
  ValidationHelpers-->>Client: Return custom-column validation error

Loading

Possibly related PRs

• NVIDIA-NeMo/nemo-platform#413: Modifies the same local and remote context validation paths.

Suggested labels: feat

Suggested reviewers: matthewgrossman, crookedstorm, mckornfield

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main change: rejecting custom columns in Data Designer configs.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch validate-custom-cols/mknepper

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Suite	Lines Covered	Line Rate	Branch Rate
Unit Tests	28236/36079	78.3%	62.7%
Integration Tests	16792/34797	48.3%	20.9%

[mmogallapalli]

LGTM