feat(cli): load a per-run RunConfig YAML in data-designer create

[eric-tramel]

Priority Level

Medium (Nice to have)

Is your feature request related to a problem? Please describe.

RunConfig controls operational behavior for a generation run, but data-designer create cannot load one from a file. Python callers can construct a RunConfig and call DataDesigner.set_run_config(...); CLI users must instead write a wrapper script or accept the defaults, apart from the existing --tui/--no-tui override.

This makes repeatable CLI execution awkward for settings such as buffer size, scheduler capacity, early-shutdown behavior, tracing, dropped-column preservation, Jinja rendering, and request-admission tuning. Adding one CLI flag per RunConfig field would duplicate a growing Pydantic schema and make the CLI harder to maintain.

The workflow/dataset config and the runtime config are different inputs. The existing positional CONFIG_SOURCE declares what data to generate; this request adds a second, optional YAML file that controls how that one create invocation runs.

Describe the solution you'd like

Add an optional --run-config / -c option to data-designer create:

data-designer create DATASET_CONFIG [OPTIONS] [--run-config RUN_CONFIG.yaml]
data-designer create DATASET_CONFIG [OPTIONS] [-c RUN_CONFIG.yaml]

DATASET_CONFIG remains required. A bare data-designer create -c run-config.yaml is not valid because the current command still needs a dataset configuration.

File contract

• Accept one local .yaml or .yml file.
• The YAML root is a direct mapping of RunConfig fields; do not require a run_config: wrapper.
• Partial files are valid. Keys present in the file override the active DataDesigner.run_config; omitted top-level fields retain the active baseline (currently the built-in RunConfig defaults, and compatible with a future persisted baseline from Support system-level RunConfig defaults in ~/.data-designer #559 or Consolidate Data Designer user configuration into a single TOML file #694).
• If request_admission is present, validate it as one nested object. Deep-merging multiple runtime files is out of scope.
• Parse with safe YAML loading and validate through RunConfig, so field constraints, nested validation, unknown-field rejection, normalization, and existing deprecation behavior stay identical to the Python API.
• Apply the validated object through DataDesigner.set_run_config(...) before generation starts.
• Show Run config: <path> in the create summary when the option is supplied.
• When the option is omitted, preserve current behavior exactly.

Precedence

From lowest to highest:

1. built-in RunConfig defaults;
2. any future persisted user/system baseline from Support system-level RunConfig defaults in ~/.data-designer #559 or Consolidate Data Designer user configuration into a single TOML file #694;
3. keys supplied by the per-run YAML;
4. explicit invocation flags, currently --tui / --no-tui.

The existing DATA_DESIGNER_ASYNC_TRACE=1 environment variable remains a field-specific exception: it forces tracing on even if YAML contains async_trace: false.

--num-records, --dataset-name, --artifact-path, --resume, and --output-format are execution arguments, not RunConfig fields, and remain separate CLI options.

Usage examples

Minimal run-config.yaml:

buffer_size: 250
max_in_flight_tasks: 128
display_tui: false
progress_interval: 10.0

Use the long option:

data-designer create dataset.yaml --run-config run-config.yaml

Use the requested short option with other create settings:

data-designer create dataset.yaml \
  -c /path/to/run-config.yaml \
  --num-records 10000 \
  --dataset-name training-data \
  --artifact-path ./artifacts

The runtime file also works when the dataset config is a local Python module:

data-designer create workflow.py -c run-config.yaml

An explicit CLI flag wins over YAML:

# Even if run-config.yaml contains display_tui: true, this invocation disables it.
data-designer create dataset.yaml -c run-config.yaml --no-tui

Resume with the same runtime-sensitive settings:

data-designer create dataset.yaml \
  -c run-config.yaml \
  --dataset-name training-data \
  --resume always

Complete current top-level schema using defaults:

disable_early_shutdown: false
shutdown_error_rate: 0.5
shutdown_error_window: 10
buffer_size: 1000
max_in_flight_tasks: 1024
non_inference_max_parallel_workers: 4
max_conversation_restarts: 5
max_conversation_correction_steps: 0
async_trace: false
display_tui: true
progress_interval: 5.0
preserve_dropped_columns: true
jinja_rendering_engine: secure
request_admission: null

Advanced request-admission tuning:

request_admission:
  multiplicative_decrease_factor: 0.75
  additive_increase_step: 1
  successes_until_increase: 25
  cooldown_seconds: 2.0
  startup_ramp_seconds: 30.0

Examples should use the canonical display_tui and request_admission names. Deprecated progress_bar and throttle input should receive the same compatibility warnings and normalization already provided by RunConfig; the CLI loader should not implement a second compatibility layer.

Validation and error behavior

Reject the runtime file before generation starts when it is:

• missing, unreadable, or a directory;
• not a .yaml or .yml file;
• empty or malformed YAML;
• a scalar or list instead of a mapping;
• using an unknown field;
• using an invalid scalar or nested request_admission value.

Report the path and relevant field without a traceback, exit nonzero, and do not call DataDesigner.create() or write generation artifacts. For example:

Failed to load run config from 'run-config.yaml':
buffer_size: Input should be greater than 0

“Match the YAML” means the validated, normalized effective RunConfig, not byte-for-byte equality. For example, disable_early_shutdown: true intentionally normalizes shutdown_error_rate to 1.0; display_tui: true still falls back to log output without a TTY.

Recommended implementation

Keep this in the existing thin CLI path; no engine or public-interface API change is needed.

1. Add run_config_source: str | None as --run-config/-c in the create command and pass it to GenerationController.run_create.
2. Add a small load_run_config helper beside load_config_builder. Reuse safe YAML parsing, validate with RunConfig.model_validate(...), and normalize failures to the existing ConfigLoadError.
3. To preserve partial-file and future persisted-default precedence, canonicalize only explicitly supplied fields (for example via a validated model's model_dump(exclude_unset=True)), shallow-overlay them onto data_designer.run_config.model_dump(), and validate the effective object again.
4. Apply an explicit TUI flag to the effective model after the YAML overlay.
5. Call DataDesigner.set_run_config(effective_run_config) exactly once. This is required because the method also rebuilds the request-admission controller.
6. Invoke DataDesigner.create().
7. Keep imports inside the lazily loaded create path so top-level CLI startup remains fast.

Likely production files:

• packages/data-designer/src/data_designer/cli/commands/create.py
• packages/data-designer/src/data_designer/cli/controllers/generation_controller.py
• packages/data-designer/src/data_designer/cli/utils/config_loader.py

Acceptance criteria

• data-designer create dataset.yaml --run-config run.yaml and data-designer create dataset.yaml -c run.yaml both work.
• create --help distinguishes the required dataset config from the optional runtime config and documents format, precedence, and examples.
• A valid partial file overlays only the supplied top-level fields; omitted fields retain the active baseline.
• A full file, including nested request_admission, becomes the active DataDesigner.run_config before generation.
• The controller calls set_run_config(); it does not mutate _run_config or its fields directly.
• --tui and --no-tui override YAML display_tui; without either flag, YAML wins.
• The create summary shows the runtime-config path when supplied.
• Existing deprecated keys receive the same warnings and normalization as direct RunConfig construction.
• Every current canonical RunConfig field is accepted and reaches the active model, without hard-coding a second field list in orchestration code.
• Missing, unreadable, empty, malformed, non-mapping, unknown-key, and constraint-invalid files fail clearly before create().
• Existing invocations without --run-config behave identically.
• Resume uses the loaded buffer_size and preserve_dropped_columns and retains the engine's existing compatibility checks and messages.
• Tests cover lazy CLI parsing/help, command delegation, full and partial YAML, nested values, baseline/YAML/TUI precedence, and all validation failures.
• Update the create-command examples, architecture/cli.md, and the RunConfig/architecture-and-performance documentation.
• Targeted CLI tests and repository formatting/lint checks pass.

Current implementation gotchas

• The positional dataset config is already the primary config source; -c is mechanically free, but --run-config must be the unambiguous documented name.
• DataDesigner.set_run_config() recreates the request-admission controller. Direct assignment or in-place mutation would leave stale request-admission state.
• RunConfig forbids unknown fields and performs compatibility translation and normalization. Reimplementing field assignment in the controller would drift as the schema evolves.
• async_trace: false cannot disable DATA_DESIGNER_ASYNC_TRACE=1; this existing OR behavior must be documented rather than silently presented as YAML winning.
• display_tui: true is conditional on a TTY.
• Resume currently requires buffer_size and preserve_dropped_columns to match the interrupted run. Most other runtime settings are not part of the dataset-config fingerprint and may change on resume.
• non_inference_max_parallel_workers is declared and documented but currently has no production consumer. This feature can load it into RunConfig, but must not claim that the existing engine knob changes execution until that separate engine gap is resolved.
• The complete effective RunConfig is not persisted in artifacts. builder_config.json contains only the dataset builder config, and metadata records selected runtime identity fields. Persisting or copying the runtime YAML is a separate reproducibility feature.
• Support system-level RunConfig defaults in ~/.data-designer #559's examples use now-deprecated progress_bar and throttle names. This issue and its docs should use the current display_tui and request_admission schema.

Non-goals

• Adding this option to preview, validate, or check-models in the first change.
• Supporting JSON, URLs, Python runtime-config modules, stdin, environment interpolation, YAML includes, multiple files, or deep merging.
• Embedding RunConfig in the dataset/workflow config.
• Adding one CLI option per runtime field.
• Changing the Python API, engine orchestration, resume fingerprint, or artifact schema.
• Persisting the complete effective RunConfig or generating a starter file.
• Implementing global/user defaults from Support system-level RunConfig defaults in ~/.data-designer #559 or Consolidate Data Designer user configuration into a single TOML file #694.
• Repairing the pre-existing non_inference_max_parallel_workers engine gap.

Describe alternatives you've considered

• Add a CLI flag for every field. This duplicates the Pydantic model, does not scale to nested settings, and will drift as RunConfig evolves.
• Embed runtime settings in the dataset config. This mixes operational tuning with the declarative dataset definition and weakens the existing separation between config and execution.
• Require a Python wrapper. This works today but bypasses the simple, standard data-designer create workflow for a data-only runtime configuration.
• Use only persisted global defaults (Support system-level RunConfig defaults in ~/.data-designer #559/Consolidate Data Designer user configuration into a single TOML file #694). Global defaults solve a different problem; they do not provide a versionable, explicit per-job input.
• Support every existing dataset-config source type. URLs, JSON, and executable Python modules are unnecessary for the requested local YAML use case and broaden the trust/error surface.

Agent Investigation

No exact duplicate was found after searching open and closed issues, PRs, and repository Discussions for RunConfig, per-run config, YAML, config-file, and CLI create variants.

Related work:

• Support system-level RunConfig defaults in ~/.data-designer #559 proposes persisted system-level RunConfig defaults. This request is the explicit per-invocation layer that should take precedence over those defaults.
• Consolidate Data Designer user configuration into a single TOML file #694 proposes consolidated global TOML configuration and explicitly keeps workflow files separate.
• feat: add preview, create, and validate CLI commands #313 introduced create, preview, and validate with a positional workflow config.
• feat: support custom CLI args for Python config modules #507 covers custom arguments forwarded to Python workflow modules. --run-config remains a first-class Data Designer option before any future -- separator.
• fix(interface): don't leak YAML default provider into user-supplied list #591 is a useful precedence precedent: explicit runtime input should not be silently replaced by persisted YAML defaults.

Code findings at current main:

• The create command already owns the relevant option surface and delegates to a generation controller: create.py.
• The controller constructs DataDesigner, applies the existing TUI override, and calls the public create() API: generation_controller.py.
• set_run_config() replaces the model and rebuilds request admission: data_designer.py.
• RunConfig owns the complete schema, constraints, deprecation translation, and normalization: run_config.py.
• The shared config base rejects unknown fields: base.py.
• Safe mapping-oriented YAML loading already exists, so no dependency is needed: io_helpers.py.
• The CLI already centralizes friendly config-loading failures: config_loader.py.
• Resume checks the runtime-sensitive buffer and dropped-column policy: dataset_builder.py.
• Async trace uses existing environment-variable OR semantics: dataset_builder.py.
• Artifacts currently serialize only BuilderConfig(data_designer=...), not the complete runtime model: dataset_builder.py.

Additional context

This is deliberately create-only and local-YAML-only for the first implementation. It gives the CLI one typed escape hatch for the whole runtime schema without adding a parallel option surface or changing engine behavior. If users need runtime files on preview/validation, remote sources, artifact provenance, or repair of currently ineffective engine settings, those can be scoped independently after this path is proven.

Checklist

• I've reviewed existing issues and the documentation
• This is a design proposal, not a "please build this" request