Skip to content

chore: refactor data-designer plugin tests - #16

Merged
mikeknep merged 17 commits into
mainfrom
dd-testing/mknepper
May 22, 2026
Merged

chore: refactor data-designer plugin tests#16
mikeknep merged 17 commits into
mainfrom
dd-testing/mknepper

Conversation

@mikeknep

Copy link
Copy Markdown
Contributor

What's here

I find integration-level tests, in which we exercise the system through end user-facing interfaces as much as possible, provide greater confidence that the plugin is working as intended, especially nowadays when agents feel perfectly at liberty to change source code and (usually unit) test code when implementing some feature. This PR mostly reorganizes and adds tests to increase the integration-level coverage. There's still more work to be done here (always!), but this is a step in the right direction and should reduce the feeling of needing to manually run some tests as a sanity check at least a little bit.

Coverage stats

There are two important packages to pay attention to:

  • plugins/nemo-data-designer/src/nemo_data_designer_plugin: where the plugin entry-point interfaces are implemented
  • packages/data_designer_nemo/src/data_designer_nemo: the "pure-library" glue code that knows how to use the Data Designer library in the NeMo Platform context

The plugin integration test coverage of these packages is:

Branch Coverage
main 68.58%
this PR 78.71% (+10 pp)

mikeknep added 15 commits May 21, 2026 15:21
Signed-off-by: Mike Knepper <mknepper@nvidia.com>
Signed-off-by: Mike Knepper <mknepper@nvidia.com>
Signed-off-by: Mike Knepper <mknepper@nvidia.com>
Signed-off-by: Mike Knepper <mknepper@nvidia.com>
Signed-off-by: Mike Knepper <mknepper@nvidia.com>
Signed-off-by: Mike Knepper <mknepper@nvidia.com>
Signed-off-by: Mike Knepper <mknepper@nvidia.com>
Signed-off-by: Mike Knepper <mknepper@nvidia.com>
Signed-off-by: Mike Knepper <mknepper@nvidia.com>
Signed-off-by: Mike Knepper <mknepper@nvidia.com>
Signed-off-by: Mike Knepper <mknepper@nvidia.com>
Signed-off-by: Mike Knepper <mknepper@nvidia.com>
Signed-off-by: Mike Knepper <mknepper@nvidia.com>
Signed-off-by: Mike Knepper <mknepper@nvidia.com>
Signed-off-by: Mike Knepper <mknepper@nvidia.com>
@mikeknep
mikeknep requested review from a team as code owners May 21, 2026 20:54
@github-actions

github-actions Bot commented May 21, 2026

Copy link
Copy Markdown
Contributor
Suite Lines Covered Line Rate Branch Rate
Unit Tests 17993/23916 75.2% 61.6%
Integration Tests 11465/22698 50.5% 25.6%

@mckornfield mckornfield left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some nice tests there

Comment thread plugins/nemo-data-designer/src/nemo_data_designer_plugin/sdk/errors.py Outdated
Comment thread plugins/nemo-data-designer/tests/integration/test_job_sdk.py Outdated
Comment thread plugins/nemo-data-designer/tests/unit/test_errors.py
mikeknep added 2 commits May 22, 2026 08:23
Signed-off-by: Mike Knepper <mknepper@nvidia.com>
Signed-off-by: Mike Knepper <mknepper@nvidia.com>
@mikeknep
mikeknep enabled auto-merge May 22, 2026 13:24
@mikeknep
mikeknep added this pull request to the merge queue May 22, 2026
Merged via the queue into main with commit f646ea4 May 22, 2026
11 checks passed
@mikeknep
mikeknep deleted the dd-testing/mknepper branch May 22, 2026 16:58
benmccown added a commit that referenced this pull request Jun 23, 2026
Composition over inheritance for the k8s reconcilers (review #2/#3):
* Extract StatusProjector (pod-status projection, crash-loop/pending-timeout
  error builders, host URL) and ResourceDeleter (idempotent 404-tolerant
  delete) as standalone collaborators.
* Reconciler is now a pure interface (the 5 verbs); NimOperatorReconciler and
  K8sReconciler compose the projector + deleter instead of inheriting them.
  The backend builds both collaborators in init() and injects them.

Thread the reconcile context through the backend interface (review #19):
* create/update/get_model_deployment_status now take a single
  ctx: ModelContext instead of (deployment, config, model_entity); applied
  across the ServiceBackend ABC and the docker / none / k8s backends, the
  deployment reconciler call sites, and the test mocks. delete stays
  (workspace, name).

Fixes + nits:
* Harden NIMService status read against a null status/state (review #15):
  (nim_status.get("state") or "").lower() can no longer raise.
* Convert nim_operator logging to structured extra={} (review #13); avoid the
  reserved LogRecord 'name' key (use resource_name / deployment_name).
* Flatten the Files-service create/update branches into a guard-clause helper
  (review #14).
* compile_puller_job: rename args -> container_args (review #17).
* Reconciler nits: import the vllm_k8s_compiler module under its full name
  (review #7), reflow the P3 (a)/(b) comment (review #8), quote values in the
  model-source error (review #9), drop the _ = image_pull_secrets dance
  (review #12), name the event-message cap MAX_EVENT_MESSAGE_CHARS (review #6),
  and document the _select_reconciler None contract (review #16).

Signed-off-by: Ben McCown <bmccown@nvidia.com>
benmccown added a commit that referenced this pull request Jun 24, 2026
Composition over inheritance for the k8s reconcilers (review #2/#3):
* Extract StatusProjector (pod-status projection, crash-loop/pending-timeout
  error builders, host URL) and ResourceDeleter (idempotent 404-tolerant
  delete) as standalone collaborators.
* Reconciler is now a pure interface (the 5 verbs); NimOperatorReconciler and
  K8sReconciler compose the projector + deleter instead of inheriting them.
  The backend builds both collaborators in init() and injects them.

Thread the reconcile context through the backend interface (review #19):
* create/update/get_model_deployment_status now take a single
  ctx: ModelContext instead of (deployment, config, model_entity); applied
  across the ServiceBackend ABC and the docker / none / k8s backends, the
  deployment reconciler call sites, and the test mocks. delete stays
  (workspace, name).

Fixes + nits:
* Harden NIMService status read against a null status/state (review #15):
  (nim_status.get("state") or "").lower() can no longer raise.
* Convert nim_operator logging to structured extra={} (review #13); avoid the
  reserved LogRecord 'name' key (use resource_name / deployment_name).
* Flatten the Files-service create/update branches into a guard-clause helper
  (review #14).
* compile_puller_job: rename args -> container_args (review #17).
* Reconciler nits: import the vllm_k8s_compiler module under its full name
  (review #7), reflow the P3 (a)/(b) comment (review #8), quote values in the
  model-source error (review #9), drop the _ = image_pull_secrets dance
  (review #12), name the event-message cap MAX_EVENT_MESSAGE_CHARS (review #6),
  and document the _select_reconciler None contract (review #16).

Signed-off-by: Ben McCown <bmccown@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants