Skip to content

Fix CI failures on mvp-external-batch-fixes (all 3 jobs) - #854

Closed
brianlball wants to merge 1 commit into
mvp-external-batch-fixesfrom
fix/mvp-external-batch-ci
Closed

Fix CI failures on mvp-external-batch-fixes (all 3 jobs)#854
brianlball wants to merge 1 commit into
mvp-external-batch-fixesfrom
fix/mvp-external-batch-ci

Conversation

@brianlball

Copy link
Copy Markdown
Contributor

Fixes the three failing jobs in run 30320382392. All failures were introduced on this branch; develop is green.

linux-test / macos-test

1. Nomad spec rewritten to match submit_nomad.rb's real contract
The spec stubbed a nomad CLI and passed --nomad-cmd, but submit_nomad.rb has no such option — it drives the Nomad HTTP API (POST /v1/jobs/parsePOST /v1/jobs) and lays out the shared location as <loc>/analysis_<id>/{package,results} (the spec expected analysis_runs/runs/...). This was masked by a second bug: the stub was written with an interpolating heredoc, so unescaped #{output_flag} raised NameError in the before hook and killed all 3 examples before the mismatch could surface. The spec now stubs the HTTP API with a minimal in-spec TCPServer, asserts on the rendered HCL (count, job name, package/results URIs), and runs the full package → submit → array-tasks → sync → ingest pipeline.

2. sync_results.rb: NFS branch no longer requires rsync
The local/NFS path shelled out to rsync; on hosts without it the sync loop spun forever (--interval 0). Local mirroring is now in-process FileUtils.cp_r; S3 and SSH branches are unchanged.

3. Ruby LHS backend: preflight histogram attach is non-fatal
sample_all_variables attaches the generated histogram via PreflightImage.add_from_disk, which runs paperclip styles through ImageMagick identify — not installed on ubuntu-24.04/macos-15 runner images or PAT-local hosts. The attach is now wrapped in a rescue (warn + continue), so sampling degrades gracefully instead of failing the analysis. This is what broke all 5 sampling_lhs_spec examples.

docker

4. Restore the CRAN snapshot pin in docker/R/install_packages.R
This branch reverted the date-frozen Posit snapshot (ba21522) back to cloud.r-project.org, which picked up RcppParallel 6.1.1 — its configure needs cmake, so RcppParalleldtwclustsensitivity cascaded and the rserve image build died. Restored develop's pinned version; the snapshot serves prebuilt jammy binaries, no cmake needed.

Verification

  • sampling_lhs_spec + external_batch_nomad_spec + external_batch_aws_spec + external_batch_spec: 30 examples, 0 failures (local-test env, on a host without ImageMagick, so the rescue path in fix 3 is genuinely exercised)
  • Fresh Rscript install_packages.R inside nrel/openstudio-r:4.4.0: exit 0; RcppParallel/dtwclust/sensitivity install as snapshot binaries

Note: this branch forked from 5554f4a and is 7 commits behind develop; a rebase is worth doing regardless (it also dropped the rserve CI caching from ba21522's workflow changes).

🤖 Generated with Claude Code

…ional histograms, restore CRAN pin

- external_batch_nomad_spec: was stubbing a nomad CLI via --nomad-cmd, but
  submit_nomad.rb drives the Nomad HTTP API (/v1/jobs/parse -> /v1/jobs) and
  lays out <loc>/analysis_<id>/{package,results}; spec also died in before
  hook on an interpolating heredoc (NameError: output_flag), masking the
  mismatch. Rewritten against the HTTP API with an in-spec stub server.
- nomad/sync_results.rb: NFS branch mirrors via FileUtils instead of rsync
  (absent on Windows dev boxes and slim containers; loop hung without it)
- sampling/lhs.rb: preflight histogram attach non-fatal; paperclip needs
  ImageMagick identify, absent on ubuntu-24.04/macos-15 runners + PAT hosts
- docker/R/install_packages.R: restore develop's CRAN snapshot pin (ba21522);
  branch's revert picked up RcppParallel 6.1.1 which needs cmake, breaking
  the rserve image build

Verified locally: 30 rspec examples 0 failures (sampling_lhs, nomad, aws,
external_batch); fresh Rscript install_packages.R in nrel/openstudio-r:4.4.0
exits 0 with snapshot binaries.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@brianlball

Copy link
Copy Markdown
Contributor Author

Superseded by #856, which rebuilds mvp-external-batch-fixes on current develop. This branch was based on develop-as-of-#840: it duplicated the external-batch work that merged as #842 and was missing #844#853, and its linux/macos failures were caused by the branch's own bin/openstudio_meta bundler config-set rewrite (test-phase bundle config poisoned the export phase). #856 keeps the Nomad/Ansible work (original authorship), the nomad spec rewrite and CI fixes from this PR, and drops the 3.10.0/mongo-6 downgrades and the openstudio_meta rewrite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant