Fix CI failures on mvp-external-batch-fixes (all 3 jobs) - #854
Closed
brianlball wants to merge 1 commit into
Closed
Conversation
…ional histograms, restore CRAN pin
- external_batch_nomad_spec: was stubbing a nomad CLI via --nomad-cmd, but
submit_nomad.rb drives the Nomad HTTP API (/v1/jobs/parse -> /v1/jobs) and
lays out <loc>/analysis_<id>/{package,results}; spec also died in before
hook on an interpolating heredoc (NameError: output_flag), masking the
mismatch. Rewritten against the HTTP API with an in-spec stub server.
- nomad/sync_results.rb: NFS branch mirrors via FileUtils instead of rsync
(absent on Windows dev boxes and slim containers; loop hung without it)
- sampling/lhs.rb: preflight histogram attach non-fatal; paperclip needs
ImageMagick identify, absent on ubuntu-24.04/macos-15 runners + PAT hosts
- docker/R/install_packages.R: restore develop's CRAN snapshot pin (ba21522);
branch's revert picked up RcppParallel 6.1.1 which needs cmake, breaking
the rserve image build
Verified locally: 30 rspec examples 0 failures (sampling_lhs, nomad, aws,
external_batch); fresh Rscript install_packages.R in nrel/openstudio-r:4.4.0
exits 0 with snapshot binaries.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
Superseded by #856, which rebuilds mvp-external-batch-fixes on current develop. This branch was based on develop-as-of-#840: it duplicated the external-batch work that merged as #842 and was missing #844–#853, and its linux/macos failures were caused by the branch's own bin/openstudio_meta bundler config-set rewrite (test-phase bundle config poisoned the export phase). #856 keeps the Nomad/Ansible work (original authorship), the nomad spec rewrite and CI fixes from this PR, and drops the 3.10.0/mongo-6 downgrades and the openstudio_meta rewrite. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the three failing jobs in run 30320382392. All failures were introduced on this branch;
developis green.linux-test / macos-test
1. Nomad spec rewritten to match
submit_nomad.rb's real contractThe spec stubbed a
nomadCLI and passed--nomad-cmd, butsubmit_nomad.rbhas no such option — it drives the Nomad HTTP API (POST /v1/jobs/parse→POST /v1/jobs) and lays out the shared location as<loc>/analysis_<id>/{package,results}(the spec expectedanalysis_runs/runs/...). This was masked by a second bug: the stub was written with an interpolating heredoc, so unescaped#{output_flag}raisedNameErrorin thebeforehook and killed all 3 examples before the mismatch could surface. The spec now stubs the HTTP API with a minimal in-specTCPServer, asserts on the rendered HCL (count, job name, package/results URIs), and runs the full package → submit → array-tasks → sync → ingest pipeline.2.
sync_results.rb: NFS branch no longer requires rsyncThe local/NFS path shelled out to
rsync; on hosts without it the sync loop spun forever (--interval 0). Local mirroring is now in-processFileUtils.cp_r; S3 and SSH branches are unchanged.3. Ruby LHS backend: preflight histogram attach is non-fatal
sample_all_variablesattaches the generated histogram viaPreflightImage.add_from_disk, which runs paperclip styles through ImageMagickidentify— not installed on ubuntu-24.04/macos-15 runner images or PAT-local hosts. The attach is now wrapped in a rescue (warn + continue), so sampling degrades gracefully instead of failing the analysis. This is what broke all 5sampling_lhs_specexamples.docker
4. Restore the CRAN snapshot pin in
docker/R/install_packages.RThis branch reverted the date-frozen Posit snapshot (ba21522) back to
cloud.r-project.org, which picked up RcppParallel 6.1.1 — its configure needs cmake, soRcppParallel→dtwclust→sensitivitycascaded and the rserve image build died. Restoreddevelop's pinned version; the snapshot serves prebuilt jammy binaries, no cmake needed.Verification
sampling_lhs_spec+external_batch_nomad_spec+external_batch_aws_spec+external_batch_spec: 30 examples, 0 failures (local-test env, on a host without ImageMagick, so the rescue path in fix 3 is genuinely exercised)Rscript install_packages.Rinsidenrel/openstudio-r:4.4.0: exit 0; RcppParallel/dtwclust/sensitivity install as snapshot binariesNote: this branch forked from 5554f4a and is 7 commits behind
develop; a rebase is worth doing regardless (it also dropped the rserve CI caching from ba21522's workflow changes).🤖 Generated with Claude Code