ci: fix visual regression tests — provision a pinned Chrome for Puppeteer + Percy#3255
Merged
Conversation
The build_lint_and_test job cached only the pnpm store. Puppeteer downloads Chrome to ~/.cache/puppeteer (outside node_modules / the store), and pnpm's side-effects cache records the postinstall as already-run on a store cache hit, so the browser binary was never preserved — the explicit `puppeteer browsers install chrome` step re-fetched it every run. When the runner image shipped a stale/partial ~/.cache/puppeteer for the pinned Chrome build, @puppeteer/browsers refused to overwrite it and failed with "the browser folder exists but the executable is missing", blocking the job. Relocate the browser to a workspace-local PUPPETEER_CACHE_DIR and cache that directory (keyed on the lockfile). The install step is now a no-op on a cache hit and a clean download into an empty dir on a miss, so it can't collide with a partially-populated system cache. No `rm -rf`, and the binary is actually preserved across runs.
|
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Resolve the installed puppeteer version and use it as the cache key, instead of the lockfile hash + restore-keys. The Chrome build is pinned transitively by the puppeteer dependency, so a Puppeteer bump now changes the key and forces a clean re-download, while same-version runs reuse the cached binary. Dropping restore-keys avoids ever falling back to a stale-version browser, and avoids the cache churning on unrelated lockfile changes.
Root cause of the install failure: with PUPPETEER_CACHE_DIR set, Puppeteer's postinstall (run during `pnpm install` under the restored pnpm store) leaves a partial Chrome — the version folder without the executable. @puppeteer/browsers 2.3.0 then refuses to repair it and fails the explicit install with "the browser folder exists but the executable is missing". Set PUPPETEER_SKIP_DOWNLOAD=true for the install step so downloadBrowser() returns before touching the cache dir (verified against puppeteer 22.15.0's node/install.js). The explicit `puppeteer browsers install chrome` step — which does not consult that flag — remains the sole, authoritative installer into the version-keyed cache dir.
Reverts the cache/relocation/skip-download experiments back to main's working flow. Adds a single step to remove a partial ~/.cache/puppeteer (browser folder without the executable) that the runner image can ship — @puppeteer/browsers refuses to repair an incomplete install (same check in 2.3.0 through 2.13.2), so a version bump can't fix it. Clearing it lets the postinstall download a clean Chrome.
tdeekens
approved these changes
Jun 11, 2026
The ubuntu-24.04 runner image (~20260525) tightened AppArmor unprivileged-userns restrictions, preventing Percy/Chromium from launching — VRT ran green but captured 0 snapshots (Percy builds 'manually failed'). Last green VRT (81 snapshots, May 27) was on image 20260518. Pin the job to ubuntu-22.04, which has no such restriction, and drop the now- obsolete 24.04-specific workarounds: the aa-exec --profile=chrome wrapper (added for 24.04 AppArmor) and the ~/.cache/puppeteer clear (the husk was a 24.04 image artifact). This restores the pre-24.04 VRT configuration on a compatible OS.
Root cause of the 0-snapshot VRT runs: @percy/core 1.31.13 pins Chromium snapshot revision 1300309 and auto-downloads it for asset discovery. That revision has been pruned from the public Chromium snapshots bucket, so the download (which took 4s on May 27) now hangs indefinitely and Percy captures no snapshots — independent of the runner OS. Set PERCY_BROWSER_EXECUTABLE to the runner's pre-installed Chrome so @percy/core uses it and skips the download (verified in @percy/core dist/browser.js: executable || install.chromium()).
Both pinned browser downloads now fail in CI, yielding 0 snapshots: - puppeteer's Chrome-for-Testing 127.0.6533.88 no longer installs (jest-puppeteer then can't find Chrome -> globalSetup error) - @percy/core's Chromium snapshot 1300309 was pruned from the public bucket, so its auto-download hangs Point both at the runner's pre-installed Chrome via PUPPETEER_EXECUTABLE_PATH and PERCY_BROWSER_EXECUTABLE, and drop the now-redundant 'Install Puppeteer's Chrome' step. jest-puppeteer.config.js already reads PUPPETEER_EXECUTABLE_PATH and runs with --no-sandbox. Fails fast if no Chrome is found rather than silently falling back to a broken download.
misama-ct
changed the title
Revert the ubuntu-22.04 pin (unnecessary: deleting the puppeteer install step removes the ~/.cache/puppeteer husk issue, and --no-sandbox in both launchers makes the 24.04 AppArmor workaround moot — so 24.04 works and avoids the 22.04 deprecation track). Provision Chrome explicitly with browser-actions/setup-chrome (SHA-pinned) and point PUPPETEER_EXECUTABLE_PATH + PERCY_BROWSER_EXECUTABLE at it, instead of relying on the runner's rolling-latest Chrome. Using chrome-version: stable for now; will pin a specific build for reproducible Percy snapshots once green.
…ome 127 The rich-text-input and localized-rich-text-input visualspecs launch their OWN puppeteer browser (not the jest-puppeteer global one) and did not pass --no-sandbox. On ubuntu-24.04 (AppArmor restricts unprivileged user namespaces) that crashes with 'No usable sandbox', failing those 2 suites and dropping a Percy snapshot. They previously only passed because the removed aa-exec wrapper covered them. Add --no-sandbox/--disable-setuid-sandbox to both launches, matching jest-puppeteer.config.js used by the other 68 specs. Also pin setup-chrome to 127.0.6533.88 (puppeteer 22.15.0's target build and the Percy baseline's browser) instead of 'stable' — 'stable' pulled Chrome 149.
misama-ct
changed the title
misama-ct
force-pushed
the
ci/cache-puppeteer-browser
branch
from
6c99829 to
ea2055c
Compare
tylermorrisford
approved these changes
Jun 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
build_lint_and_testjob's visual-regression step started capturing 0 snapshots, so Percy builds came back "manually failed" / "no snapshots uploaded." The last healthy run was build 13795 (May 27, 81 snapshots); every run after that captured nothing — with no relevant change in this repo.Causes
Three independent things broke, all external to our source (the first two surfaced first; the third only became visible once the browsers were available again):
puppeteer browsers install chrome(Chrome-for-Testing127.0.6533.88, via@puppeteer/browsers@2.3.0) silently stopped installing, sojest-puppeteer'sglobalSetupfailed withCould not find Chrome (ver. 127.0.6533.88).@percy/core@1.31.13pins Chromium snapshot revision1300309and auto-downloads it; that revision was pruned from the public Chromium snapshots bucket, so the download went from ~4s (May 27) to a permanent hang (this.executable = executable || (await install.chromium())).ubuntu-24.04.rich-text-inputandlocalized-rich-text-inputlaunch their own Puppeteer browser (not the jest-puppeteer global one) without--no-sandbox. Onubuntu-24.04, AppArmor restricts unprivileged user namespaces, so those launches die with "No usable sandbox". They had only ever passed because of a now-removedaa-exec --profile=chromewrapper.Solution
Provision one explicit, pinned Chrome and use it for everything; stop relying on the tools' own (now-broken) downloads.
browser-actions/setup-chrome(SHA-pinned) installs127.0.6533.88— the buildpuppeteer@22.15.0targets and the one the Percy baseline was captured with.chrome-version: stablewas rejected: it pulled Chrome 149 (~22 majors newer).PUPPETEER_EXECUTABLE_PATH(read byjest-puppeteer.config.js) andPERCY_BROWSER_EXECUTABLE. This removes both failing downloads (Setup CI #1, Setup release script #2).--no-sandbox/--disable-setuid-sandboxto the two self-launching rich-text specs, matchingjest-puppeteer.config.jsused by the other 68 specs. Fixes chore: upgrade storybook to alpha.18 #3 without a privileged runner sysctl, on any OS.aa-execwrapper. Stay onubuntu-24.04(no longer needs either, and avoids theubuntu-22.04deprecation track).Verification
Green run on this branch → Percy build 13802:
Test Suites: 71 passed, 71 total(the two rich-text suites pass again)Finalized build #13802→exited with status: 0percy/ui-kit: pass — "automatically approved, no visual changes found" (Chrome 127 renders identically to the baseline — no baseline shift)Notes / follow-ups
~/.cache/puppeteerhusk, aubuntu-22.04pin, a brief unpinned-Chrome attempt). The net change is only the items under Solution.chrome-version— bump it in lockstep withpuppeteer.