fix(aws-lambda): fail build-zip when ffmpeg-static binary isn't Linux x86-64#1058
Open
kiyeonjeon21 wants to merge 1 commit into
Open
fix(aws-lambda): fail build-zip when ffmpeg-static binary isn't Linux x86-64#1058kiyeonjeon21 wants to merge 1 commit into
kiyeonjeon21 wants to merge 1 commit into
Conversation
…x x86-64 `ffmpeg-static` materializes a single platform-selected binary at install time. By default that follows the install host, so a default macOS/arm64 or linux/arm64 install can stage a binary that Lambda's x86-64 runtime cannot execute. Verify the ELF header (ELFCLASS64, EM_X86_64) before copying the binary into `bin/ffmpeg`. When the check fails, surface the canonical workarounds: run the build inside a linux/amd64 container, or pre-install with `npm_config_platform=linux npm_config_arch=x64`. In the distributed pipeline this mismatch can fail as early as Plan's `ffmpeg -version` probe after browser probing and before chunks are scheduled. Later encode/assemble spawns would fail for the same reason. ffprobe already goes through the platform-segmented `ffprobe-static/bin/linux/x64` path, so that side doesn't need the same check.
354a7f3 to
9b34114
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
stageFfmpeginpackages/aws-lambda/scripts/build-zip.tsnow reads the ELF header offfmpeg-static/ffmpegbefore copying it into the deploy zip, and throws if the binary isn't a Linux x86-64 ELF64 executable. This catches the case where the install step materialized the wrong ffmpeg for Lambda's x86-64 runtime (for example, a default macOS arm64 install), which otherwise produces a zip Lambda accepts but can't actually run.Closes #1057.
Why
ffmpeg-staticmaterializes a single platform-selected binary at install time. By default that is the install host's platform/arch. Anyone runningbun run build:zipfrom macOS or a Linux/arm64 box without target-platform overrides can get a zip that uploads to Lambda cleanly and then dies the first time Lambda invokes ffmpeg:In the current distributed path,
plan()callsffmpeg -versionafter the browser probe and before returning the plan result, so the observed failure can happen in the Plan state before any chunks are scheduled. Any later encode/assemble spawn would fail for the same binary-mismatch reason.ffprobe-staticdoesn't have this problem — it ships every platform underbin/<os>/<arch>/, andstageFfmpegalready picks the linux/x64 path explicitly. The bug is specific to the ffmpeg side.A defensive check is the smallest blast-radius fix: it doesn't change dependencies or the install flow, it just refuses to ship a broken zip. The error message points at the canonical workarounds (
docker run --platform=linux/amd64, ornpm_config_platform=linux npm_config_arch=x64) so the operator can recover without digging into CloudWatch logs.Heavier alternatives (swapping
ffmpeg-staticfor@ffmpeg-installer/ffmpeg, downloading the binary on demand) are spelled out in #1057 — happy to follow up with one of those if it's a better fit for the codebase, but they felt out of scope for an upstream patch.How
assertLinuxX86_64Elf(path, label)reads the first 20 bytes of the binary and checks:0x7F 'E' 'L' 'F'2(ELFCLASS64)0x3E(EM_X86_64)On mismatch it throws with the parsed header values so the failure mode is obvious from the build log, plus the workaround commands.
stageFfmpegcalls it right after theexistsSyncguard so the check runs beforecpSynctouches the staging dir.A small
readFileHeadhelper keepsassertLinuxX86_64Elfshaped like the rest of the file (one job per function, no nested fs handles).Test plan
bun run --cwd packages/aws-lambda typecheckbun run --cwd packages/aws-lambda test— 109 pass, 0 failbun run lint— cleanbun run format:check— cleanbunx fallow audit --base origin/main --fail-on-issues— no new findings (5 inherited findings excluded by the default--gate new-only)build:zipagainst affmpeg-staticinstall with the macOS arm64 binary in place and confirmed the build now fails with a descriptive error instead of producing a broken zip.build:zipagainst affmpeg-staticinstall hydrated vianpm_config_platform=linux npm_config_arch=x64 bun installand confirmed it produces the expected linux/x64 zip without complaint.Notes
ffprobe-static/bin/linux/x64/ffprobeexplicitly.openSync+ 20-bytereadSyncper build.