Skip to content

firecracker: fix docker -p#11427

Merged
bduffany merged 1 commit intomasterfrom
firecracker-iptables
Mar 2, 2026
Merged

firecracker: fix docker -p#11427
bduffany merged 1 commit intomasterfrom
firecracker-iptables

Conversation

@bduffany
Copy link
Member

@bduffany bduffany commented Feb 26, 2026

Docker v28 configures direct-access filtering via iptables -t raw for published ports. As a result, commands like docker run -p <port> fail with

iptables ... can't initialize iptables table `raw': Table does not exist

because our current guest kernel configs all have CONFIG_IP_NF_RAW disabled.

Repro: run the same bb execute command with firecracker isolation, with and without --action_env=DOCKER_INSECURE_NO_IPTABLES_RAW=1; manually start dockerd &, sleep for a bit then run docker run -p ...

Fix:

  • Set DOCKER_INSECURE_NO_IPTABLES_RAW=1 unconditionally when starting dockerd in goinit. Includes an inline note explaining why this is safe for our VM sandboxing model.
  • Add TODOs at the top of all guest kernel config files to enable CONFIG_NF_TABLES and remove the env-var fallback. (I don't want to do this right now since I think it's slightly risky, so I'd rather save this for when we incrementally start migrating to our new 6.1 config)
  • Update docker-in-firecracker coverage to include docker run -p ... on a newer docker version.

Context: https://buildbuddy-corp.slack.com/archives/C0AFN0SMNG5/p1772129387671119?thread_ts=1772012498.295389&cid=C0AFN0SMNG5

…upport

Starting in Docker 28 (the version used by our Ubuntu 24.04 image),
dockerd introduced direct-access filtering for published ports as a
hardening change. That path programs iptables `-t raw` rules, which in
Firecracker guest VMs can fail with:

  iptables ... can't initialize iptables table `raw': Table does not exist

because our current guest kernel configs all have `CONFIG_IP_NF_RAW` disabled.

Changes:
- Set `DOCKER_INSECURE_NO_IPTABLES_RAW=1` unconditionally when starting
  dockerd in goinit.
- Keep an inline note explaining why this is safe for our VM sandboxing model.
- Update docker-in-firecracker coverage to include `docker run -p ...` in
  `TestFirecrackerRunWithDockerOverUDS`.
- Update guest API hash expectation.
- Add TODOs at the top of all guest kernel config files to enable
  `CONFIG_NF_TABLES` and remove the env-var fallback.

Repro (before/after): run the same `bb execute` command with and without
`--action_env=DOCKER_INSECURE_NO_IPTABLES_RAW=1`; `docker run -p ...` fails
without it and succeeds with it.
@bduffany bduffany force-pushed the firecracker-iptables branch from 96aad30 to d633570 Compare February 26, 2026 18:15
Copy link
Contributor

@dan-stowell dan-stowell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch!

@bduffany bduffany merged commit 7016df1 into master Mar 2, 2026
13 checks passed
@bduffany bduffany deleted the firecracker-iptables branch March 2, 2026 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants