Limit aproxy nftables redirection to egress traffic only#807
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates the OpenStack runner cloud-init template to scope aproxy’s transparent-proxy nftables DNAT rules to traffic that will egress via the runner’s default-route interface(s), reducing the need to exclude broad private IP ranges and avoiding local/overlay traffic being captured.
Changes:
- Add default-route interface filtering to aproxy nftables
preroutingandoutputchains. - Update unit-test expected nftables snippets to match the new rules.
- Bump
github-runner-managerversion and document the user-facing change indocs/changelog.md.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
github-runner-manager/src/github_runner_manager/templates/openstack-userdata.sh.j2 |
Adds default-route NIC detection and applies interface-based filtering to aproxy nftables redirection. |
github-runner-manager/tests/unit/openstack_cloud/test_openstack_runner_manager.py |
Updates expected nftables rules in unit tests to match the new template output. |
github-runner-manager/pyproject.toml |
Version bump from 0.18.2 to 0.18.3. |
docs/changelog.md |
Adds a 2026-07-14 entry describing the aproxy redirection behavior change. |
| @@ -15,6 +15,7 @@ snap install aproxy --edge | |||
| snap set aproxy proxy={{ aproxy_address }} listen=:54969 | |||
| cat << EOF > /etc/nftables.conf | |||
| define default-ipv4 = $(ip route get $(ip route show 0.0.0.0/0 | grep -oP 'via \K\S+') | grep -oP 'src \K\S+') | |||
There was a problem hiding this comment.
Note: This reply was written by an AI agent on behalf of the PR author.
I empirically tested this on Ubuntu 24.04 (iproute2 6.1.0) with two default routes configured. The pipeline does not fail under set -e: ip route get <gw1> <gw2> returns exit 0 and define default-ipv4 resolves to a valid address, so cloud-init is not broken.
That said, the underlying point is fair: with multiple default routes the current expression silently uses the last gateway, which is non-deterministic. Adding head -n1 (or sort-ing by metric) is a reasonable robustness/determinism improvement even though it isn't a correctness bug.
| } | ||
| chain output { | ||
| type nat hook output priority -100; policy accept; | ||
| oifname != \\$nics return |
There was a problem hiding this comment.
Note: This reply was written by an AI agent on behalf of the PR author.
This depends on the fib daddr oifname suggestion for the output chain (see the template thread). I verified that fib daddr oifname is Operation not supported in the nat output hook, and that plain oifname is correctly set there. So the expected test output matching oifname != $nics return reflects the correct, loadable ruleset — no change needed.
| } | ||
| chain output { | ||
| type nat hook output priority -100; policy accept; | ||
| oifname != \\$nics return |
There was a problem hiding this comment.
Note: This reply was written by an AI agent on behalf of the PR author.
Same as the L108 thread: fib daddr oifname is not supported in the nat output hook (Operation not supported), while plain oifname is correctly populated there. The expected oifname != $nics return in the test matches the correct implementation, so no change is needed.
| } | ||
| chain output { | ||
| type nat hook output priority -100; policy accept; | ||
| oifname != \$nics return |
There was a problem hiding this comment.
Note: This reply was written by an AI agent on behalf of the PR author.
I tested this claim directly and it does not hold on the runner kernel (Ubuntu 24.04). Two findings:
-
oifnameis correctly populated in the natoutputhook (priority -100). A probe ruleoifname "vhost" ip daddr 10.200.0.2 countermatched (packets 1) for traffic egressing a non-uplink interface, and uplink traffic (egressenp5s0) was correctly DNAT'd while non-uplink traffic returned. Ifoifnamewere unset/empty,oifname != $nicswould always be true and nothing would ever be redirected — but redirection works. -
The suggested replacement
fib daddr oifnamedoes not even load in theoutputhook:Error: Could not process rule: Operation not supported. It only works inprerouting.
This asymmetry is intentional: in prerouting the packet is not yet routed, so a FIB lookup (fib daddr oifname) is needed to predict egress; in output the packet is already routed, so plain oifname is available (and fib daddr oifname is EOPNOTSUPP there). The current implementation is correct as written.
What this PR does
Update the aproxy transparent proxy nftables rules so they only redirect traffic leaving the runner's default-route ("physical") interfaces.
Why we need it
This can simplify the
aproxy-exclude-addressesconfiguration, as we don't need to include the private IP ranges by default, and we only need to include the actual IP addresses that we need to exclude. Also, this would help reduce the chances of IP range conflicts between internal services, like MicroK8s, Docker, and LXD, and external services within the infrastructure using private IP addresses.Checklist
docs/changelog.mdwith user-relevant changesterraform fmtpasses andtflintreports no errorsgithub-runner-manager/pyproject.toml.