Skip to content

🐛 Supervise background processes in AuthBridge entrypoints #308

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
huang195 merged 1 commit into from
Apr 14, 2026
Merged

🐛 Supervise background processes in AuthBridge entrypoints #308

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 43 additions & 17 deletions authbridge/authproxy/entrypoint-authbridge.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,42 +1,56 @@
#!/bin/sh
#!/bin/bash
set -eu

# AuthBridge combined entrypoint
# AuthBridge combined entrypoint with process supervision.
# Manages: spiffe-helper, client-registration, go-processor, envoy
#
# Startup order preserves current multi-container timing:
# 1. spiffe-helper (background, long-running) -- writes JWT SVID
# 2. go-processor (background) -- handles missing credentials via waitForCredentials
# 3. client-registration (background one-shot) -- writes credentials when ready
# 4. envoy (exec, foreground) -- inbound JWT validation works immediately
# 4. envoy (background) -- inbound JWT validation works immediately
#
# Runs as UID 1337 (Envoy UID, excluded from iptables redirect).
#
# Process management: Envoy (PID 1 via exec) is the liveness-critical process.
# Background subprocesses (spiffe-helper, go-processor) are expected to be
# long-running but their failures are not monitored here — Kubernetes liveness
# probes on the Envoy admin port detect the overall container health.
# Client-registration failures are non-fatal (go-processor handles missing
# credentials gracefully via passthrough mode).
# Process management: The shell stays as PID 1 and monitors critical long-running
# processes (envoy, go-processor, spiffe-helper). If any critical process exits,
# all others are killed and the container exits non-zero so Kubernetes restarts it.
# Client-registration is a one-shot process and is NOT monitored -- go-processor
# handles missing credentials gracefully via passthrough mode.

# PIDs of critical long-running processes to monitor.
CRITICAL_PIDS=""

cleanup() {
echo "[AuthBridge] Received signal, shutting down..."
# shellcheck disable=SC2086
kill $CRITICAL_PIDS 2>/dev/null || true
wait
exit 0
}
trap cleanup TERM INT

# --- Phase 1: Start spiffe-helper (if enabled) ---
if [ "${SPIRE_ENABLED}" = "true" ]; then
if [ "${SPIRE_ENABLED:-}" = "true" ]; then
echo "[AuthBridge] Starting spiffe-helper..."
/usr/local/bin/spiffe-helper -config=/etc/spiffe-helper/helper.conf run &
CRITICAL_PIDS="$CRITICAL_PIDS $!"
fi

# --- Phase 2: Start go-processor ---
# go-processor waits internally for credential files (waitForCredentials, 60s timeout).
# Inbound JWT validation works immediately (doesn't need credentials).
echo "[AuthBridge] Starting go-processor..."
/usr/local/bin/go-processor &
CRITICAL_PIDS="$CRITICAL_PIDS $!"
sleep 2

@cwiklik cwiklik Apr 14, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Clean pattern — CRITICAL_PIDS accumulation, wait -n, trap with cleanup. Client-registration correctly excluded from monitoring. The ${SPIRE_ENABLED:-} default is a good defensive fix too.


# --- Phase 3: Start client-registration (background, non-blocking) ---
# This runs asynchronously so Envoy starts immediately.
# Failures are non-fatal: go-processor handles missing credentials gracefully.
# NOT added to CRITICAL_PIDS -- this is a one-shot process.
(
if [ "${SPIRE_ENABLED}" = "true" ]; then
if [ "${SPIRE_ENABLED:-}" = "true" ]; then
echo "[AuthBridge] Waiting for SPIFFE credentials..."
while [ ! -f /opt/jwt_svid.token ]; do sleep 1; done
echo "[AuthBridge] SPIFFE credentials ready"
Expand All @@ -59,17 +73,29 @@ sleep 2
echo "[AuthBridge] Client ID: $CLIENT_NAME"
fi

if [ "${CLIENT_REGISTRATION_ENABLED}" != "false" ]; then
if [ "${CLIENT_REGISTRATION_ENABLED:-}" != "false" ]; then
echo "[AuthBridge] Starting client registration..."
python3 /app/client_registration.py || \
echo "[AuthBridge] WARNING: Client registration failed, continuing without"
echo "[AuthBridge] Client registration phase complete"
fi
) &

# --- Phase 4: Start Envoy (foreground) ---
# exec replaces this shell with Envoy (becomes PID 1).
# Kubernetes sends SIGTERM to PID 1 on termination.
# --- Phase 4: Start Envoy (background) ---
# Envoy runs in the background so the shell can monitor all critical processes.
# Kubernetes sends SIGTERM to PID 1 (this shell), which forwards via trap.
echo "[AuthBridge] Starting Envoy..."
exec /usr/local/bin/envoy -c /etc/envoy/envoy.yaml \
--service-cluster auth-proxy --service-node auth-proxy
/usr/local/bin/envoy -c /etc/envoy/envoy.yaml \
--service-cluster auth-proxy --service-node auth-proxy &
CRITICAL_PIDS="$CRITICAL_PIDS $!"

# Wait for the first critical process to exit.
# If any critical process dies, restart the container.
# shellcheck disable=SC2086
wait -n $CRITICAL_PIDS
EXIT_CODE=$?
echo "[AuthBridge] A critical process exited unexpectedly (exit code $EXIT_CODE), terminating container"
# shellcheck disable=SC2086
kill $CRITICAL_PIDS 2>/dev/null || true
wait
exit 1
32 changes: 28 additions & 4 deletions authbridge/authproxy/entrypoint-envoy.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,37 @@
#!/bin/sh
#!/bin/bash

@cwiklik cwiklik Apr 14, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

must-fix: #!/bin/bash + wait -n will fail at runtime. Dockerfile.envoy uses ubi9/ubi-minimal which only installs ca-certificates — no bash. The container will crash on startup.

Two options:

  1. Add microdnf install -y bash to Dockerfile.envoy
  2. Rewrite using POSIX sh (e.g., background both processes, poll with kill -0 in a loop)

Option 1 is simpler but adds ~5MB to the image. Option 2 keeps the minimal image but is more complex.

entrypoint-authbridge.sh is fine — Dockerfile.authbridge is based on envoyproxy/envoy (Debian), which includes bash.

@huang195 huang195 Apr 14, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ubi9/ubi-minimal ships with bash 5.1.8 out of the box — no microdnf install needed:

$ podman run --rm ubi9/ubi-minimal bash --version
GNU bash, version 5.1.8(1)-release (aarch64-redhat-linux-gnu)

Verified end-to-end on a Kind cluster: built the image, deployed the quickstart demo, confirmed both processes start under the supervision loop, killed go-processor inside the container, and watched Kubernetes restart it. No issues.

set -eu

# Start the go-processor in the background
# Envoy + go-processor entrypoint with process supervision.
# Both processes run in the background; the shell stays as PID 1.
# If either process exits, the other is killed and the container exits
# non-zero so Kubernetes restarts it.

cleanup() {
echo "Received signal, shutting down..."
kill "$GO_PROCESSOR_PID" "$ENVOY_PID" 2>/dev/null || true

@pdettori pdettori Apr 14, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

must-fix: GO_PROCESSOR_PID and ENVOY_PID are referenced here but not yet defined when the trap is installed (line 15). If SIGTERM arrives before both PID assignments (lines 20 and 29), set -u triggers a fatal unbound-variable error that bypasses || true — the error occurs at variable expansion, before kill runs.

Initialize both to empty before the trap, matching the CRITICAL_PIDS="" pattern in entrypoint-authbridge.sh:

GO_PROCESSOR_PID=""
ENVOY_PID=""

cleanup() {
  ...
}
trap cleanup TERM INT

wait
exit 0
}
trap cleanup TERM INT

# Start go-processor in the background
echo "Starting go-processor..."
/usr/local/bin/go-processor &
GO_PROCESSOR_PID=$!

# Give go-processor a moment to start
sleep 2

# Start Envoy in the foreground
# Start Envoy in the background (not exec — shell must stay PID 1 for supervision)
echo "Starting Envoy..."
exec /usr/local/bin/envoy -c /etc/envoy/envoy.yaml --service-cluster auth-proxy --service-node auth-proxy --log-level debug
/usr/local/bin/envoy -c /etc/envoy/envoy.yaml \
--service-cluster auth-proxy --service-node auth-proxy --log-level debug &
ENVOY_PID=$!

# Wait for the first child to exit. If either process dies, restart the container.
wait -n "$GO_PROCESSOR_PID" "$ENVOY_PID"
EXIT_CODE=$?
echo "A process exited unexpectedly (exit code $EXIT_CODE), terminating container"
kill "$GO_PROCESSOR_PID" "$ENVOY_PID" 2>/dev/null || true
wait
exit 1
Loading