Inter-SP communication setup

[redpanda-f]

Currently @rvagg 's work demands inter-SP (curio) communication for Piece replication. However, current setup separates each curio service provider into their own isolated network. This is great for security, but hard for inter-SP communications setup.

Figure out a way for multiple SPs to be able to interact with each other without editing hosts file on the machine. Partly this is attempted here in: #34

[rvagg]

fwiw what I did in #34 seemed to be the best option of the terrible options available and is also something done by others experiencing this same hassle. Making it work across Linux and macOS is also a bit of a hassle, it's possible to make it work on either one with other work-arounds, but not consistently between them, hence that solution as a consistent one.

[redpanda-f]

Inter-SP Communication Solution Analysis

TL;DR

Where we are: localhost:<port> doesn't work for inter-SP comms because localhost means different things inside vs. outside a container.

The fix: Use host.docker.internal — it resolves to 127.0.0.1 consistently from both the host and inside containers, letting SPs talk to each other while staying accessible from the host.

---

The Problem

So here's the thing: Docker networking makes this tricky. You've got SPs running in containers that need to:

1. Talk to each other (inter-SP communication)
2. Be reachable from the host (for testing/synapse)

But localhost doesn't work because it's context-dependent:

• On the host, localhost = the machine
• Inside a container, localhost = that container
• So when SP1 tries to hit localhost:8080 thinking it's reaching SP2, it's actually hitting itself

Docker DNS names (foc-curio-1, etc.) work great within containers but don't resolve from the host at all. Catch-22.

The Solution: host.docker.internal

Rod's approach (PR #34) is actually pretty elegant. Use host.docker.internal as the unified endpoint:

How it works:

• Add 127.0.0.1 host.docker.internal to /etc/hosts on the host (one-liner, macOS Docker Desktop does this automatically)
• Pass --add-host=host.docker.internal:host-gateway when launching Curio containers
• SPs register themselves in the registry as http://host.docker.internal:<port> instead of localhost:<port>
• Set CURIO_PULL_ALLOW_INSECURE=1 to bypass TLS cert validation for this internal hostname

Why it works:

• From the host: host.docker.internal resolves to 127.0.0.1 (via /etc/hosts)
• From containers: Docker's host-gateway mapping points it back to the host's IP
• Same hostname, same port, works everywhere — SP1 can reach SP2, and both can be reached from the host

The Good Stuff ✅

• Single endpoint — one URL works everywhere (host + all containers)
• Enables inter-SP comms — SPs can actually call each other
• Host still accessible — testing code on the host works fine
• Low overhead — literally just DNS + one Docker flag, no architectural changes

The Tradeoffs ⚠️

• Host-level modification — Need to edit /etc/hosts. Automated with clear instructions; Linux users do it once.
• CURIO_PULL_ALLOW_INSECURE=1 — Bypasses TLS validation. Fine for devnet; production would need proper certs.
• CI/CD setup — GitHub Actions et al. need the /etc/hosts line too (documented in error messages).
• Docker dependency — Relies on host-gateway being available (it's standard in modern Docker).

None of these are dealbreakers for a devnet tool. The setup friction is minimal and the benefit is immediate.

Validation & Safety

The code already checks that host.docker.internal resolves correctly before startup:

fn check_host_docker_internal() -> Result<(), Box<dyn std::error::Error>> {
    match "host.docker.internal:80".to_socket_addrs() {
        Ok(mut addrs) => {
            let is_localhost = addrs.any(|addr| addr.ip().is_loopback());
            if is_localhost { /* ✓ we're good */ }
            // Otherwise: friendly error + exact fix instructions
        }
    }
}

Catches the problem early with clear, copy-pasteable fix instructions for different platforms.

What's Actually Implemented

• Service URL registration: http://host.docker.internal:{port} ✓
• Container launch flag: --add-host=host.docker.internal:host-gateway ✓
• Env var: CURIO_PULL_ALLOW_INSECURE=1 ✓
• Pre-flight DNS check: validates setup before startup ✓