diff --git a/architecture/sandbox.md b/architecture/sandbox.md index d2cb44d7b8..f105b8a27e 100644 --- a/architecture/sandbox.md +++ b/architecture/sandbox.md @@ -103,6 +103,70 @@ handled by the inference interception path: External inference endpoints that do not use `inference.local` are treated like ordinary network traffic and must be allowed by policy. +In proxy-required networks, the supervisor chains upstream TLS tunnels through +a corporate forward proxy with HTTP CONNECT instead of connecting directly, +once policy and SSRF checks pass. Only TLS (CONNECT) egress is chained: +plain-HTTP requests always dial the destination directly, because forwarding +plain HTTP through a proxy requires absolute-form request forwarding rather +than CONNECT tunneling and is out of scope. The proxy configuration is an +operator-owned boundary delivered on the supervisor's command line +(`--upstream-proxy` and friends) by the compute driver; sandbox and template +environment — and `ENV` values baked into the sandbox image — cannot +influence it, since none of these can alter the argv the driver sets. The +conventional `HTTPS_PROXY`/`HTTP_PROXY`/`NO_PROXY` variables a sandbox +controls are ignored on this path. Operator `NO_PROXY` destinations and +loopback always dial directly; add driver-injected host aliases (e.g. +`host.containers.internal`) to the operator `NO_PROXY` list when the corporate +proxy cannot reach the container host. `NO_PROXY` matching is port-aware and +resolution-aware: an entry with a `:port` qualifier only bypasses that port, +and IP/CIDR entries also match hostnames through their validated resolved +addresses, with the direct dial limited to the addresses the entry contains. Only `http://` proxy URLs in explicit +`http://host:port` form are supported — the scheme and port are both +required, and a path, query, or fragment is rejected. Local DNS resolution +and SSRF validation still run before the proxied dial, and the CONNECT +target sent to the corporate proxy is a validated resolved address, so the +proxy performs no DNS resolution of its own and the tunnel stays bound to +the answer that passed SSRF and `allowed_ips` validation. The hostname still +travels inside the tunnel (TLS SNI, application `Host`). In split-horizon +networks, point the gateway host at the corporate resolver so internal names +validate to their internal addresses; the `proxy_connect_by_hostname` +opt-in exists as a +last resort for proxies whose ACLs filter on hostnames and reject IP CONNECT +targets — with it, the proxy resolves the name itself and its ACLs become +the effective egress control for proxied TLS. (Resolving through the proxy's +own DNS view, e.g. DoH tunneled via CONNECT, is a possible future +enhancement and out of scope.) The workload child's proxy variables are +unaffected — they are always rewritten to point at the local policy proxy. + +The configuration is fail-closed: a setting that is present but invalid — an +empty value, an unsupported or malformed proxy URL, an unreadable auth file, +a malformed credential, or an auth file or `NO_PROXY` list set while no proxy +URL is configured — is fatal to supervisor startup instead of being treated +as unset, so a misconfiguration can never silently degrade to direct dialing +or unauthenticated proxy access. Only an omitted argument means "no proxy". +The driver validates the same rules at sandbox-create time through +validators shared with the supervisor +(`openshell_core::driver_utils::parse_upstream_proxy_url` and +`parse_upstream_proxy_credential`). + +Proxy credentials are never embedded in the URL: an inline `user:pass@` is +rejected because it would be stored in `gateway.toml` and exposed in container +metadata. Operators supply credentials via `proxy_auth_file`; the driver +stages them as a root-only secret mounted at a fixed path and passes only +that path on the supervisor's command line. The supervisor reads the +file and builds the `Proxy-Authorization: Basic` header; a credential that is +empty, contains control characters, or is not in `user:pass` form is fatal on +both sides. + +The Basic header travels over the plain-TCP connection to the `http://` proxy, +so it is readable on the network path between sandbox host and proxy. +Configuring `proxy_auth_file` therefore requires the explicit opt-in +`proxy_auth_allow_insecure = true`. Both the +driver (at sandbox-create time) and the supervisor (at startup) reject an +auth file without the acknowledgement, and the acknowledgement without an +auth file, so credentials are never sent in cleartext without an explicit +operator decision. + ## Credentials Provider credentials are stored at the gateway and fetched by the supervisor at diff --git a/crates/openshell-core/src/driver_utils.rs b/crates/openshell-core/src/driver_utils.rs index 9e4411b2a1..7d7393b924 100644 --- a/crates/openshell-core/src/driver_utils.rs +++ b/crates/openshell-core/src/driver_utils.rs @@ -72,6 +72,221 @@ pub const TLS_KEY_MOUNT_PATH: &str = "/etc/openshell/tls/client/tls.key"; /// Container-side mount path for the per-sandbox JWT token. pub const SANDBOX_TOKEN_MOUNT_PATH: &str = "/etc/openshell/auth/sandbox.jwt"; +/// Container-side mount path for the corporate upstream-proxy credentials. +/// +/// The file holds the `user:pass` userinfo used to build the +/// `Proxy-Authorization` header. It is delivered through a root-only secret +/// mount so the credential never appears in container environment/metadata. +pub const UPSTREAM_PROXY_AUTH_MOUNT_PATH: &str = "/etc/openshell/auth/upstream-proxy"; + +/// A validated corporate upstream-proxy address. +/// +/// Produced by [`parse_upstream_proxy_url`], which is the single source of +/// truth for what counts as a valid upstream proxy URL. Compute drivers use +/// it to reject bad operator config at sandbox-create time, and the +/// in-container supervisor applies the same rules to its driver-supplied +/// arguments so a value one side accepts is never rejected (or silently +/// ignored) by the other. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct UpstreamProxyAddr { + /// Proxy hostname, IPv4, or IPv6 address (IPv6 without brackets). + pub host: String, + /// Proxy TCP port (always explicit in the accepted URL grammar). + pub port: u16, +} + +/// Why an upstream proxy URL was rejected by [`parse_upstream_proxy_url`]. +/// +/// Kept as a typed error so each consumer (driver config validation, +/// supervisor startup) can phrase the message for its own surface while +/// enforcing identical semantics. +#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)] +pub enum UpstreamProxyUrlError { + /// The value is empty or whitespace-only. + #[error("proxy URL is empty")] + Empty, + /// The value does not parse as a URL. + #[error("not a valid proxy URL: {0}")] + Invalid(url::ParseError), + /// The value has no `scheme://` prefix. Bare `host[:port]` forms are + /// rejected so the accepted grammar matches the documented + /// `http://host:port` contract exactly. + #[error("proxy URL must include an explicit scheme, e.g. http://proxy.corp.com:3128")] + MissingScheme, + /// The URL uses a scheme other than `http` (TLS and SOCKS proxies are + /// not supported by the sandbox supervisor). + #[error( + "unsupported proxy scheme '{0}': only http:// forward proxies are \ + supported by the sandbox supervisor" + )] + UnsupportedScheme(String), + /// The URL has no explicit port. Corporate proxies rarely listen on the + /// scheme default (80), so a forgotten port is rejected instead of + /// silently dialing port 80. + #[error("proxy URL must include an explicit proxy port, e.g. http://proxy.corp.com:3128")] + MissingPort, + /// The URL embeds `user:pass@` credentials, which would leak into config + /// and container metadata. Credentials must come from the proxy auth file. + #[error("proxy URL must not embed credentials; supply them via the proxy auth file")] + InlineCredentials, + /// The URL has no host component. + #[error("proxy URL is missing a proxy host")] + MissingHost, + /// The URL carries a path, query, or fragment. A forward proxy is + /// addressed by `host:port` only, so extra components indicate a + /// misconfiguration (e.g. a pasted endpoint URL) and are rejected instead + /// of being silently discarded. + #[error("proxy URL must not contain a {0}; use scheme://host:port only")] + UnexpectedComponent(&'static str), +} + +/// Parse and validate a corporate upstream-proxy URL. +/// +/// The accepted grammar is exactly `http://host:port`: the scheme and the +/// port must both be explicit, only `http://` proxies are accepted, and +/// inline userinfo is rejected. The URL must address the proxy only: a path +/// (other than a bare trailing `/`), query, or fragment is rejected rather +/// than silently discarded. +/// +/// # Errors +/// +/// Returns an [`UpstreamProxyUrlError`] describing the first rule the value +/// violates. +pub fn parse_upstream_proxy_url(raw: &str) -> Result { + let trimmed = raw.trim(); + if trimmed.is_empty() { + return Err(UpstreamProxyUrlError::Empty); + } + if !trimmed.contains("://") { + return Err(UpstreamProxyUrlError::MissingScheme); + } + let parsed = url::Url::parse(trimmed).map_err(UpstreamProxyUrlError::Invalid)?; + + if !parsed.scheme().eq_ignore_ascii_case("http") { + return Err(UpstreamProxyUrlError::UnsupportedScheme( + parsed.scheme().to_string(), + )); + } + if !parsed.username().is_empty() || parsed.password().is_some() { + return Err(UpstreamProxyUrlError::InlineCredentials); + } + let host = match parsed.host() { + // `Host::Ipv6` renders without brackets, which is what socket + // connect APIs expect. + Some(url::Host::Ipv6(ip)) => ip.to_string(), + Some(host) => host.to_string(), + None => return Err(UpstreamProxyUrlError::MissingHost), + }; + if host.is_empty() { + return Err(UpstreamProxyUrlError::MissingHost); + } + // The `url` crate normalizes an absent path to "/" for http URLs, so a + // bare trailing slash is indistinguishable from no path and is accepted. + if !matches!(parsed.path(), "" | "/") { + return Err(UpstreamProxyUrlError::UnexpectedComponent("path")); + } + if parsed.query().is_some() { + return Err(UpstreamProxyUrlError::UnexpectedComponent("query")); + } + if parsed.fragment().is_some() { + return Err(UpstreamProxyUrlError::UnexpectedComponent("fragment")); + } + if !authority_has_explicit_port(trimmed) { + return Err(UpstreamProxyUrlError::MissingPort); + } + Ok(UpstreamProxyAddr { + host, + // Explicit-port presence was verified above; `port()` is `None` only + // when the URL spells out the scheme default (`:80`), which the url + // crate normalizes away. + port: parsed.port().unwrap_or(80), + }) +} + +/// Return `true` when the raw URL's authority carries an explicit `:port`. +/// +/// The `url` crate normalizes a scheme-default port (`:80` for http) to +/// `None`, making it indistinguishable from an absent port in the parsed +/// form, so the raw authority must be inspected instead. +fn authority_has_explicit_port(raw: &str) -> bool { + let after_scheme = raw.split_once("://").map_or(raw, |(_, rest)| rest); + let authority_end = after_scheme + .find(['/', '?', '#']) + .unwrap_or(after_scheme.len()); + let authority = &after_scheme[..authority_end]; + // Userinfo is rejected by the caller, but strip it anyway so this check + // never misreads a `user:pass@` colon as a port. + let host_port = authority.rsplit_once('@').map_or(authority, |(_, hp)| hp); + host_port.rfind(']').map_or_else( + || { + host_port + .rsplit_once(':') + .is_some_and(|(_, port)| !port.is_empty()) + }, + // Bracketed IPv6 literal: a port can only follow the bracket, and a + // bare trailing `]:` is no more explicit than no port at all. + |end| { + host_port[end + 1..] + .strip_prefix(':') + .is_some_and(|port| !port.is_empty()) + }, + ) +} + +/// Why an upstream proxy credential was rejected by +/// [`parse_upstream_proxy_credential`]. +/// +/// Variants carry no payload so an error can never leak credential content +/// into logs or user-facing messages. +#[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)] +pub enum UpstreamProxyCredentialError { + /// The credential is empty or whitespace-only. + #[error("credential is empty")] + Empty, + /// The credential contains control characters (CR, LF, NUL, tab, ...) + /// that could inject additional HTTP headers. + #[error("credential contains control characters")] + ControlCharacters, + /// The credential has no `:` separating user from password. + #[error("credential must use the user:pass form (missing ':')")] + MissingSeparator, + /// The credential has an empty user before the `:` separator. + #[error("credential must use the user:pass form (empty user)")] + EmptyUser, +} + +/// Validate a corporate upstream-proxy credential read from the proxy auth +/// file, returning the trimmed `user:pass` value. +/// +/// Single source of truth for what counts as a valid proxy credential: the +/// compute driver applies it at sandbox-create time (before staging the +/// secret) and the in-container supervisor applies it again before building +/// the `Proxy-Authorization: Basic` header, so a credential one side accepts +/// is never rejected by the other. +/// +/// Surrounding whitespace (including the conventional trailing newline) is +/// trimmed. The user part must be non-empty; the password may be empty and +/// may itself contain `:` (per RFC 7617 the first `:` is the separator). +/// +/// # Errors +/// +/// Returns an [`UpstreamProxyCredentialError`] describing the first rule the +/// value violates. Errors never contain the credential itself. +pub fn parse_upstream_proxy_credential(raw: &str) -> Result<&str, UpstreamProxyCredentialError> { + let credential = raw.trim(); + if credential.is_empty() { + return Err(UpstreamProxyCredentialError::Empty); + } + if credential.contains(|c: char| c.is_control()) { + return Err(UpstreamProxyCredentialError::ControlCharacters); + } + match credential.split_once(':') { + None => Err(UpstreamProxyCredentialError::MissingSeparator), + Some(("", _)) => Err(UpstreamProxyCredentialError::EmptyUser), + Some(_) => Ok(credential), + } +} + /// Return the XDG state path for a driver's sandbox JWT token file. /// /// The resulting path is `$XDG_STATE_HOME/openshell/[/]//sandbox.jwt`. @@ -160,3 +375,157 @@ pub fn supervisor_image_tag(image: &str) -> Option<&str> { pub fn supervisor_image_should_refresh(image: &str) -> bool { matches!(supervisor_image_tag(image), Some("dev" | "latest")) } + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn upstream_proxy_url_accepts_http_with_port() { + let addr = parse_upstream_proxy_url("http://proxy.corp.com:8080").unwrap(); + assert_eq!(addr.host, "proxy.corp.com"); + assert_eq!(addr.port, 8080); + } + + #[test] + fn upstream_proxy_url_rejects_missing_scheme() { + for url in [ + "proxy.corp.com", + "proxy.corp.com:3128", + "user:pass@proxy.corp.com:8080", + ] { + assert_eq!( + parse_upstream_proxy_url(url), + Err(UpstreamProxyUrlError::MissingScheme), + "{url}" + ); + } + } + + #[test] + fn upstream_proxy_url_rejects_missing_port() { + for url in [ + "http://proxy.corp.com", + "http://proxy.corp.com/", + "http://proxy.corp.com:", + "http://[fd00::1]", + "http://[fd00::1]:", + ] { + assert_eq!( + parse_upstream_proxy_url(url), + Err(UpstreamProxyUrlError::MissingPort), + "{url}" + ); + } + // An explicit scheme-default port is accepted even though the url + // crate normalizes it away in the parsed form. + let addr = parse_upstream_proxy_url("http://proxy.corp.com:80").unwrap(); + assert_eq!(addr.port, 80); + } + + #[test] + fn upstream_proxy_url_ipv6_host_is_bracket_free() { + let addr = parse_upstream_proxy_url("http://[fd00::1]:8080").unwrap(); + assert_eq!(addr.host, "fd00::1"); + assert_eq!(addr.port, 8080); + } + + #[test] + fn upstream_proxy_url_rejects_tls_and_socks_schemes() { + for url in ["https://proxy:443", "socks5://proxy:1080"] { + assert!(matches!( + parse_upstream_proxy_url(url), + Err(UpstreamProxyUrlError::UnsupportedScheme(_)) + )); + } + } + + #[test] + fn upstream_proxy_url_rejects_inline_credentials() { + for url in ["http://user:pass@proxy:8080", "http://user@proxy:8080"] { + assert_eq!( + parse_upstream_proxy_url(url), + Err(UpstreamProxyUrlError::InlineCredentials) + ); + } + } + + #[test] + fn upstream_proxy_url_rejects_empty_and_invalid() { + assert_eq!( + parse_upstream_proxy_url(" "), + Err(UpstreamProxyUrlError::Empty) + ); + assert!(matches!( + parse_upstream_proxy_url("http://proxy:notaport"), + Err(UpstreamProxyUrlError::Invalid(_)) + )); + assert!(parse_upstream_proxy_url("http://").is_err()); + } + + #[test] + fn upstream_proxy_url_rejects_path_query_and_fragment() { + for (url, component) in [ + ("http://proxy.corp.com:8080/some/path", "path"), + ("http://proxy.corp.com:8080?x=1", "query"), + ("http://proxy.corp.com:8080/?x=1", "query"), + ("http://proxy.corp.com:8080#frag", "fragment"), + ] { + assert_eq!( + parse_upstream_proxy_url(url), + Err(UpstreamProxyUrlError::UnexpectedComponent(component)), + "{url}" + ); + } + // A bare trailing slash is URL normalization, not a real path. + let addr = parse_upstream_proxy_url("http://proxy.corp.com:8080/").unwrap(); + assert_eq!(addr.host, "proxy.corp.com"); + assert_eq!(addr.port, 8080); + } + + #[test] + fn upstream_proxy_credential_accepts_user_pass_and_trims() { + assert_eq!( + parse_upstream_proxy_credential("user:pass\n"), + Ok("user:pass") + ); + // The password may be empty and may contain further colons. + assert_eq!(parse_upstream_proxy_credential("user:"), Ok("user:")); + assert_eq!( + parse_upstream_proxy_credential("user:p@:ss"), + Ok("user:p@:ss") + ); + } + + #[test] + fn upstream_proxy_credential_rejects_empty() { + for raw in ["", " ", "\n"] { + assert_eq!( + parse_upstream_proxy_credential(raw), + Err(UpstreamProxyCredentialError::Empty) + ); + } + } + + #[test] + fn upstream_proxy_credential_rejects_control_characters() { + for raw in ["user:pa\r\nss", "user:pa\0ss", "user:pa\tss"] { + assert_eq!( + parse_upstream_proxy_credential(raw), + Err(UpstreamProxyCredentialError::ControlCharacters) + ); + } + } + + #[test] + fn upstream_proxy_credential_rejects_malformed_user_pass_form() { + assert_eq!( + parse_upstream_proxy_credential("userpass"), + Err(UpstreamProxyCredentialError::MissingSeparator) + ); + assert_eq!( + parse_upstream_proxy_credential(":pass"), + Err(UpstreamProxyCredentialError::EmptyUser) + ); + } +} diff --git a/crates/openshell-core/src/sandbox_env.rs b/crates/openshell-core/src/sandbox_env.rs index f066580636..f15ce34bb1 100644 --- a/crates/openshell-core/src/sandbox_env.rs +++ b/crates/openshell-core/src/sandbox_env.rs @@ -114,3 +114,8 @@ pub const SANDBOX_UID: &str = "OPENSHELL_SANDBOX_UID"; /// Used alongside UID for PVC init container `chown` operations and when the /// supervisor drops privileges to a group other than the UID's primary group. pub const SANDBOX_GID: &str = "OPENSHELL_SANDBOX_GID"; + +// The corporate upstream-proxy configuration deliberately has no reserved +// environment variables: it travels on the supervisor's argv +// (`--upstream-proxy` and friends), which a sandbox image cannot forge the +// way it could bake `ENV` values. diff --git a/crates/openshell-driver-podman/README.md b/crates/openshell-driver-podman/README.md index c6a619f3d1..25185baf31 100644 --- a/crates/openshell-driver-podman/README.md +++ b/crates/openshell-driver-podman/README.md @@ -345,6 +345,38 @@ Podman resources after out-of-band container removal or label drift. | `OPENSHELL_PODMAN_TLS_CA` | `--podman-tls-ca` | unset | Host path to the CA certificate mounted for sandbox mTLS. | | `OPENSHELL_PODMAN_TLS_CERT` | `--podman-tls-cert` | unset | Host path to the client certificate mounted for sandbox mTLS. | | `OPENSHELL_PODMAN_TLS_KEY` | `--podman-tls-key` | unset | Host path to the client private key mounted for sandbox mTLS. | +| `OPENSHELL_SANDBOX_HTTPS_PROXY` | `--sandbox-https-proxy` | unset | Corporate forward proxy URL for the supervisor's upstream TLS dials, chained with HTTP CONNECT. Only credential-free `http://host:port` URLs are supported (scheme and port required). Plain-HTTP requests always dial directly. | +| `OPENSHELL_SANDBOX_NO_PROXY` | `--sandbox-no-proxy` | unset | Comma-separated `NO_PROXY` list (hostnames, domain suffixes, IPs, CIDRs, each with an optional `:port` qualifier) dialed directly instead of through the corporate proxy. IP/CIDR entries also match hostnames through their validated DNS resolution. | +| `OPENSHELL_SANDBOX_PROXY_AUTH_FILE` | `--sandbox-proxy-auth-file` | unset | Path to a file containing the proxy credentials as `user:pass`. Staged as a root-only Podman secret so credentials never appear in config or container metadata. Requires the insecure-auth acknowledgement below. | +| `OPENSHELL_SANDBOX_PROXY_AUTH_ALLOW_INSECURE` | `--sandbox-proxy-auth-allow-insecure` | unset | Explicit acknowledgement (`true`) that the credential is sent as cleartext Basic auth over the plain-TCP connection to the `http://` proxy. Required when the auth file is set; rejected when it is not. | +| `OPENSHELL_SANDBOX_PROXY_CONNECT_BY_HOSTNAME` | `--sandbox-proxy-connect-by-hostname` | unset | Send the destination hostname in CONNECT requests instead of a validated IP. Last resort for proxies whose ACLs filter on hostnames: the proxy then resolves the name itself, so sandbox SSRF/`allowed_ips` validation no longer binds the connection. | + +Through the gateway, the same settings are the `https_proxy`, `no_proxy`, +`proxy_auth_file`, `proxy_auth_allow_insecure`, and +`proxy_connect_by_hostname` keys under `[openshell.drivers.podman]`; see +`docs/reference/gateway-config.mdx`. + +This is an operator-owned egress boundary: the driver passes the settings on +the supervisor's command line, so sandbox and template environment — and any +`ENV` baked into the sandbox image — cannot override them, and the +conventional `HTTPS_PROXY`/`HTTP_PROXY`/`NO_PROXY` variables a sandbox +controls do not steer it. Credentials must be supplied through +`proxy_auth_file`; an inline `user:pass@` in the URL is rejected at startup. + +Basic auth over an `http://` proxy is cleartext on the wire: anyone on the +network path between the sandbox host and the proxy can recover the +credential. Setting `proxy_auth_file` therefore requires +`proxy_auth_allow_insecure = true`; both the driver and the in-container +supervisor reject credentials without that explicit acknowledgement. + +CONNECT requests target a validated resolved IP by default, so the proxy +performs no DNS resolution and the tunnel stays bound to the address that +passed the sandbox's SSRF and `allowed_ips` checks; the hostname still +travels inside the tunnel (TLS SNI, application `Host`). In split-horizon +networks, point the gateway host at the corporate resolver. Set +`proxy_connect_by_hostname = true` only when the proxy's ACLs filter on +hostnames and reject IP CONNECT targets — it re-opens proxy-side DNS +resolution, making the proxy's ACLs the effective egress control. ## Rootless-Specific Adaptations diff --git a/crates/openshell-driver-podman/src/config.rs b/crates/openshell-driver-podman/src/config.rs index def8e5f3dc..b528b5011f 100644 --- a/crates/openshell-driver-podman/src/config.rs +++ b/crates/openshell-driver-podman/src/config.rs @@ -134,6 +134,57 @@ pub struct PodmanComputeConfig { /// Set to `0` to disable health checks entirely. /// Defaults to [`DEFAULT_HEALTH_CHECK_INTERVAL_SECS`] (10 seconds). pub health_check_interval_secs: u64, + /// Corporate forward proxy URL passed to the in-container supervisor + /// (e.g. `http://proxy.corp.com:8080`). + /// + /// The supervisor chains policy-approved TLS tunnels through this proxy + /// with HTTP CONNECT instead of dialing upstream destinations directly. + /// Only `http://` proxy URLs in explicit `http://host:port` form (scheme + /// and port required) are supported. This is an operator-owned egress + /// boundary delivered on the supervisor's command line, so + /// sandbox/template environment cannot override it, and the conventional + /// `HTTPS_PROXY` variables are not used. + pub https_proxy: Option, + /// Comma-separated `NO_PROXY` list passed alongside the proxy URL (e.g. + /// `*.svc.cluster.local,10.0.0.0/8`). Destinations matching an entry are + /// dialed directly instead of through the corporate proxy. Entries take + /// an optional `:port` qualifier that limits them to that destination + /// port, and IP/CIDR entries also match hostnames through their + /// validated DNS resolution. + pub no_proxy: Option, + /// Path (on the gateway host) to a file containing the corporate proxy + /// credentials as `user:pass`. + /// + /// Credentials must be supplied through this file, never embedded in the + /// proxy URL: an inline `user:pass@` in `https_proxy` is + /// rejected at startup because it would leak into `gateway.toml` and + /// container metadata. The gateway reads this file at sandbox-create time + /// and delivers it to the supervisor through a root-only secret mount. + pub proxy_auth_file: Option, + /// Explicit acknowledgement that proxy credentials are sent in cleartext. + /// + /// `Proxy-Authorization: Basic` is base64, not encryption, and the + /// connection to an `http://` corporate proxy is plain TCP, so anyone on + /// the network path between the sandbox host and the proxy can recover + /// the credential. Setting `proxy_auth_file` therefore requires + /// `proxy_auth_allow_insecure = true`; without it the configuration is + /// rejected at startup. Set it only when the path to the proxy is a + /// trusted network segment. + pub proxy_auth_allow_insecure: Option, + /// Send the destination *hostname* in CONNECT requests to the corporate + /// proxy instead of a validated IP. + /// + /// By default the supervisor CONNECTs to an address that already passed + /// SSRF and `allowed_ips` validation, so the proxy performs no DNS + /// resolution and the tunnel stays bound to the validated answer. Set + /// this to `true` only when the proxy's ACLs filter on hostnames and + /// reject IP CONNECT targets: the proxy then resolves the name itself, + /// so a name that resolves differently at the proxy (split-horizon DNS, + /// rebinding) can reach destinations the sandbox policy never approved, + /// and the proxy's own ACLs become the effective egress control. Prefer + /// pointing the gateway host at the corporate resolver so validated-IP + /// CONNECT works in split-horizon networks. + pub proxy_connect_by_hostname: Option, } pub const DEFAULT_HEALTH_CHECK_INTERVAL_SECS: u64 = 10; @@ -189,6 +240,94 @@ impl PodmanComputeConfig { Ok(()) } + /// Validate optional corporate proxy configuration. + /// + /// Shares validation semantics with the in-container supervisor through + /// [`openshell_core::driver_utils::parse_upstream_proxy_url`], so a value + /// accepted here can never be rejected by the supervisor at sandbox + /// startup (or vice versa). The supervisor only supports `http://` + /// forward proxies, so other schemes are rejected at config time instead + /// of failing inside every sandbox. Credentials must be supplied through + /// `proxy_auth_file`; an inline `user:pass@` in the URL is rejected + /// because it would otherwise be stored in `gateway.toml` and exposed in + /// container metadata. + pub fn validate_proxy_config(&self) -> Result<(), crate::client::PodmanApiError> { + use openshell_core::driver_utils::{UpstreamProxyUrlError, parse_upstream_proxy_url}; + if let Some(url) = &self.https_proxy { + parse_upstream_proxy_url(url).map_err(|err| { + crate::client::PodmanApiError::InvalidInput(match err { + UpstreamProxyUrlError::Empty => { + "https_proxy must not be empty when set".to_string() + } + UpstreamProxyUrlError::InlineCredentials => { + "https_proxy must not embed credentials in the URL; supply them via \ + proxy_auth_file so they are not stored in config or container metadata" + .to_string() + } + err => format!("https_proxy {err}"), + }) + })?; + } + + // The supervisor treats a present-but-empty reserved variable as a + // fatal misconfiguration, so never accept (and later inject) one. + if let Some(list) = self.no_proxy.as_deref() { + if list.trim().is_empty() { + return Err(crate::client::PodmanApiError::InvalidInput( + "no_proxy must not be empty when set; omit it instead".to_string(), + )); + } + // A bypass list only makes sense relative to a proxy boundary. An + // operator who set one believed proxying was in effect, so accepting + // it while all egress dials directly would hide a fail-open state. + if self.https_proxy.is_none() { + return Err(crate::client::PodmanApiError::InvalidInput( + "no_proxy is set but no https_proxy is configured".to_string(), + )); + } + } + + if let Some(path) = self.proxy_auth_file.as_deref() { + if path.trim().is_empty() { + return Err(crate::client::PodmanApiError::InvalidInput( + "proxy_auth_file must not be empty when set".to_string(), + )); + } + if self.https_proxy.is_none() { + return Err(crate::client::PodmanApiError::InvalidInput( + "proxy_auth_file is set but no https_proxy is configured".to_string(), + )); + } + // Basic auth over the plain-TCP proxy connection is readable by + // anyone on the network path; sending it requires an explicit + // operator acknowledgement rather than being an implicit side + // effect of configuring credentials. + if self.proxy_auth_allow_insecure != Some(true) { + return Err(crate::client::PodmanApiError::InvalidInput( + "proxy_auth_file sends the credential as cleartext Basic auth over the \ + plain-TCP connection to the http:// proxy; set proxy_auth_allow_insecure \ + = true to accept that exposure, or remove proxy_auth_file" + .to_string(), + )); + } + } else if self.proxy_auth_allow_insecure.is_some() { + // The acknowledgement without credentials means the operator + // believed an auth file was configured; surface the mismatch. + return Err(crate::client::PodmanApiError::InvalidInput( + "proxy_auth_allow_insecure is set but no proxy_auth_file is configured".to_string(), + )); + } + + // The CONNECT-target mode only means something relative to a proxy + // boundary the operator believed was in effect. + if self.proxy_connect_by_hostname.is_some() && self.https_proxy.is_none() { + return Err(crate::client::PodmanApiError::InvalidInput( + "proxy_connect_by_hostname is set but no https_proxy is configured".to_string(), + )); + } + Ok(()) + } + /// Validate optional host gateway override. pub fn validate_host_gateway_ip(&self) -> Result<(), crate::client::PodmanApiError> { let trimmed = self.host_gateway_ip.trim(); @@ -262,6 +401,11 @@ impl Default for PodmanComputeConfig { sandbox_pids_limit: DEFAULT_SANDBOX_PIDS_LIMIT, enable_bind_mounts: false, health_check_interval_secs: DEFAULT_HEALTH_CHECK_INTERVAL_SECS, + https_proxy: None, + no_proxy: None, + proxy_auth_file: None, + proxy_auth_allow_insecure: None, + proxy_connect_by_hostname: None, } } } @@ -288,6 +432,12 @@ impl std::fmt::Debug for PodmanComputeConfig { "health_check_interval_secs", &self.health_check_interval_secs, ) + // Proxy URLs may embed credentials in userinfo; log presence only. + .field("https_proxy", &self.https_proxy.is_some()) + .field("no_proxy", &self.no_proxy) + .field("proxy_auth_file", &self.proxy_auth_file.is_some()) + .field("proxy_auth_allow_insecure", &self.proxy_auth_allow_insecure) + .field("proxy_connect_by_hostname", &self.proxy_connect_by_hostname) .finish() } } @@ -397,6 +547,200 @@ mod tests { }); } + // ── Proxy config validation ─────────────────────────────────────── + + #[test] + fn validate_proxy_config_accepts_unset_and_http() { + assert!( + PodmanComputeConfig::default() + .validate_proxy_config() + .is_ok() + ); + let cfg = PodmanComputeConfig { + https_proxy: Some("http://proxy.corp.com:8080".to_string()), + no_proxy: Some("*.svc.cluster.local".to_string()), + ..PodmanComputeConfig::default() + }; + assert!(cfg.validate_proxy_config().is_ok()); + } + + #[test] + fn validate_proxy_config_rejects_non_http_schemes() { + for url in ["https://proxy:443", "socks5://proxy:1080"] { + let cfg = PodmanComputeConfig { + https_proxy: Some(url.to_string()), + ..PodmanComputeConfig::default() + }; + let err = cfg.validate_proxy_config().unwrap_err(); + assert!( + err.to_string().contains("unsupported proxy scheme"), + "{url}: {err}" + ); + } + } + + #[test] + fn validate_proxy_config_rejects_url_components() { + for url in [ + "http://proxy.corp.com:8080/path", + "http://proxy.corp.com:8080?x=1", + "http://proxy.corp.com:8080#frag", + ] { + let cfg = PodmanComputeConfig { + https_proxy: Some(url.to_string()), + ..PodmanComputeConfig::default() + }; + let err = cfg.validate_proxy_config().unwrap_err(); + assert!( + err.to_string().contains("scheme://host:port"), + "{url}: {err}" + ); + } + } + + #[test] + fn validate_proxy_config_rejects_missing_scheme_or_port() { + // A scheme-less value (previously normalized to http://) and a + // port-less value (previously defaulted to 80) are both rejected so + // gateway.toml matches the documented http://host:port grammar. + for url in ["proxy.corp.com:8080", "http://proxy.corp.com"] { + let cfg = PodmanComputeConfig { + https_proxy: Some(url.to_string()), + ..PodmanComputeConfig::default() + }; + let err = cfg.validate_proxy_config().unwrap_err(); + assert!(err.to_string().contains("explicit"), "{url}: {err}"); + } + } + + #[test] + fn validate_proxy_config_rejects_empty_value() { + let cfg = PodmanComputeConfig { + https_proxy: Some(" ".to_string()), + ..PodmanComputeConfig::default() + }; + let err = cfg.validate_proxy_config().unwrap_err(); + assert!(err.to_string().contains("https_proxy"), "{err}"); + } + + #[test] + fn validate_proxy_config_rejects_empty_no_proxy() { + let cfg = PodmanComputeConfig { + https_proxy: Some("http://proxy.corp.com:8080".to_string()), + no_proxy: Some(" ".to_string()), + ..PodmanComputeConfig::default() + }; + let err = cfg.validate_proxy_config().unwrap_err(); + assert!(err.to_string().contains("no_proxy"), "{err}"); + } + + #[test] + fn validate_proxy_config_rejects_no_proxy_without_proxy() { + let cfg = PodmanComputeConfig { + no_proxy: Some("*.svc.cluster.local".to_string()), + ..PodmanComputeConfig::default() + }; + let err = cfg.validate_proxy_config().unwrap_err(); + assert!(err.to_string().contains("no_proxy"), "{err}"); + } + + #[test] + fn validate_proxy_config_rejects_inline_credentials() { + for url in [ + "http://user:pass@proxy.corp.com:8080", + "http://user@proxy.corp.com:8080", + ] { + let cfg = PodmanComputeConfig { + https_proxy: Some(url.to_string()), + ..PodmanComputeConfig::default() + }; + let err = cfg.validate_proxy_config().unwrap_err(); + assert!( + err.to_string().contains("proxy_auth_file"), + "{url} should be rejected and point at proxy_auth_file: {err}" + ); + } + } + + #[test] + fn validate_proxy_config_accepts_auth_file_with_proxy_and_acknowledgement() { + let cfg = PodmanComputeConfig { + https_proxy: Some("http://proxy.corp.com:8080".to_string()), + proxy_auth_file: Some("/etc/openshell/secrets/proxy-auth".to_string()), + proxy_auth_allow_insecure: Some(true), + ..PodmanComputeConfig::default() + }; + assert!(cfg.validate_proxy_config().is_ok()); + } + + #[test] + fn validate_proxy_config_rejects_auth_file_without_insecure_acknowledgement() { + // Basic auth over the plain-TCP proxy connection is readable on the + // network path; sending it must be an explicit operator decision. + for allow in [None, Some(false)] { + let cfg = PodmanComputeConfig { + https_proxy: Some("http://proxy.corp.com:8080".to_string()), + proxy_auth_file: Some("/etc/openshell/secrets/proxy-auth".to_string()), + proxy_auth_allow_insecure: allow, + ..PodmanComputeConfig::default() + }; + let err = cfg.validate_proxy_config().unwrap_err(); + assert!( + err.to_string().contains("proxy_auth_allow_insecure"), + "{allow:?}: {err}" + ); + assert!(err.to_string().contains("cleartext"), "{allow:?}: {err}"); + } + } + + #[test] + fn validate_proxy_config_accepts_connect_by_hostname_with_proxy() { + for by_hostname in [Some(true), Some(false)] { + let cfg = PodmanComputeConfig { + https_proxy: Some("http://proxy.corp.com:8080".to_string()), + proxy_connect_by_hostname: by_hostname, + ..PodmanComputeConfig::default() + }; + assert!(cfg.validate_proxy_config().is_ok(), "{by_hostname:?}"); + } + } + + #[test] + fn validate_proxy_config_rejects_connect_by_hostname_without_proxy() { + let cfg = PodmanComputeConfig { + proxy_connect_by_hostname: Some(true), + ..PodmanComputeConfig::default() + }; + let err = cfg.validate_proxy_config().unwrap_err(); + assert!(err.to_string().contains("no https_proxy"), "{err}"); + } + + #[test] + fn validate_proxy_config_rejects_acknowledgement_without_auth_file() { + for allow in [Some(true), Some(false)] { + let cfg = PodmanComputeConfig { + https_proxy: Some("http://proxy.corp.com:8080".to_string()), + proxy_auth_allow_insecure: allow, + ..PodmanComputeConfig::default() + }; + let err = cfg.validate_proxy_config().unwrap_err(); + assert!( + err.to_string().contains("no proxy_auth_file"), + "{allow:?}: {err}" + ); + } + } + + #[test] + fn validate_proxy_config_rejects_auth_file_without_proxy() { + let cfg = PodmanComputeConfig { + proxy_auth_file: Some("/etc/openshell/secrets/proxy-auth".to_string()), + ..PodmanComputeConfig::default() + }; + let err = cfg.validate_proxy_config().unwrap_err(); + assert!(err.to_string().contains("proxy_auth_file"), "{err}"); + } + // ── TLS config validation ───────────────────────────────────────── #[test] diff --git a/crates/openshell-driver-podman/src/container.rs b/crates/openshell-driver-podman/src/container.rs index 9ee26c0735..1d09bef162 100644 --- a/crates/openshell-driver-podman/src/container.rs +++ b/crates/openshell-driver-podman/src/container.rs @@ -55,12 +55,15 @@ const VOLUME_PREFIX: &str = "openshell-sandbox-"; /// Secret name prefix for per-sandbox gateway JWTs. const TOKEN_SECRET_PREFIX: &str = "openshell-token-"; +const PROXY_AUTH_SECRET_PREFIX: &str = "openshell-proxy-auth-"; /// Container-side mount paths for client TLS materials and the sandbox token. const TLS_CA_MOUNT_PATH: &str = openshell_core::driver_utils::TLS_CA_MOUNT_PATH; const TLS_CERT_MOUNT_PATH: &str = openshell_core::driver_utils::TLS_CERT_MOUNT_PATH; const TLS_KEY_MOUNT_PATH: &str = openshell_core::driver_utils::TLS_KEY_MOUNT_PATH; const SANDBOX_TOKEN_MOUNT_PATH: &str = openshell_core::driver_utils::SANDBOX_TOKEN_MOUNT_PATH; +const UPSTREAM_PROXY_AUTH_MOUNT_PATH: &str = + openshell_core::driver_utils::UPSTREAM_PROXY_AUTH_MOUNT_PATH; /// Directory inside sandbox containers where the supervisor binary is mounted. const SUPERVISOR_MOUNT_DIR: &str = openshell_core::driver_utils::SUPERVISOR_CONTAINER_DIR; @@ -162,6 +165,12 @@ pub fn token_secret_name(sandbox_id: &str) -> String { format!("{TOKEN_SECRET_PREFIX}{sandbox_id}") } +/// Build the per-sandbox Podman secret name for the corporate proxy credentials. +#[must_use] +pub fn proxy_auth_secret_name(sandbox_id: &str) -> String { + format!("{PROXY_AUTH_SECRET_PREFIX}{sandbox_id}") +} + /// Truncate a container ID to 12 characters (standard short form). #[must_use] pub fn short_id(id: &str) -> String { @@ -349,6 +358,41 @@ pub fn resolve_image<'a>(sandbox: &'a DriverSandbox, config: &'a PodmanComputeCo /// User-supplied vars are inserted first so that the required driver /// vars always win -- preventing spec/template overrides of security- /// critical values like `OPENSHELL_ENDPOINT` or `OPENSHELL_SANDBOX_ID`. +/// Build the corporate upstream-proxy command-line arguments passed to the +/// supervisor. +/// +/// This operator-owned egress boundary travels on argv, which sandbox +/// spec/template environment and image `ENV` cannot influence. Credentials +/// are never on argv — only the root-only mount path is passed; the +/// supervisor reads the secret from the mount. +fn upstream_proxy_cli_args(config: &PodmanComputeConfig) -> Vec { + let mut args = Vec::new(); + if let Some(url) = &config.https_proxy { + args.push("--upstream-proxy".to_string()); + args.push(url.clone()); + } + if let Some(list) = &config.no_proxy { + args.push("--upstream-no-proxy".to_string()); + args.push(list.clone()); + } + if config.proxy_auth_file.is_some() { + args.push("--upstream-proxy-auth-file".to_string()); + args.push(UPSTREAM_PROXY_AUTH_MOUNT_PATH.to_string()); + } + // Config validation guarantees the acknowledgement is `true` whenever an + // auth file is configured; the supervisor independently refuses + // credentials without it. + if config.proxy_auth_allow_insecure == Some(true) { + args.push("--upstream-proxy-auth-allow-insecure".to_string()); + } + // Absent means the default validated-IP CONNECT binding; only the + // explicit hostname opt-in is passed through. + if config.proxy_connect_by_hostname == Some(true) { + args.push("--upstream-proxy-connect-by-hostname".to_string()); + } + args +} + fn build_env( sandbox: &DriverSandbox, config: &PodmanComputeConfig, @@ -386,6 +430,12 @@ fn build_env( } // 2. Required driver vars (highest priority -- always overwrite). + + // The operator's corporate egress proxy settings are not environment + // variables: they travel on the supervisor's argv (see + // `upstream_proxy_cli_args`), which sandbox spec/template environment + // and image ENV cannot influence. + env.insert( openshell_core::sandbox_env::SANDBOX.into(), sandbox.name.clone(), @@ -899,7 +949,10 @@ pub fn build_container_spec_with_token_and_gpu_devices( // Without this, the container would run the entrypoint binary with // the supervisor path as an argument instead of executing it directly. entrypoint: vec![SUPERVISOR_BINARY_PATH.into()], - command: vec![], + // Operator-owned corporate proxy flags. The workload command is not + // part of argv (the supervisor takes it from the reserved command + // env var), so these flags are the whole command list. + command: upstream_proxy_cli_args(config), // Force the supervisor to run as root (UID 0). Sandbox images may // set a non-root USER directive (e.g. `USER sandbox`), but the // supervisor needs root to create network namespaces, set up the @@ -993,15 +1046,32 @@ pub fn build_container_spec_with_token_and_gpu_devices( }, resource_limits, secret_env: BTreeMap::new(), - secrets: token_secret_name.map_or_else(Vec::new, |source| { - vec![SecretMount { - source: source.to_string(), - target: SANDBOX_TOKEN_MOUNT_PATH.into(), - uid: 0, - gid: 0, - mode: 0o400, - }] - }), + secrets: { + let mut secrets = Vec::new(); + if let Some(source) = token_secret_name { + secrets.push(SecretMount { + source: source.to_string(), + target: SANDBOX_TOKEN_MOUNT_PATH.into(), + uid: 0, + gid: 0, + mode: 0o400, + }); + } + // Corporate proxy credentials, when configured, are mounted as a + // root-only secret. The driver creates a matching Podman secret + // (see `create_sandbox_proxy_auth_secret`) named deterministically + // from the sandbox id, so no name needs threading through here. + if config.proxy_auth_file.is_some() { + secrets.push(SecretMount { + source: proxy_auth_secret_name(&sandbox.id), + target: UPSTREAM_PROXY_AUTH_MOUNT_PATH.into(), + uid: 0, + gid: 0, + mode: 0o400, + }); + } + secrets + }, stop_timeout: config.stop_timeout_secs, // Inject stable host aliases into /etc/hosts so sandbox containers can // reach services on the host. `host.openshell.internal` is the driver- @@ -1646,6 +1716,125 @@ mod tests { ); } + /// Extract the container spec's supervisor argv (`command`) as strings. + fn spec_command(spec: &Value) -> Vec { + spec["command"] + .as_array() + .expect("command should be an array") + .iter() + .map(|v| { + v.as_str() + .expect("command arg should be a string") + .to_string() + }) + .collect() + } + + #[test] + fn container_spec_passes_operator_proxy_on_supervisor_argv() { + let sandbox = test_sandbox("test-id", "test-name"); + let mut config = test_config(); + config.https_proxy = Some("http://proxy.corp.com:8080".to_string()); + config.no_proxy = Some("*.svc.cluster.local,10.0.0.0/8".to_string()); + + let spec = build_container_spec(&sandbox, &config); + let command = spec_command(&spec); + + // Config travels on argv (the image cannot forge process arguments), + // as flag/value pairs. + let idx = command + .iter() + .position(|a| a == "--upstream-proxy") + .expect("proxy URL flag present"); + assert_eq!( + command.get(idx + 1).map(String::as_str), + Some("http://proxy.corp.com:8080") + ); + let idx = command + .iter() + .position(|a| a == "--upstream-no-proxy") + .expect("no_proxy flag present"); + assert_eq!( + command.get(idx + 1).map(String::as_str), + Some("*.svc.cluster.local,10.0.0.0/8") + ); + + // The proxy settings are argv-only; nothing about them lands in the + // environment, and the conventional proxy variables (which belong to + // the sandbox creator) are not touched by operator config. + let env_map = spec["env"].as_object().expect("env should be an object"); + for key in ["HTTPS_PROXY", "https_proxy", "HTTP_PROXY", "http_proxy"] { + assert!( + !env_map.contains_key(key), + "{key} must not be populated from operator proxy config" + ); + } + } + + #[test] + fn container_spec_omits_proxy_argv_when_unconfigured() { + let sandbox = test_sandbox("test-id", "test-name"); + let spec = build_container_spec(&sandbox, &test_config()); + let command = spec_command(&spec); + + assert!( + !command.iter().any(|a| a.starts_with("--upstream-proxy")), + "no proxy flags without operator proxy config: {command:?}" + ); + } + + #[test] + fn container_spec_sandbox_env_cannot_influence_proxy_argv() { + use openshell_core::proto::compute::v1::{DriverSandboxSpec, DriverSandboxTemplate}; + + // A sandbox creator tries to steer the egress boundary through spec + // and template environment (image-baked ENV behaves the same at the + // runtime layer). The supervisor takes proxy config only from the + // argv the driver builds out of operator config, so none of it has + // any effect. + let mut sandbox = test_sandbox("test-id", "test-name"); + sandbox.spec = Some(DriverSandboxSpec { + environment: std::collections::HashMap::from([ + ( + "HTTPS_PROXY".to_string(), + "http://attacker:9999".to_string(), + ), + ("NO_PROXY".to_string(), "*".to_string()), + ]), + template: Some(DriverSandboxTemplate { + environment: std::collections::HashMap::from([( + "NO_PROXY".to_string(), + "*".to_string(), + )]), + ..Default::default() + }), + ..Default::default() + }); + let mut config = test_config(); + config.https_proxy = Some("http://proxy.corp.com:8080".to_string()); + + let spec = build_container_spec(&sandbox, &config); + let command = spec_command(&spec); + + // Only the operator's proxy is delivered, and only on argv. + let idx = command + .iter() + .position(|a| a == "--upstream-proxy") + .expect("operator proxy flag present"); + assert_eq!( + command.get(idx + 1).map(String::as_str), + Some("http://proxy.corp.com:8080") + ); + assert!( + !command.iter().any(|a| a == "--upstream-no-proxy"), + "sandbox environment must not add a NO_PROXY bypass: {command:?}" + ); + assert!( + !command.iter().any(|a| a.contains("attacker")), + "attacker proxy must not reach argv: {command:?}" + ); + } + #[test] fn container_spec_required_labels_cannot_be_overridden() { use openshell_core::proto::compute::v1::{DriverSandboxSpec, DriverSandboxTemplate}; @@ -2380,6 +2569,104 @@ mod tests { ); } + #[test] + fn container_spec_proxy_auth_file_mounts_secret_and_sets_path_only() { + let sandbox = test_sandbox("proxy-id", "proxy-name"); + let mut config = test_config(); + config.https_proxy = Some("http://proxy.corp.com:8080".to_string()); + config.proxy_auth_file = Some("/etc/openshell/secrets/proxy-auth".to_string()); + config.proxy_auth_allow_insecure = Some(true); + + let spec = build_container_spec(&sandbox, &config); + let command = spec_command(&spec); + + // The supervisor gets only the mount *path* on argv, never the + // credential itself. + let idx = command + .iter() + .position(|a| a == "--upstream-proxy-auth-file") + .expect("auth-file flag present"); + assert_eq!( + command.get(idx + 1).map(String::as_str), + Some(UPSTREAM_PROXY_AUTH_MOUNT_PATH) + ); + // The cleartext-credential acknowledgement travels with the auth + // file so the supervisor's fail-closed pairing check passes. + assert!( + command + .iter() + .any(|a| a == "--upstream-proxy-auth-allow-insecure"), + "acknowledgement flag present: {command:?}" + ); + // The raw credential path from config never appears anywhere in the + // spec (only the fixed mount path does). + assert!( + !command + .iter() + .any(|a| a == "/etc/openshell/secrets/proxy-auth"), + "host-side credential path must not reach the container: {command:?}" + ); + + let secrets = spec["secrets"] + .as_array() + .expect("secrets should be an array"); + assert!( + secrets.iter().any(|secret| { + secret["source"].as_str() == Some(proxy_auth_secret_name(&sandbox.id).as_str()) + && secret["target"].as_str() == Some(UPSTREAM_PROXY_AUTH_MOUNT_PATH) + && secret["mode"].as_u64() == Some(0o400) + }), + "proxy credentials must be delivered through a root-only secret mount" + ); + } + + #[test] + fn container_spec_omits_proxy_auth_mount_when_unconfigured() { + let sandbox = test_sandbox("proxy-id", "proxy-name"); + let mut config = test_config(); + config.https_proxy = Some("http://proxy.corp.com:8080".to_string()); + + let spec = build_container_spec(&sandbox, &config); + let command = spec_command(&spec); + assert!( + !command.iter().any(|a| a == "--upstream-proxy-auth-file"), + "auth-file flag must be absent when no proxy_auth_file is configured: {command:?}" + ); + assert!( + !command + .iter() + .any(|a| a == "--upstream-proxy-auth-allow-insecure"), + "acknowledgement flag must be absent when no proxy_auth_file is configured: {command:?}" + ); + } + + #[test] + fn container_spec_connect_by_hostname_passed_only_on_opt_in() { + let sandbox = test_sandbox("proxy-id", "proxy-name"); + let mut config = test_config(); + config.https_proxy = Some("http://proxy.corp.com:8080".to_string()); + + // Default: no flag, the supervisor uses validated-IP CONNECT binding. + let spec = build_container_spec(&sandbox, &config); + let command = spec_command(&spec); + assert!( + !command + .iter() + .any(|a| a == "--upstream-proxy-connect-by-hostname"), + "hostname CONNECT must be absent without the operator opt-in: {command:?}" + ); + + config.proxy_connect_by_hostname = Some(true); + let spec = build_container_spec(&sandbox, &config); + let command = spec_command(&spec); + assert!( + command + .iter() + .any(|a| a == "--upstream-proxy-connect-by-hostname"), + "hostname CONNECT flag present on opt-in: {command:?}" + ); + } + #[test] fn container_spec_omits_tls_without_config() { let sandbox = test_sandbox("notls-id", "notls-name"); diff --git a/crates/openshell-driver-podman/src/driver.rs b/crates/openshell-driver-podman/src/driver.rs index a86d58dee0..c30cfe901b 100644 --- a/crates/openshell-driver-podman/src/driver.rs +++ b/crates/openshell-driver-podman/src/driver.rs @@ -114,6 +114,50 @@ async fn cleanup_sandbox_token_secret(client: &PodmanClient, secret_name: &str) } } +/// Read the operator's proxy credentials file and stage it as a per-sandbox +/// Podman secret, so the credentials reach the supervisor through a root-only +/// mount rather than the container environment. +/// +/// Fails closed: when `proxy_auth_file` is configured but cannot be read or +/// does not hold a valid `user:pass` credential, the sandbox is not created. +/// Credential validation is shared with the in-container supervisor through +/// [`openshell_core::driver_utils::parse_upstream_proxy_credential`], so a +/// credential staged here can never be rejected at supervisor startup. +async fn create_sandbox_proxy_auth_secret( + client: &PodmanClient, + config: &PodmanComputeConfig, + sandbox: &DriverSandbox, +) -> Result, ComputeDriverError> { + let Some(path) = config.proxy_auth_file.as_deref() else { + return Ok(None); + }; + + let raw = tokio::fs::read_to_string(path).await.map_err(|e| { + ComputeDriverError::Message(format!("failed to read proxy_auth_file '{path}': {e}")) + })?; + let credential = + openshell_core::driver_utils::parse_upstream_proxy_credential(&raw).map_err(|err| { + ComputeDriverError::InvalidArgument(format!("proxy_auth_file '{path}': {err}")) + })?; + + let secret_name = container::proxy_auth_secret_name(&sandbox.id); + client + .create_secret(&secret_name, format!("{credential}\n").as_bytes()) + .await + .map_err(ComputeDriverError::from)?; + Ok(Some(secret_name)) +} + +async fn cleanup_sandbox_proxy_auth_secret(client: &PodmanClient, secret_name: &str) { + if let Err(err) = client.remove_secret(secret_name).await { + warn!( + secret = %secret_name, + error = %err, + "Failed to remove Podman sandbox proxy-auth secret" + ); + } +} + fn local_podman_cdi_gpu_inventory_from(dev_root: &Path) -> CdiGpuInventory { let mut device_ids = std::fs::read_dir(dev_root) .ok() @@ -185,6 +229,7 @@ impl PodmanComputeDriver { config.validate_tls_config()?; config.validate_runtime_limits()?; config.validate_host_gateway_ip()?; + config.validate_proxy_config()?; let client = PodmanClient::new(config.socket_path.clone()); @@ -516,6 +561,29 @@ impl PodmanComputeDriver { return Err(e); } }; + let proxy_auth_secret_name = + match create_sandbox_proxy_auth_secret(&self.client, &self.config, sandbox).await { + Ok(name) => name, + Err(e) => { + let _ = self.client.remove_volume(&vol_name).await; + if let Some(secret) = token_secret_name.as_deref() { + cleanup_sandbox_token_secret(&self.client, secret).await; + } + return Err(e); + } + }; + + // Clean up the volume and both per-sandbox secrets on any failure past + // this point. + let cleanup_created = || async { + let _ = self.client.remove_volume(&vol_name).await; + if let Some(secret) = token_secret_name.as_deref() { + cleanup_sandbox_token_secret(&self.client, secret).await; + } + if let Some(secret) = proxy_auth_secret_name.as_deref() { + cleanup_sandbox_proxy_auth_secret(&self.client, secret).await; + } + }; // 3. Create container. let gpu_devices = match self.resolve_gpu_cdi_devices( @@ -525,10 +593,7 @@ impl PodmanComputeDriver { ) { Ok(devices) => devices, Err(e) => { - let _ = self.client.remove_volume(&vol_name).await; - if let Some(secret) = token_secret_name.as_deref() { - cleanup_sandbox_token_secret(&self.client, secret).await; - } + cleanup_created().await; return Err(e); } }; @@ -540,10 +605,7 @@ impl PodmanComputeDriver { ) { Ok(spec) => spec, Err(e) => { - let _ = self.client.remove_volume(&vol_name).await; - if let Some(secret) = token_secret_name.as_deref() { - cleanup_sandbox_token_secret(&self.client, secret).await; - } + cleanup_created().await; return Err(e); } }; @@ -554,17 +616,11 @@ impl PodmanComputeDriver { // sandbox's ID, not the conflicting container's ID (which // has the same name but a different ID), so it would be // orphaned otherwise. - let _ = self.client.remove_volume(&vol_name).await; - if let Some(secret) = token_secret_name.as_deref() { - cleanup_sandbox_token_secret(&self.client, secret).await; - } + cleanup_created().await; return Err(ComputeDriverError::AlreadyExists); } Err(e) => { - let _ = self.client.remove_volume(&vol_name).await; - if let Some(secret) = token_secret_name.as_deref() { - cleanup_sandbox_token_secret(&self.client, secret).await; - } + cleanup_created().await; return Err(ComputeDriverError::from(e)); } } @@ -577,10 +633,7 @@ impl PodmanComputeDriver { "Failed to start container; cleaning up" ); let _ = self.client.remove_container(&name).await; - let _ = self.client.remove_volume(&vol_name).await; - if let Some(secret) = token_secret_name.as_deref() { - cleanup_sandbox_token_secret(&self.client, secret).await; - } + cleanup_created().await; return Err(ComputeDriverError::from(e)); } @@ -679,6 +732,11 @@ impl PodmanComputeDriver { ); } cleanup_sandbox_token_secret(&self.client, &container::token_secret_name(sandbox_id)).await; + cleanup_sandbox_proxy_auth_secret( + &self.client, + &container::proxy_auth_secret_name(sandbox_id), + ) + .await; Ok(container_existed) } diff --git a/crates/openshell-driver-podman/src/main.rs b/crates/openshell-driver-podman/src/main.rs index c57aff4277..ba15fd2556 100644 --- a/crates/openshell-driver-podman/src/main.rs +++ b/crates/openshell-driver-podman/src/main.rs @@ -103,6 +103,36 @@ struct Args { /// Host path to the client private key for sandbox mTLS. #[arg(long, env = "OPENSHELL_PODMAN_TLS_KEY")] podman_tls_key: Option, + + /// Corporate forward proxy URL for the supervisor's upstream TLS dials, + /// in explicit `http://host:port` form (scheme and port required). + /// Credentials must not be embedded in the URL; use + /// `--sandbox-proxy-auth-file` instead. + #[arg(long, env = "OPENSHELL_SANDBOX_HTTPS_PROXY")] + sandbox_https_proxy: Option, + + /// Comma-separated `NO_PROXY` list injected alongside the proxy URL. + #[arg(long, env = "OPENSHELL_SANDBOX_NO_PROXY")] + sandbox_no_proxy: Option, + + /// Path to a file containing the corporate proxy credentials as + /// `user:pass`. Delivered to the supervisor through a root-only secret + /// mount so the credentials never appear in config or container metadata. + #[arg(long, env = "OPENSHELL_SANDBOX_PROXY_AUTH_FILE")] + sandbox_proxy_auth_file: Option, + + /// Explicit acknowledgement (`true`) that the proxy credential is sent + /// as cleartext Basic auth over the plain-TCP connection to the http:// + /// proxy. Required when `--sandbox-proxy-auth-file` is set. + #[arg(long, env = "OPENSHELL_SANDBOX_PROXY_AUTH_ALLOW_INSECURE")] + sandbox_proxy_auth_allow_insecure: Option, + + /// Send the destination hostname in CONNECT requests to the corporate + /// proxy instead of a validated IP. Only for proxies whose ACLs filter + /// on hostnames: the proxy then resolves the name itself, so sandbox + /// SSRF/`allowed_ips` validation no longer binds the connection. + #[arg(long, env = "OPENSHELL_SANDBOX_PROXY_CONNECT_BY_HOSTNAME")] + sandbox_proxy_connect_by_hostname: Option, } #[tokio::main] @@ -137,6 +167,11 @@ async fn main() -> Result<()> { guest_tls_cert: args.podman_tls_cert, guest_tls_key: args.podman_tls_key, sandbox_pids_limit: args.sandbox_pids_limit, + https_proxy: args.sandbox_https_proxy, + no_proxy: args.sandbox_no_proxy, + proxy_auth_file: args.sandbox_proxy_auth_file, + proxy_auth_allow_insecure: args.sandbox_proxy_auth_allow_insecure, + proxy_connect_by_hostname: args.sandbox_proxy_connect_by_hostname, ..PodmanComputeConfig::default() }) .await diff --git a/crates/openshell-sandbox/src/lib.rs b/crates/openshell-sandbox/src/lib.rs index 78e5027953..bafd3e81b8 100644 --- a/crates/openshell-sandbox/src/lib.rs +++ b/crates/openshell-sandbox/src/lib.rs @@ -106,6 +106,7 @@ pub async fn run_sandbox( ocsf_enabled: Arc, network_enabled: bool, process_enabled: bool, + upstream_proxy_args: openshell_supervisor_network::upstream_proxy::UpstreamProxyArgs, ) -> Result { let (program, args) = command .split_first() @@ -384,6 +385,7 @@ pub async fn run_sandbox( inference_routes.as_deref(), denial_tx, activity_tx, + &upstream_proxy_args, ) .await?, ) diff --git a/crates/openshell-sandbox/src/main.rs b/crates/openshell-sandbox/src/main.rs index 71f881f68a..62ae37b5a1 100644 --- a/crates/openshell-sandbox/src/main.rs +++ b/crates/openshell-sandbox/src/main.rs @@ -101,6 +101,9 @@ impl std::str::FromStr for Mode { } /// `OpenShell` Sandbox - process isolation and monitoring. +// CLI flags are naturally boolean switches; grouping them into structs would +// only obscure the clap definition. +#[allow(clippy::struct_excessive_bools)] #[derive(Parser, Debug)] #[command(name = "openshell-sandbox")] #[command(version = openshell_core::VERSION)] @@ -200,6 +203,32 @@ struct Args { /// Shared TLS work directory between the network init container and sidecar. #[arg(long, env = "OPENSHELL_PROXY_TLS_DIR", default_value = SIDECAR_TLS_DIR)] sidecar_tls_dir: String, + + // Corporate upstream proxy. Operator-owned egress boundary: accepted + // only as command-line arguments (no `env =`), because the driver + // controls the supervisor's argv while a sandbox image could bake + // matching `ENV` values. + /// Corporate forward proxy URL (`http://host:port`) for upstream TLS egress. + #[arg(long)] + upstream_proxy: Option, + + /// Comma-separated `NO_PROXY` list for the corporate proxy. + #[arg(long)] + upstream_no_proxy: Option, + + /// Path to the root-only file holding corporate proxy credentials (`user:pass`). + #[arg(long)] + upstream_proxy_auth_file: Option, + + /// Acknowledge that proxy credentials travel as cleartext Basic auth over + /// the plain-TCP connection to the `http://` proxy. + #[arg(long)] + upstream_proxy_auth_allow_insecure: bool, + + /// Send the destination hostname in CONNECT instead of a validated IP + /// (for proxies whose ACLs filter on hostnames). + #[arg(long)] + upstream_proxy_connect_by_hostname: bool, } /// Copy the running executable to `dest`, creating parent directories as @@ -581,6 +610,14 @@ fn main() -> Result<()> { // is not yet initialized at this point (run_sandbox hasn't been called). // The shorthand layer will render it in fallback format. + let upstream_proxy_args = openshell_supervisor_network::upstream_proxy::UpstreamProxyArgs { + https_proxy: args.upstream_proxy, + no_proxy: args.upstream_no_proxy, + proxy_auth_file: args.upstream_proxy_auth_file, + proxy_auth_allow_insecure: args.upstream_proxy_auth_allow_insecure, + proxy_connect_by_hostname: args.upstream_proxy_connect_by_hostname, + }; + run_sandbox( command, args.workdir, @@ -598,6 +635,7 @@ fn main() -> Result<()> { ocsf_enabled, args.mode.network, args.mode.process, + upstream_proxy_args, ) .await })?; diff --git a/crates/openshell-supervisor-network/src/l7/tls.rs b/crates/openshell-supervisor-network/src/l7/tls.rs index 70e198f420..f7c923c690 100644 --- a/crates/openshell-supervisor-network/src/l7/tls.rs +++ b/crates/openshell-supervisor-network/src/l7/tls.rs @@ -184,7 +184,7 @@ pub async fn tls_terminate_client( /// /// Returns a TLS stream for re-encrypted upstream communication. pub async fn tls_connect_upstream( - upstream: TcpStream, + upstream: impl AsyncRead + AsyncWrite + Unpin + Send, hostname: &str, client_config: &Arc, ) -> Result { diff --git a/crates/openshell-supervisor-network/src/lib.rs b/crates/openshell-supervisor-network/src/lib.rs index c6c1af5aed..ac0cb120a2 100644 --- a/crates/openshell-supervisor-network/src/lib.rs +++ b/crates/openshell-supervisor-network/src/lib.rs @@ -19,3 +19,4 @@ pub mod run; pub mod sigv4; mod spiffe_endpoint; mod token_grant; +pub mod upstream_proxy; diff --git a/crates/openshell-supervisor-network/src/proxy.rs b/crates/openshell-supervisor-network/src/proxy.rs index ab2313b890..5ef0de76e7 100644 --- a/crates/openshell-supervisor-network/src/proxy.rs +++ b/crates/openshell-supervisor-network/src/proxy.rs @@ -7,6 +7,7 @@ use crate::identity::BinaryIdentityCache; use crate::l7::tls::ProxyTlsState; use crate::opa::{NetworkAction, OpaEngine, PolicyGenerationGuard}; use crate::policy_local::{POLICY_LOCAL_HOST, PolicyLocalContext}; +use crate::upstream_proxy::{self, UpstreamProxyConfig}; use miette::{IntoDiagnostic, Result}; use openshell_core::activity::{ActivitySender, try_record_activity}; use openshell_core::denial::DenialEvent; @@ -200,6 +201,7 @@ impl ProxyHandle { denial_tx: Option>, activity_tx: Option, engine_ready: tokio::sync::watch::Receiver, + upstream_proxy_args: &upstream_proxy::UpstreamProxyArgs, ) -> Result { // Use override bind_addr, fall back to policy http_addr, then default // to loopback:3128. The default allows the proxy to function when no @@ -240,6 +242,45 @@ impl ProxyHandle { ); } + // Corporate egress proxy configured by the operator and delivered on + // the supervisor's command line by the compute driver; the + // conventional HTTPS_PROXY/NO_PROXY variables the sandbox controls + // are ignored here. + // + // This is an operator-owned security boundary, so a present-but-invalid + // value (bad proxy URL, unreadable auth file, malformed credential) is + // fatal to proxy startup: failing closed prevents a misconfiguration + // from silently degrading to direct dialing or unauthenticated proxy + // access. + let upstream_proxy: Arc> = Arc::new( + UpstreamProxyConfig::from_args(upstream_proxy_args).map_err(|err| { + let event = + openshell_ocsf::ConfigStateChangeBuilder::new(openshell_ocsf::ctx::ctx()) + .severity(SeverityId::High) + .status(StatusId::Failure) + .state(openshell_ocsf::StateId::Disabled, "invalid") + .message(format!( + "Upstream corporate proxy configuration invalid; \ + refusing to start: {err}" + )) + .build(); + ocsf_emit!(event); + miette::miette!("invalid upstream corporate proxy configuration: {err}") + })?, + ); + if let Some(cfg) = upstream_proxy.as_ref() { + let event = openshell_ocsf::ConfigStateChangeBuilder::new(openshell_ocsf::ctx::ctx()) + .severity(SeverityId::Informational) + .status(StatusId::Success) + .state(openshell_ocsf::StateId::Enabled, "enabled") + .message(format!( + "Upstream corporate proxy enabled: {}", + cfg.summary() + )) + .build(); + ocsf_emit!(event); + } + let join = tokio::spawn(async move { // Wait for the OPA engine's symlink resolution reload to complete // before accepting connections. This prevents requests from @@ -275,6 +316,7 @@ impl ProxyHandle { let inf = inference_ctx.clone(); let policy_local = policy_local_ctx.clone(); let gw = trusted_host_gateway.clone(); + let up_proxy = upstream_proxy.clone(); let resolver = provider_credentials .as_ref() .and_then(ProviderCredentialState::resolver); @@ -296,6 +338,7 @@ impl ProxyHandle { inf, policy_local, gw, + up_proxy, resolver, dynamic_credentials, dtx, @@ -627,6 +670,7 @@ async fn handle_tcp_connection( inference_ctx: Option>, policy_local_ctx: Option>, trusted_host_gateway: Arc>, + upstream_proxy: Arc>, secret_resolver: Option>, dynamic_credentials: Option< Arc< @@ -1169,7 +1213,7 @@ async fn handle_tcp_connection( return Ok(()); } - let mut upstream = TcpStream::connect(validated_addrs.as_slice()) + let mut upstream = dial_upstream(&upstream_proxy, &host_lc, port, &validated_addrs) .await .into_diagnostic()?; @@ -2897,6 +2941,67 @@ fn validate_declared_endpoint_resolved_addrs( Ok(()) } +/// Dial a validated upstream destination for a TLS (CONNECT) tunnel. +/// +/// Connects directly to the SSRF-checked resolved addresses, or chains +/// through the corporate proxy (HTTP CONNECT) when one is configured for +/// this destination via the supervisor's reserved upstream proxy variables +/// and not excluded by the reserved `NO_PROXY` list. Policy evaluation and +/// SSRF validation must have already succeeded; only the final TCP dial +/// changes. Plain-HTTP requests never take this path: they always dial the +/// destination directly. +/// +/// `NO_PROXY` evaluation is port-aware and sees the validated resolved +/// addresses: an entry with a `:port` qualifier only bypasses that port, +/// and an IP/CIDR entry that matches through resolution limits the direct +/// dial to the addresses it contains. +/// +/// The CONNECT target sent to the corporate proxy is a validated resolved +/// address, so the proxy performs no DNS resolution of its own and the +/// tunnel stays bound to the answer that passed SSRF and `allowed_ips` +/// validation; the hostname still travels inside the tunnel (TLS SNI, +/// application `Host`). The operator opt-in `proxy_connect_by_hostname` +/// sends the client-requested hostname instead, for proxies whose ACLs +/// filter on hostnames, at the cost of re-opening proxy-side resolution. +/// +/// Both paths return a [`upstream_proxy::PrefixedStream`]: for proxied +/// dials it replays any tunneled bytes that arrived in the same read as the +/// CONNECT response; for direct dials it is a plain passthrough. +async fn dial_upstream( + upstream_proxy: &Option, + host_lc: &str, + port: u16, + addrs: &[SocketAddr], +) -> std::io::Result { + if let Some(cfg) = upstream_proxy.as_ref() { + return match cfg.decision(host_lc, port, addrs) { + upstream_proxy::ProxyDecision::Proxy(endpoint) => { + if cfg.connect_by_hostname() { + upstream_proxy::connect_via( + endpoint, + host_lc, + port, + upstream_proxy::ConnectTarget::Hostname, + ) + .await + } else { + // Try every validated address in order, matching the + // fallback the direct path's `TcpStream::connect` does. + upstream_proxy::connect_via_validated(endpoint, host_lc, port, addrs).await + } + } + upstream_proxy::ProxyDecision::Direct(direct_addrs) => { + Ok(upstream_proxy::PrefixedStream::without_prefix( + TcpStream::connect(&direct_addrs[..]).await?, + )) + } + }; + } + Ok(upstream_proxy::PrefixedStream::without_prefix( + TcpStream::connect(addrs).await?, + )) +} + /// Resolve a host:port using sandbox `/etc/hosts` first (when available), then /// reject if any resolved address is internal. /// @@ -4569,8 +4674,12 @@ async fn handle_forward_proxy( return Ok(()); } - // 6. Connect upstream - let mut upstream = match TcpStream::connect(addrs.as_slice()).await { + // 6. Connect upstream. Plain-HTTP requests always dial the destination + // directly: only TLS (CONNECT) tunnels chain through the corporate + // proxy, since plain-HTTP forwarding would need absolute-form requests + // rather than a CONNECT tunnel. + let dial_result = TcpStream::connect(addrs.as_slice()).await; + let mut upstream = match dial_result { Ok(s) => s, Err(e) => { let event = HttpActivityBuilder::new(openshell_ocsf::ctx::ctx()) @@ -5037,6 +5146,7 @@ network_policies: {} None, None, Arc::new(None), + Arc::new(None), None, None, None, @@ -9323,6 +9433,7 @@ network_policies: None, // inference_ctx None, // policy_local_ctx Arc::new(None), // trusted_host_gateway + Arc::new(None), // upstream_proxy None, // secret_resolver None, // dynamic_credentials Some(denial_tx), // denial_tx — positive allow/deny signal diff --git a/crates/openshell-supervisor-network/src/run.rs b/crates/openshell-supervisor-network/src/run.rs index d68db6f1c3..c9eaa08753 100644 --- a/crates/openshell-supervisor-network/src/run.rs +++ b/crates/openshell-supervisor-network/src/run.rs @@ -86,6 +86,7 @@ pub async fn run_networking( inference_routes: Option<&str>, denial_tx: Option>, activity_tx: Option, + upstream_proxy_args: &crate::upstream_proxy::UpstreamProxyArgs, ) -> Result { // Build the policy-local route context. The orchestrator's policy poll // loop also holds an `Arc` clone (via `Networking::policy_local_ctx`) so @@ -309,6 +310,7 @@ pub async fn run_networking( denial_tx, activity_tx, engine_ready_rx, + upstream_proxy_args, ) .await?; Some(proxy_handle) diff --git a/crates/openshell-supervisor-network/src/upstream_proxy.rs b/crates/openshell-supervisor-network/src/upstream_proxy.rs new file mode 100644 index 0000000000..1f73738604 --- /dev/null +++ b/crates/openshell-supervisor-network/src/upstream_proxy.rs @@ -0,0 +1,1684 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +//! Upstream corporate proxy chaining for the sandbox egress proxy. +//! +//! In proxy-required enterprise networks (issue #1792) the supervisor cannot +//! dial policy-approved destinations directly: all outbound traffic must go +//! through a corporate forward proxy. The operator-owned proxy configuration +//! reaches the supervisor as command-line arguments ([`UpstreamProxyArgs`]) +//! that the compute driver sets when it launches the supervisor — never +//! through the environment, which sandbox images could bake `ENV` values +//! into — and this module chains approved TLS tunnels through the corporate +//! proxy with HTTP CONNECT. +//! +//! Only TLS (CONNECT) egress is chained: plain-HTTP requests always dial the +//! destination directly. Forwarding plain HTTP through a corporate proxy +//! requires absolute-form request forwarding rather than CONNECT tunneling +//! and is out of scope for this feature. +//! +//! The conventional `HTTPS_PROXY` / `HTTP_PROXY` / `ALL_PROXY` / `NO_PROXY` +//! variables are intentionally ignored: those are controlled by the sandbox +//! creator and are rewritten separately to point the workload child at the +//! local policy proxy, so honoring them would let a sandbox pick an arbitrary +//! upstream proxy or disable proxying with `NO_PROXY=*`. +//! +//! Scope and invariants: +//! - Only `http://` proxy URLs are supported. Configuration is fail-closed: +//! any present-but-invalid reserved value — a present-but-empty variable, +//! an unsupported (`https://`, SOCKS) or malformed proxy URL, an unreadable +//! auth file, a malformed credential, or an auth file without the explicit +//! cleartext-credential acknowledgement — is a fatal startup error rather +//! than being silently ignored, so a typo can never quietly downgrade the +//! operator's egress boundary to direct dialing or unauthenticated proxy +//! access. Validation semantics are shared with the compute driver via +//! [`openshell_core::driver_utils::parse_upstream_proxy_url`] and +//! [`openshell_core::driver_utils::parse_upstream_proxy_credential`]. +//! - Policy evaluation, DNS resolution, and SSRF checks run exactly as in the +//! direct-dial path; the corporate proxy only replaces the final TCP dial. +//! - CONNECT requests target a validated resolved address by default, so the +//! proxy performs no DNS resolution and the tunnel stays bound to the +//! answer that passed SSRF/`allowed_ips` validation. The reserved +//! `--upstream-proxy-connect-by-hostname` opt-in sends the hostname +//! instead, for proxies whose ACLs filter on hostnames. +//! - The reserved `NO_PROXY` list decides which destinations bypass the +//! corporate proxy and keep dialing directly (cluster-internal services, +//! host gateway, etc.). Loopback destinations always bypass the proxy. + +use std::io::{Error as IoError, ErrorKind}; +use std::net::{IpAddr, SocketAddr}; +use std::pin::Pin; +use std::task::{Context, Poll}; +use std::time::Duration; + +use base64::Engine as _; +use tokio::io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt, ReadBuf}; +use tokio::net::TcpStream; +use tracing::debug; + +/// Upper bound on the corporate proxy's CONNECT response header block. +const MAX_CONNECT_RESPONSE_BYTES: usize = 8 * 1024; + +/// End-to-end budget for dialing the corporate proxy and completing the +/// CONNECT handshake. +const CONNECT_HANDSHAKE_TIMEOUT: Duration = Duration::from_secs(30); + +/// A parsed corporate proxy endpoint. +#[derive(Clone)] +pub struct ProxyEndpoint { + host: String, + port: u16, + /// Pre-computed `Basic ` header value from the proxy auth file. + /// Never logged. + proxy_authorization: Option, +} + +impl ProxyEndpoint { + /// `host:port` label for logs. Excludes credentials. + #[must_use] + pub fn display_addr(&self) -> String { + format!("{}:{}", self.host, self.port) + } +} + +impl std::fmt::Debug for ProxyEndpoint { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("ProxyEndpoint") + .field("host", &self.host) + .field("port", &self.port) + .field("proxy_authorization", &self.proxy_authorization.is_some()) + .finish() + } +} + +/// The pattern half of one parsed `NO_PROXY` entry. +#[derive(Debug, Clone)] +enum NoProxyPattern { + /// `*` — bypass the proxy for every destination. + Wildcard, + /// Domain suffix match: `corp.com` matches `corp.com` and `x.corp.com`. + Domain(String), + /// Exact IP match, against an IP-literal host or its resolved addresses. + Ip(IpAddr), + /// CIDR match, against an IP-literal host or its resolved addresses. + Cidr(ipnet::IpNet), +} + +/// One parsed `NO_PROXY` entry. +#[derive(Debug, Clone)] +struct NoProxyEntry { + pattern: NoProxyPattern, + /// When set (`internal.corp:8443`), the entry only applies to this + /// destination port instead of every port. + port: Option, +} + +/// Parsed `NO_PROXY` list. +#[derive(Debug, Clone, Default)] +struct NoProxy { + entries: Vec, +} + +impl NoProxy { + fn parse(raw: &str) -> Self { + let mut entries = Vec::new(); + for item in raw.split(',') { + let item = item.trim(); + if item.is_empty() { + continue; + } + if item == "*" { + entries.push(NoProxyEntry { + pattern: NoProxyPattern::Wildcard, + port: None, + }); + continue; + } + // Whole-item IP/CIDR forms first: a bare IPv6 literal contains + // colons that must never be misread as a port qualifier. + if let Ok(net) = item.parse::() { + entries.push(NoProxyEntry { + pattern: NoProxyPattern::Cidr(net), + port: None, + }); + continue; + } + if let Ok(ip) = item.trim_matches(['[', ']']).parse::() { + entries.push(NoProxyEntry { + pattern: NoProxyPattern::Ip(ip), + port: None, + }); + continue; + } + // Optional `:port` qualifier: only a valid trailing u16 counts; + // anything else stays part of the pattern. + let (head, port) = item + .rsplit_once(':') + .map_or((item, None), |(head, port_str)| { + port_str + .parse::() + .map_or((item, None), |port| (head, Some(port))) + }); + let pattern = if let Ok(net) = head.parse::() { + NoProxyPattern::Cidr(net) + } else if let Ok(ip) = head.trim_matches(['[', ']']).parse::() { + NoProxyPattern::Ip(ip) + } else { + // Domain entry. Strip any leading `*.` or `.` so + // `.corp.com`, `*.corp.com`, and `corp.com` all behave + // identically. + let name = head + .strip_prefix("*.") + .or_else(|| head.strip_prefix('.')) + .unwrap_or(head) + .to_ascii_lowercase(); + if name.is_empty() { + continue; + } + NoProxyPattern::Domain(name) + }; + entries.push(NoProxyEntry { pattern, port }); + } + Self { entries } + } + + /// The validated addresses that may be dialed directly for + /// `(host, port)`, or `None` when no entry matches and the corporate + /// proxy must be used. + /// + /// Hostname-level matches — loopback, `*`, a domain entry, or an IP/CIDR + /// entry matching an IP-literal host — authorize every validated + /// address. IP/CIDR entries match hostnames through their *resolved* + /// addresses and authorize only the addresses they contain, so a bypass + /// scoped to an internal range can never widen into a direct dial of an + /// address outside that range. + fn direct_addrs( + &self, + host: &str, + port: u16, + resolved: &[SocketAddr], + ) -> Option> { + let host_ip = host.trim_matches(['[', ']']).parse::().ok(); + if host == "localhost" || host_ip.is_some_and(|ip| ip.is_loopback()) { + return Some(resolved.to_vec()); + } + let mut subset: Vec = Vec::new(); + for entry in &self.entries { + if entry.port.is_some_and(|entry_port| entry_port != port) { + continue; + } + match &entry.pattern { + NoProxyPattern::Wildcard => return Some(resolved.to_vec()), + NoProxyPattern::Domain(suffix) => { + if host == suffix + || host + .strip_suffix(suffix) + .is_some_and(|prefix| prefix.ends_with('.')) + { + return Some(resolved.to_vec()); + } + } + NoProxyPattern::Ip(ip) => { + if host_ip == Some(*ip) { + return Some(resolved.to_vec()); + } + for addr in resolved { + if addr.ip() == *ip && !subset.contains(addr) { + subset.push(*addr); + } + } + } + NoProxyPattern::Cidr(net) => { + if host_ip.is_some_and(|ip| net.contains(&ip)) { + return Some(resolved.to_vec()); + } + for addr in resolved { + if net.contains(&addr.ip()) && !subset.contains(addr) { + subset.push(*addr); + } + } + } + } + } + if subset.is_empty() { + None + } else { + Some(subset) + } + } +} + +/// How a validated destination must be dialed, per the reserved `NO_PROXY` +/// contract. Produced by [`UpstreamProxyConfig::decision`]. +#[derive(Debug)] +pub enum ProxyDecision<'a> { + /// Chain through the corporate proxy with HTTP CONNECT. + Proxy(&'a ProxyEndpoint), + /// Dial directly, restricted to this subset of the validated addresses: + /// all of them for hostname-level matches (loopback, `*`, domain + /// entries, IP-literal hosts), only the addresses contained in the + /// matching IP/CIDR entries otherwise. + Direct(Vec), +} + +/// What the supervisor puts in the CONNECT request line sent to the +/// corporate proxy. +#[derive(Debug, Clone, Copy)] +pub enum ConnectTarget { + /// CONNECT to this validated address; the proxy performs no DNS + /// resolution, so the tunnel is bound to the answer that already passed + /// SSRF and `allowed_ips` validation. The destination hostname still + /// travels inside the tunnel (TLS SNI, application `Host`). + Ip(IpAddr), + /// CONNECT by hostname; the proxy resolves the name itself. Operator + /// opt-in for hostname-filtering proxy ACLs + /// (`--upstream-proxy-connect-by-hostname`). + Hostname, +} + +/// Corporate proxy configuration built from the driver-supplied +/// command-line arguments ([`UpstreamProxyArgs`]). +#[derive(Debug, Clone)] +pub struct UpstreamProxyConfig { + https: ProxyEndpoint, + no_proxy: NoProxy, + connect_by_hostname: bool, +} + +/// Operator-owned corporate proxy settings passed to the supervisor as +/// command-line arguments by the compute driver. +/// +/// Argv is set by the driver when it creates the container/VM and cannot be +/// influenced by sandbox environment or image `ENV`, so a field left `None` +/// / `false` genuinely means "not configured by the operator". +#[derive(Debug, Clone, Default)] +pub struct UpstreamProxyArgs { + /// `http://host:port` corporate proxy URL, or `None` for direct egress. + pub https_proxy: Option, + /// Comma-separated `NO_PROXY` list. + pub no_proxy: Option, + /// Path to the root-only credential mount (`user:pass`). + pub proxy_auth_file: Option, + /// Operator acknowledgement that credentials travel as cleartext Basic + /// auth over the plain-TCP proxy connection. + pub proxy_auth_allow_insecure: bool, + /// Send the destination hostname in CONNECT instead of a validated IP. + pub proxy_connect_by_hostname: bool, +} + +// Supervisor CLI flag names for the corporate-proxy settings, used as the +// dispatch keys in `from_lookup` and in operator-facing error messages. +const ARG_HTTPS_PROXY: &str = "--upstream-proxy"; +const ARG_NO_PROXY: &str = "--upstream-no-proxy"; +const ARG_PROXY_AUTH_FILE: &str = "--upstream-proxy-auth-file"; +const ARG_PROXY_AUTH_ALLOW_INSECURE: &str = "--upstream-proxy-auth-allow-insecure"; +const ARG_PROXY_CONNECT_BY_HOSTNAME: &str = "--upstream-proxy-connect-by-hostname"; + +impl UpstreamProxyConfig { + /// Build the corporate proxy configuration from the driver-supplied + /// command-line [`UpstreamProxyArgs`]. Returns `Ok(None)` when no proxy + /// is configured. + /// + /// The conventional `HTTPS_PROXY` / `HTTP_PROXY` / `ALL_PROXY` / + /// `NO_PROXY` environment variables are intentionally never consulted: + /// they are controlled by the sandbox creator (and rewritten to point + /// workload children at the local policy proxy), so honoring them would + /// let a sandbox choose an arbitrary upstream proxy or disable proxying + /// entirely. + /// + /// # Errors + /// + /// This is an operator-owned security boundary, so a present-but-invalid + /// value is fatal instead of being treated as unset: an empty (or + /// whitespace-only) argument, an invalid or unsupported proxy URL, an + /// auth file that is set but unreadable or holds a malformed credential, + /// an auth file without the cleartext-credential acknowledgement, or an + /// auth file / `NO_PROXY` list / acknowledgement / connect-by-hostname + /// flag with no proxy configured. Failing closed prevents a + /// misconfiguration from silently degrading to direct dialing or + /// unauthenticated proxy access. + pub fn from_args(args: &UpstreamProxyArgs) -> Result, String> { + // Present the typed argv fields under their canonical identifiers so + // the shared validation runs once. Booleans map to Some("true") / + // None, so every pairing rule (auth file needs the acknowledgement, + // no auxiliary setting without a proxy) applies unchanged. + Self::from_lookup(|name| { + if name == ARG_HTTPS_PROXY { + args.https_proxy.clone() + } else if name == ARG_NO_PROXY { + args.no_proxy.clone() + } else if name == ARG_PROXY_AUTH_FILE { + args.proxy_auth_file.clone() + } else if name == ARG_PROXY_AUTH_ALLOW_INSECURE { + args.proxy_auth_allow_insecure.then(|| "true".to_string()) + } else if name == ARG_PROXY_CONNECT_BY_HOSTNAME { + args.proxy_connect_by_hostname.then(|| "true".to_string()) + } else { + None + } + }) + } + + fn from_lookup(lookup: impl Fn(&str) -> Option) -> Result, String> { + // A missing setting means "not configured". A present-but-empty value + // is a misconfiguration (the driver never emits one), so it is fatal + // rather than silently downgrading the boundary to direct dialing or + // unauthenticated proxy access. + let var = |name: &str| -> Result, String> { + match lookup(name) { + None => Ok(None), + Some(value) if value.trim().is_empty() => { + Err(format!("{name} must not be empty when set")) + } + Some(value) => Ok(Some(value)), + } + }; + let https = var(ARG_HTTPS_PROXY)? + .map(|url| parse_proxy_url(&url, ARG_HTTPS_PROXY)) + .transpose()?; + let auth_file = var(ARG_PROXY_AUTH_FILE)?; + let auth_allow_insecure = var(ARG_PROXY_AUTH_ALLOW_INSECURE)?; + let connect_by_hostname_raw = var(ARG_PROXY_CONNECT_BY_HOSTNAME)?; + let no_proxy_list = var(ARG_NO_PROXY)?; + let Some(mut https) = https else { + // Auxiliary proxy settings without a proxy mean the operator + // believed a proxy boundary was in effect; refuse rather than + // silently running with direct egress. + for (name, value) in [ + (ARG_PROXY_AUTH_FILE, &auth_file), + (ARG_PROXY_AUTH_ALLOW_INSECURE, &auth_allow_insecure), + (ARG_PROXY_CONNECT_BY_HOSTNAME, &connect_by_hostname_raw), + (ARG_NO_PROXY, &no_proxy_list), + ] { + if value.is_some() { + return Err(format!("{name} is set but no upstream proxy is configured")); + } + } + return Ok(None); + }; + + // CONNECT-target mode. The default binds the tunnel to a validated + // address; hostname CONNECT re-opens proxy-side DNS resolution and + // is only honored as the exact opt-in value the driver writes. + let connect_by_hostname = match connect_by_hostname_raw.as_deref().map(str::trim) { + None => false, + Some("true") => true, + Some(_) => { + return Err(format!( + "{ARG_PROXY_CONNECT_BY_HOSTNAME} must be 'true' when set" + )); + } + }; + + // Cleartext-credential acknowledgement. `Proxy-Authorization: Basic` + // travels over plain TCP to the http:// proxy, so credentials are + // only sent when the operator explicitly opted in. The compute driver + // enforces the same pairing at sandbox-create time; enforcing it here + // as well keeps the supervisor fail-closed against a bypassed or + // foreign driver. + let allow_insecure = match auth_allow_insecure.as_deref().map(str::trim) { + None => false, + Some("true") => true, + Some(_) => { + return Err(format!( + "{ARG_PROXY_AUTH_ALLOW_INSECURE} must be 'true' when set" + )); + } + }; + if auth_file.is_none() && auth_allow_insecure.is_some() { + return Err(format!( + "{ARG_PROXY_AUTH_ALLOW_INSECURE} is set but no {ARG_PROXY_AUTH_FILE} is configured" + )); + } + if auth_file.is_some() && !allow_insecure { + return Err(format!( + "{ARG_PROXY_AUTH_FILE} sends the credential as cleartext Basic auth \ + over the plain-TCP proxy connection; refusing without \ + {ARG_PROXY_AUTH_ALLOW_INSECURE}" + )); + } + + // Load proxy credentials from the reserved auth file, if configured. + // The file is delivered through a root-only secret mount so the + // credentials never appear in the environment or container metadata. + if let Some(path) = auth_file { + let credential = std::fs::read_to_string(&path).map_err(|err| { + format!("failed to read upstream proxy auth file '{path}': {err}") + })?; + let header = basic_auth_header(&credential).map_err(|err| { + format!("invalid credential in upstream proxy auth file '{path}': {err}") + })?; + https.proxy_authorization = Some(header); + } + + Ok(Some(Self { + https, + no_proxy: NoProxy::parse(&no_proxy_list.unwrap_or_default()), + connect_by_hostname, + })) + } + + /// Whether CONNECT requests carry the destination hostname instead of a + /// validated IP (operator opt-in for hostname-filtering proxy ACLs). + #[must_use] + pub fn connect_by_hostname(&self) -> bool { + self.connect_by_hostname + } + + /// How to dial the validated destination `(host, port, resolved)`, + /// honoring the reserved `NO_PROXY` list. + /// + /// Entries may carry a `:port` qualifier that limits them to that + /// destination port. IP/CIDR entries also match a hostname through its + /// already-validated resolved addresses; such a match authorizes a + /// direct dial of only the addresses inside the entry (see + /// [`ProxyDecision::Direct`]). + #[must_use] + pub fn decision<'a>( + &'a self, + host: &str, + port: u16, + resolved: &[SocketAddr], + ) -> ProxyDecision<'a> { + self.no_proxy + .direct_addrs(host, port, resolved) + .map_or(ProxyDecision::Proxy(&self.https), ProxyDecision::Direct) + } + + /// Credential-free summary for startup logging. + #[must_use] + pub fn summary(&self) -> String { + format!( + "https_proxy={} no_proxy_entries={} connect_target={}", + self.https.display_addr(), + self.no_proxy.entries.len(), + if self.connect_by_hostname { + "hostname" + } else { + "validated-ip" + } + ) + } +} + +/// Parse an `http://host:port` proxy URL with the same validation rules the +/// compute driver applies at sandbox-create time +/// ([`parse_upstream_proxy_url`](openshell_core::driver_utils::parse_upstream_proxy_url)). +/// +/// Credentials are never taken from the URL: they are delivered out of band +/// through the root-only auth-file mount (`--upstream-proxy-auth-file`) so +/// they never appear in config or container metadata. +/// +/// # Errors +/// +/// Rejects unsupported schemes (TLS or SOCKS proxies), inline `user:pass@` +/// credentials, and malformed addresses. The error names `var_name` so the +/// operator can locate the offending setting. +fn parse_proxy_url(raw: &str, var_name: &str) -> Result { + let addr = openshell_core::driver_utils::parse_upstream_proxy_url(raw) + .map_err(|err| format!("{var_name} is invalid: {err}"))?; + Ok(ProxyEndpoint { + host: addr.host, + port: addr.port, + proxy_authorization: None, + }) +} + +/// Build a `Proxy-Authorization: Basic ` header value from a raw +/// `user:pass` credential. +/// +/// The credential is used verbatim after trimming: it is delivered through a +/// trusted operator file, not a URL, so there is no percent-encoding to +/// decode. Validation is shared with the compute driver through +/// [`parse_upstream_proxy_credential`](openshell_core::driver_utils::parse_upstream_proxy_credential), +/// so a credential the driver staged at sandbox-create time is never rejected +/// here. +/// +/// # Errors +/// +/// Rejects an empty credential, one containing control characters that could +/// inject additional HTTP headers, and one not in `user:pass` form. Error +/// messages never include the credential content. +fn basic_auth_header(credential: &str) -> Result { + let credential = openshell_core::driver_utils::parse_upstream_proxy_credential(credential) + .map_err(|err| err.to_string())?; + Ok(format!( + "Basic {}", + base64::engine::general_purpose::STANDARD.encode(credential) + )) +} + +/// A `TcpStream` that first replays bytes already read from the socket. +/// +/// The CONNECT handshake reads from the proxy socket in chunks, so the read +/// that completes the response header block may also contain the first +/// tunneled payload bytes (a server-speaks-first destination, or a proxy +/// that coalesces writes). Those bytes belong to the destination byte +/// stream and must reach the caller rather than being discarded; this +/// wrapper yields them before reading from the socket again. Writes pass +/// straight through. +#[derive(Debug)] +pub struct PrefixedStream { + inner: TcpStream, + /// Bytes read past the CONNECT header terminator, replayed first. + prefix: Vec, + /// Read offset into `prefix`. + pos: usize, +} + +impl PrefixedStream { + /// Wrap `inner`, replaying `prefix` before socket reads. + #[must_use] + pub fn new(inner: TcpStream, prefix: Vec) -> Self { + Self { + inner, + prefix, + pos: 0, + } + } + + /// Wrap a directly dialed stream with nothing to replay. + #[must_use] + pub fn without_prefix(inner: TcpStream) -> Self { + Self::new(inner, Vec::new()) + } +} + +impl AsyncRead for PrefixedStream { + fn poll_read( + self: Pin<&mut Self>, + cx: &mut Context<'_>, + buf: &mut ReadBuf<'_>, + ) -> Poll> { + let this = self.get_mut(); + if this.pos < this.prefix.len() { + let n = buf.remaining().min(this.prefix.len() - this.pos); + buf.put_slice(&this.prefix[this.pos..this.pos + n]); + this.pos += n; + if this.pos == this.prefix.len() { + // Drained: release the buffer instead of holding it for the + // tunnel's lifetime. + this.prefix = Vec::new(); + this.pos = 0; + } + return Poll::Ready(Ok(())); + } + Pin::new(&mut this.inner).poll_read(cx, buf) + } +} + +impl AsyncWrite for PrefixedStream { + fn poll_write( + self: Pin<&mut Self>, + cx: &mut Context<'_>, + buf: &[u8], + ) -> Poll> { + Pin::new(&mut self.get_mut().inner).poll_write(cx, buf) + } + + fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> { + Pin::new(&mut self.get_mut().inner).poll_flush(cx) + } + + fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> { + Pin::new(&mut self.get_mut().inner).poll_shutdown(cx) + } + + fn poll_write_vectored( + self: Pin<&mut Self>, + cx: &mut Context<'_>, + bufs: &[std::io::IoSlice<'_>], + ) -> Poll> { + Pin::new(&mut self.get_mut().inner).poll_write_vectored(cx, bufs) + } + + fn is_write_vectored(&self) -> bool { + self.inner.is_write_vectored() + } +} + +/// Open a tunnel to the destination through the corporate proxy with HTTP +/// CONNECT. +/// +/// Returns the connected stream once the proxy answers 200; after that the +/// stream is a transparent byte pipe to the destination. Any tunneled bytes +/// received in the same read as the CONNECT response are preserved and +/// replayed by the returned [`PrefixedStream`]. +/// +/// `target` selects the CONNECT request target. The default mode sends a +/// validated IP ([`ConnectTarget::Ip`]) so the proxy performs no DNS +/// resolution and the tunnel is bound to an address that already passed +/// SSRF and `allowed_ips` validation; `host` then only labels logs and is +/// used by the caller for TLS SNI inside the tunnel. With the operator +/// opt-in ([`ConnectTarget::Hostname`]) the hostname is sent instead so +/// hostname-filtering proxy ACLs keep working, at the cost of proxy-side +/// resolution. Local DNS resolution and SSRF validation must already have +/// happened at the call site in both modes. +/// +/// # Errors +/// +/// Returns an error when the proxy is unreachable, the handshake times out, +/// or the proxy answers with a non-200 status. +pub async fn connect_via( + endpoint: &ProxyEndpoint, + host: &str, + port: u16, + target: ConnectTarget, +) -> std::io::Result { + tokio::time::timeout( + CONNECT_HANDSHAKE_TIMEOUT, + connect_via_inner(endpoint, host, port, target), + ) + .await + .map_err(|_| { + IoError::new( + ErrorKind::TimedOut, + format!( + "upstream proxy {} CONNECT handshake timed out", + endpoint.display_addr() + ), + ) + })? +} + +/// Open a validated-IP CONNECT tunnel, trying each validated address in +/// order until one succeeds. +/// +/// The direct-dial path hands `TcpStream::connect` the whole validated list +/// and it falls back across addresses; this is the proxied equivalent, so a +/// dual-stack destination whose first validated address is unreachable +/// through the corporate proxy still connects via a later one. All attempts +/// share one aggregate handshake budget ([`CONNECT_HANDSHAKE_TIMEOUT`]), +/// matching the single-attempt paths; within it, each attempt is capped at +/// its fair share of the remaining time (`remaining / attempts_left`) so a +/// proxy that accepts a CONNECT but never answers cannot starve the +/// remaining validated addresses. Time an attempt fails without using +/// rolls over to later attempts. +/// +/// # Errors +/// +/// Returns an error when `addrs` is empty, or when every attempt fails or +/// times out (the last attempt's error, annotated with the attempt count +/// when there were several). +pub async fn connect_via_validated( + endpoint: &ProxyEndpoint, + host: &str, + port: u16, + addrs: &[SocketAddr], +) -> std::io::Result { + connect_via_validated_with_budget(endpoint, host, port, addrs, CONNECT_HANDSHAKE_TIMEOUT).await +} + +/// [`connect_via_validated`] with an explicit aggregate budget, split out so +/// tests can exercise the hang-fallback behavior without real 30s waits. +async fn connect_via_validated_with_budget( + endpoint: &ProxyEndpoint, + host: &str, + port: u16, + addrs: &[SocketAddr], + budget: Duration, +) -> std::io::Result { + if addrs.is_empty() { + return Err(IoError::new( + ErrorKind::InvalidInput, + format!("no validated addresses to CONNECT to for {host}"), + )); + } + let deadline = tokio::time::Instant::now() + budget; + let mut last_err = None; + let mut attempted = 0usize; + for (index, addr) in addrs.iter().enumerate() { + let remaining = deadline.saturating_duration_since(tokio::time::Instant::now()); + if remaining.is_zero() { + break; + } + let attempts_left = u32::try_from(addrs.len() - index).unwrap_or(u32::MAX); + let attempt_budget = remaining / attempts_left; + attempted += 1; + match tokio::time::timeout( + attempt_budget, + connect_via_inner(endpoint, host, port, ConnectTarget::Ip(addr.ip())), + ) + .await + { + Ok(Ok(stream)) => return Ok(stream), + Ok(Err(err)) => last_err = Some(err), + Err(_) => { + last_err = Some(IoError::new( + ErrorKind::TimedOut, + format!( + "upstream proxy {} CONNECT to {} timed out", + endpoint.display_addr(), + addr.ip(), + ), + )); + } + } + } + // The first iteration always runs (`remaining` starts at the full + // budget), so at least one attempt recorded an error. + let last = last_err.expect("at least one CONNECT attempt"); + if addrs.len() == 1 { + return Err(last); + } + let scope = if attempted == addrs.len() { + format!("all {} validated addresses", addrs.len()) + } else { + format!( + "{attempted} of {} validated addresses (handshake budget exhausted)", + addrs.len() + ) + }; + Err(IoError::new( + last.kind(), + format!("CONNECT failed for {scope} of {host}; last: {last}"), + )) +} + +async fn connect_via_inner( + endpoint: &ProxyEndpoint, + host: &str, + port: u16, + target: ConnectTarget, +) -> std::io::Result { + let mut stream = TcpStream::connect((endpoint.host.as_str(), endpoint.port)).await?; + + let target = match target { + ConnectTarget::Ip(IpAddr::V6(ip)) => format!("[{ip}]:{port}"), + ConnectTarget::Ip(ip) => format!("{ip}:{port}"), + ConnectTarget::Hostname if host.contains(':') => format!("[{host}]:{port}"), + ConnectTarget::Hostname => format!("{host}:{port}"), + }; + let mut request = format!("CONNECT {target} HTTP/1.1\r\nHost: {target}\r\n"); + if let Some(auth) = &endpoint.proxy_authorization { + request.push_str("Proxy-Authorization: "); + request.push_str(auth); + request.push_str("\r\n"); + } + request.push_str("\r\n"); + stream.write_all(request.as_bytes()).await?; + + // Read the proxy's response header block. A read may run past the + // `\r\n\r\n` terminator into tunneled payload; those bytes are preserved + // below, never discarded. + let mut buf = vec![0u8; MAX_CONNECT_RESPONSE_BYTES]; + let mut used = 0; + let header_end = loop { + if used == buf.len() { + return Err(IoError::other(format!( + "upstream proxy {} CONNECT response headers exceed {MAX_CONNECT_RESPONSE_BYTES} bytes", + endpoint.display_addr() + ))); + } + let n = stream.read(&mut buf[used..]).await?; + if n == 0 { + return Err(IoError::new( + ErrorKind::UnexpectedEof, + format!( + "upstream proxy {} closed the connection during CONNECT", + endpoint.display_addr() + ), + )); + } + used += n; + if let Some(pos) = buf[..used].windows(4).position(|win| win == b"\r\n\r\n") { + break pos + 4; + } + }; + + let response = String::from_utf8_lossy(&buf[..header_end]); + let status_line = response.lines().next().unwrap_or_default(); + let status_code = status_line + .split_whitespace() + .nth(1) + .and_then(|code| code.parse::().ok()); + match status_code { + Some(200) => { + debug!( + proxy = %endpoint.display_addr(), + target = %target, + "upstream proxy CONNECT tunnel established" + ); + buf.truncate(used); + let overflow = buf.split_off(header_end); + Ok(PrefixedStream::new(stream, overflow)) + } + Some(code) => Err(IoError::other(format!( + "upstream proxy {} refused CONNECT to {target}: HTTP {code}", + endpoint.display_addr() + ))), + None => Err(IoError::new( + ErrorKind::InvalidData, + format!( + "upstream proxy {} sent a malformed CONNECT response", + endpoint.display_addr() + ), + )), + } +} + +#[cfg(test)] +mod tests { + use super::*; + use super::{ + ARG_HTTPS_PROXY as HTTPS_PROXY, ARG_NO_PROXY as NO_PROXY, + ARG_PROXY_AUTH_ALLOW_INSECURE as PROXY_AUTH_ALLOW_INSECURE, + ARG_PROXY_AUTH_FILE as PROXY_AUTH_FILE, + ARG_PROXY_CONNECT_BY_HOSTNAME as PROXY_CONNECT_BY_HOSTNAME, + }; + + fn config_from(pairs: &[(&str, &str)]) -> Result, String> { + UpstreamProxyConfig::from_lookup(|name| { + pairs + .iter() + .find(|(k, _)| *k == name) + .map(|(_, v)| (*v).to_string()) + }) + } + + /// Shorthand for tests exercising a configuration that must load. + fn config_ok(pairs: &[(&str, &str)]) -> UpstreamProxyConfig { + config_from(pairs).unwrap().unwrap() + } + + #[test] + fn no_env_yields_none() { + assert!(config_from(&[]).unwrap().is_none()); + } + + #[test] + fn from_args_maps_cli_arguments_to_config() { + // The driver-supplied argv is the authoritative source; the shared + // validation applies exactly as for the reserved-name lookup. + let args = UpstreamProxyArgs { + https_proxy: Some("http://proxy.corp.com:8080".to_string()), + no_proxy: Some("*.svc.cluster.local".to_string()), + proxy_connect_by_hostname: true, + ..UpstreamProxyArgs::default() + }; + let cfg = UpstreamProxyConfig::from_args(&args).unwrap().unwrap(); + assert!(cfg.connect_by_hostname()); + assert!(bypasses(&cfg, "kubernetes.default.svc.cluster.local")); + + // No proxy URL means no configuration. + assert!( + UpstreamProxyConfig::from_args(&UpstreamProxyArgs::default()) + .unwrap() + .is_none() + ); + } + + #[test] + fn from_args_enforces_the_auth_file_acknowledgement_pairing() { + // An auth file without the acknowledgement is fatal, exactly as when + // the same pairing arrives via the reserved names. + let args = UpstreamProxyArgs { + https_proxy: Some("http://proxy.corp.com:8080".to_string()), + proxy_auth_file: Some("/etc/openshell/auth/upstream-proxy".to_string()), + ..UpstreamProxyArgs::default() + }; + let err = UpstreamProxyConfig::from_args(&args).unwrap_err(); + assert!(err.contains(PROXY_AUTH_ALLOW_INSECURE), "{err}"); + + // Auxiliary settings without a proxy are fatal. + let args = UpstreamProxyArgs { + proxy_connect_by_hostname: true, + ..UpstreamProxyArgs::default() + }; + let err = UpstreamProxyConfig::from_args(&args).unwrap_err(); + assert!(err.contains("no upstream proxy"), "{err}"); + } + + #[test] + fn conventional_proxy_vars_are_ignored() { + // The sandbox creator controls these names; they must not steer the + // supervisor's operator-owned egress boundary. + assert!( + config_from(&[ + ("HTTPS_PROXY", "http://attacker:9999"), + ("HTTP_PROXY", "http://attacker:9999"), + ("ALL_PROXY", "http://attacker:9999"), + ("https_proxy", "http://attacker:9999"), + ]) + .unwrap() + .is_none() + ); + } + + #[test] + fn present_but_empty_values_are_fatal() { + // A reserved variable the operator did not set is absent, never + // empty: the driver only writes configured values. A present-but + // -blank value is therefore a misconfiguration and must not silently + // mean "no proxy". + for (name, value) in [ + (HTTPS_PROXY, ""), + (HTTPS_PROXY, " "), + (NO_PROXY, " "), + (PROXY_AUTH_FILE, ""), + (PROXY_AUTH_ALLOW_INSECURE, ""), + ] { + let err = config_from(&[(name, value)]).unwrap_err(); + assert!(err.contains(name), "{err}"); + assert!(err.contains("empty"), "{err}"); + } + } + + /// Shorthand: the proxy endpoint chosen for `host:443` with no resolved + /// addresses in play, or `None` when the destination dials directly. + fn proxy_endpoint<'a>(cfg: &'a UpstreamProxyConfig, host: &str) -> Option<&'a ProxyEndpoint> { + match cfg.decision(host, 443, &[]) { + ProxyDecision::Proxy(ep) => Some(ep), + ProxyDecision::Direct(_) => None, + } + } + + #[test] + fn https_proxy_parsed_with_port() { + let cfg = config_ok(&[(HTTPS_PROXY, "http://proxy.corp.com:8080")]); + let ep = proxy_endpoint(&cfg, "api.stripe.com").unwrap(); + assert_eq!(ep.display_addr(), "proxy.corp.com:8080"); + assert!(ep.proxy_authorization.is_none()); + } + + #[test] + fn scheme_less_or_port_less_proxy_url_is_fatal() { + // The accepted grammar is exactly `http://host:port`; lenient + // defaulting would let a typo silently target the wrong proxy. + for url in [ + "proxy.corp.com", + "proxy.corp.com:3128", + "http://proxy.corp.com", + ] { + let err = config_from(&[(HTTPS_PROXY, url)]).unwrap_err(); + assert!(err.contains(HTTPS_PROXY), "{url}: {err}"); + assert!(err.contains("explicit"), "{url}: {err}"); + } + } + + // -- Fail-closed configuration validation -- + // + // Present-but-invalid reserved values must be fatal, never silently + // treated as unset: a typo must not downgrade the operator's egress + // boundary to direct dialing or unauthenticated proxy access. + + #[test] + fn tls_and_socks_proxies_are_fatal() { + for url in ["https://proxy:443", "socks5://proxy:1080"] { + let err = config_from(&[(HTTPS_PROXY, url)]).unwrap_err(); + assert!(err.contains(HTTPS_PROXY), "{err}"); + assert!(err.contains("scheme"), "{err}"); + } + } + + #[test] + fn url_userinfo_is_fatal_not_used_as_credentials() { + // Inline credentials in the URL must never become the proxy auth; + // credentials come only from the auth file. Matching the compute + // driver, a URL that embeds them is rejected outright. + let err = config_from(&[(HTTPS_PROXY, "http://user:secret@proxy:8080")]).unwrap_err(); + assert!(err.contains(HTTPS_PROXY), "{err}"); + assert!( + !err.contains("secret"), + "error must not leak the credential: {err}" + ); + } + + #[test] + fn malformed_proxy_address_is_fatal() { + let err = config_from(&[(HTTPS_PROXY, "http://proxy:notaport")]).unwrap_err(); + assert!(err.contains(HTTPS_PROXY), "{err}"); + // A path/query/fragment addresses an endpoint, not a proxy; it is + // rejected rather than silently discarded. + for url in [ + "http://proxy:8080/path", + "http://proxy:8080?x=1", + "http://proxy:8080#frag", + ] { + let err = config_from(&[(HTTPS_PROXY, url)]).unwrap_err(); + assert!(err.contains(HTTPS_PROXY), "{url}: {err}"); + } + } + + #[test] + fn unreadable_auth_file_is_fatal() { + let err = config_from(&[ + (HTTPS_PROXY, "http://proxy.corp.com:8080"), + (PROXY_AUTH_FILE, "/nonexistent/upstream-proxy-auth"), + (PROXY_AUTH_ALLOW_INSECURE, "true"), + ]) + .unwrap_err(); + assert!(err.contains("auth file"), "{err}"); + } + + #[test] + fn auth_file_without_insecure_acknowledgement_is_fatal() { + // Basic auth over the plain-TCP proxy connection is readable on the + // network path; sending it requires the explicit opt-in. + let err = config_from(&[ + (HTTPS_PROXY, "http://proxy.corp.com:8080"), + (PROXY_AUTH_FILE, "/etc/openshell/auth/upstream-proxy"), + ]) + .unwrap_err(); + assert!(err.contains(PROXY_AUTH_ALLOW_INSECURE), "{err}"); + assert!(err.contains("cleartext"), "{err}"); + } + + #[test] + fn invalid_insecure_acknowledgement_value_is_fatal() { + for value in ["false", "yes", "1", "TRUE"] { + let err = config_from(&[ + (HTTPS_PROXY, "http://proxy.corp.com:8080"), + (PROXY_AUTH_FILE, "/etc/openshell/auth/upstream-proxy"), + (PROXY_AUTH_ALLOW_INSECURE, value), + ]) + .unwrap_err(); + assert!(err.contains(PROXY_AUTH_ALLOW_INSECURE), "{value}: {err}"); + } + } + + #[test] + fn insecure_acknowledgement_without_auth_file_is_fatal() { + let err = config_from(&[ + (HTTPS_PROXY, "http://proxy.corp.com:8080"), + (PROXY_AUTH_ALLOW_INSECURE, "true"), + ]) + .unwrap_err(); + assert!(err.contains(PROXY_AUTH_ALLOW_INSECURE), "{err}"); + assert!(err.contains(PROXY_AUTH_FILE), "{err}"); + } + + #[test] + fn malformed_auth_file_credential_is_fatal() { + // Empty, header-injecting, and non-`user:pass` credentials are all + // rejected by the parser shared with the compute driver. + for credential in [" ", "user:pa\r\nss", "userpass", ":pass"] { + let file = tempfile::NamedTempFile::new().unwrap(); + std::fs::write(file.path(), credential).unwrap(); + let path = file.path().to_str().unwrap().to_string(); + let err = config_from(&[ + (HTTPS_PROXY, "http://proxy.corp.com:8080"), + (PROXY_AUTH_FILE, &path), + (PROXY_AUTH_ALLOW_INSECURE, "true"), + ]) + .unwrap_err(); + assert!(err.contains("auth file"), "{err}"); + assert!( + !err.contains("pa\r\nss"), + "error must not leak the credential: {err}" + ); + } + } + + #[test] + fn auxiliary_settings_without_proxy_are_fatal() { + // An auth file or NO_PROXY list only makes sense relative to a proxy + // boundary the operator believed was in effect. + for (name, value) in [ + (PROXY_AUTH_FILE, "/etc/openshell/auth/upstream-proxy"), + (PROXY_AUTH_ALLOW_INSECURE, "true"), + (NO_PROXY, "*.svc.cluster.local"), + ] { + let err = config_from(&[(name, value)]).unwrap_err(); + assert!(err.contains(name), "{err}"); + assert!(err.contains("no upstream proxy"), "{err}"); + } + } + + #[test] + fn auth_file_credentials_are_applied_to_the_endpoint() { + let file = tempfile::NamedTempFile::new().unwrap(); + std::fs::write(file.path(), "user:secret\n").unwrap(); + let path = file.path().to_str().unwrap().to_string(); + let cfg = config_ok(&[ + (HTTPS_PROXY, "http://proxy:8080"), + (PROXY_AUTH_FILE, &path), + (PROXY_AUTH_ALLOW_INSECURE, "true"), + ]); + let expected = format!( + "Basic {}", + base64::engine::general_purpose::STANDARD.encode("user:secret") + ); + let ep = proxy_endpoint(&cfg, "example.com").unwrap(); + assert_eq!(ep.proxy_authorization.as_deref(), Some(expected.as_str())); + } + + #[test] + fn basic_auth_header_encodes_and_rejects_malformed_credentials() { + assert_eq!( + basic_auth_header("user:p@ss").as_deref(), + Ok(format!( + "Basic {}", + base64::engine::general_purpose::STANDARD.encode("user:p@ss") + ) + .as_str()) + ); + assert!(basic_auth_header(" ").is_err()); + assert!(basic_auth_header("user:pa\r\nss").is_err()); + assert!(basic_auth_header("user:pa\nInjected: header").is_err()); + assert!(basic_auth_header("no-separator").is_err()); + } + + #[test] + fn debug_output_hides_credentials() { + let mut cfg = config_ok(&[(HTTPS_PROXY, "http://proxy:8080")]); + cfg.https.proxy_authorization = Some(basic_auth_header("user:secret").unwrap()); + let debug = format!("{cfg:?}"); + assert!(!debug.contains("secret")); + assert!(!cfg.summary().contains("secret")); + } + + #[test] + fn ipv6_proxy_address_parses() { + let cfg = config_ok(&[(HTTPS_PROXY, "http://[fd00::1]:8080")]); + let ep = proxy_endpoint(&cfg, "example.com").unwrap(); + assert_eq!(ep.display_addr(), "fd00::1:8080"); + } + + // -- NO_PROXY matching -- + + fn no_proxy_cfg(no_proxy: &str) -> UpstreamProxyConfig { + config_ok(&[(HTTPS_PROXY, "http://proxy:8080"), (NO_PROXY, no_proxy)]) + } + + /// Hostname-level bypass check at an arbitrary port with no resolved + /// addresses in play. + fn bypasses(cfg: &UpstreamProxyConfig, host: &str) -> bool { + bypasses_port(cfg, host, 443) + } + + fn bypasses_port(cfg: &UpstreamProxyConfig, host: &str, port: u16) -> bool { + matches!(cfg.decision(host, port, &[]), ProxyDecision::Direct(_)) + } + + fn sock(ip: &str, port: u16) -> SocketAddr { + SocketAddr::new(ip.parse().unwrap(), port) + } + + #[test] + fn no_proxy_domain_suffix_matches_host_and_subdomains() { + let cfg = no_proxy_cfg("corp.com,other.example"); + assert!(bypasses(&cfg, "corp.com")); + assert!(bypasses(&cfg, "api.corp.com")); + assert!(!bypasses(&cfg, "notcorp.com")); + assert!(!bypasses(&cfg, "corp.com.evil.io")); + } + + #[test] + fn no_proxy_leading_dot_and_wildcard_prefix_are_equivalent() { + for entry in [".svc.cluster.local", "*.svc.cluster.local"] { + let cfg = no_proxy_cfg(entry); + assert!( + bypasses(&cfg, "kubernetes.default.svc.cluster.local"), + "{entry}" + ); + assert!(!bypasses(&cfg, "example.com"), "{entry}"); + } + } + + #[test] + fn no_proxy_cidr_matches_ip_literals() { + let cfg = no_proxy_cfg("10.96.0.0/12"); + assert!(bypasses(&cfg, "10.96.0.1")); + assert!(bypasses(&cfg, "10.100.20.30")); + assert!(!bypasses(&cfg, "10.200.0.9")); + assert!(!bypasses(&cfg, "93.184.216.34")); + } + + #[test] + fn no_proxy_exact_ip_matches() { + let cfg = no_proxy_cfg("192.168.1.5"); + assert!(bypasses(&cfg, "192.168.1.5")); + assert!(!bypasses(&cfg, "192.168.1.6")); + } + + #[test] + fn no_proxy_wildcard_bypasses_everything() { + let cfg = no_proxy_cfg("*"); + assert!(bypasses(&cfg, "example.com")); + } + + #[test] + fn no_proxy_port_qualifier_limits_bypass_to_that_port() { + // `internal.corp:8443` documents a bypass for one port; broadening + // it to every port would bypass the proxy for traffic the operator + // never excluded. + let cfg = no_proxy_cfg("internal.corp:8443"); + assert!(bypasses_port(&cfg, "internal.corp", 8443)); + assert!(bypasses_port(&cfg, "svc.internal.corp", 8443)); + assert!(!bypasses_port(&cfg, "internal.corp", 443)); + assert!(!bypasses_port(&cfg, "svc.internal.corp", 80)); + } + + #[test] + fn no_proxy_port_qualifier_applies_to_ip_and_cidr_entries() { + let cfg = no_proxy_cfg("192.168.1.5:8443,10.96.0.0/12:6443"); + assert!(bypasses_port(&cfg, "192.168.1.5", 8443)); + assert!(!bypasses_port(&cfg, "192.168.1.5", 443)); + assert!(bypasses_port(&cfg, "10.96.0.1", 6443)); + assert!(!bypasses_port(&cfg, "10.96.0.1", 443)); + } + + #[test] + fn no_proxy_invalid_port_qualifier_is_not_stripped() { + // A trailing qualifier that is not a valid port stays part of the + // pattern instead of silently widening the entry to every port. + let cfg = no_proxy_cfg("internal.corp:99999"); + assert!(!bypasses(&cfg, "internal.corp")); + } + + #[test] + fn no_proxy_cidr_matches_resolved_addresses_of_hostnames() { + // The operator's `10.96.0.0/12` bypass covers cluster-internal + // destinations however they are named; a hostname whose validated + // resolution lands in the range must dial directly. + let cfg = no_proxy_cfg("10.96.0.0/12"); + let inside = sock("10.96.0.7", 443); + match cfg.decision("svc.internal", 443, &[inside]) { + ProxyDecision::Direct(addrs) => assert_eq!(addrs, vec![inside]), + ProxyDecision::Proxy(ep) => panic!("expected direct dial, got proxy {ep:?}"), + } + // Resolution outside the range keeps the proxy. + assert!(matches!( + cfg.decision("example.com", 443, &[sock("93.184.216.34", 443)]), + ProxyDecision::Proxy(_) + )); + } + + #[test] + fn no_proxy_resolved_address_match_limits_direct_dial_to_matching_addrs() { + // Split resolution: only the addresses inside the bypassed range may + // be dialed directly; the others are not covered by the operator's + // exclusion and must not ride along. + let cfg = no_proxy_cfg("10.96.0.0/12"); + let inside = sock("10.96.0.7", 443); + let outside = sock("93.184.216.34", 443); + match cfg.decision("svc.internal", 443, &[outside, inside]) { + ProxyDecision::Direct(addrs) => assert_eq!(addrs, vec![inside]), + ProxyDecision::Proxy(ep) => panic!("expected direct dial, got proxy {ep:?}"), + } + } + + #[test] + fn no_proxy_ip_entry_matches_resolved_addresses() { + let cfg = no_proxy_cfg("10.0.0.5"); + let matching = sock("10.0.0.5", 443); + match cfg.decision("db.internal", 443, &[matching, sock("10.0.0.6", 443)]) { + ProxyDecision::Direct(addrs) => assert_eq!(addrs, vec![matching]), + ProxyDecision::Proxy(ep) => panic!("expected direct dial, got proxy {ep:?}"), + } + } + + #[test] + fn no_proxy_bracketed_ipv6_entry_honors_port_qualifier() { + // The colons of an IPv6 literal must not be misread as a port + // qualifier; only the bracketed form can carry one. + let cfg = no_proxy_cfg("[fd00::1]:8443"); + assert!(bypasses_port(&cfg, "fd00::1", 8443)); + assert!(bypasses_port(&cfg, "[fd00::1]", 8443)); + assert!(!bypasses_port(&cfg, "fd00::1", 443)); + assert!(!bypasses_port(&cfg, "fd00::2", 8443)); + + // A bare IPv6 entry keeps every-port semantics. + let cfg = no_proxy_cfg("fd00::1"); + assert!(bypasses_port(&cfg, "fd00::1", 8443)); + assert!(bypasses_port(&cfg, "fd00::1", 443)); + } + + #[test] + fn no_proxy_ipv6_cidr_matches_resolved_addresses_of_hostnames() { + let cfg = no_proxy_cfg("fd00::/8"); + // An IPv6-literal host inside the range bypasses at hostname level. + assert!(bypasses(&cfg, "fd00::7")); + assert!(!bypasses(&cfg, "2001:db8::1")); + + // A hostname resolving into the range dials directly, restricted to + // the resolved addresses the entry contains. + let inside = sock("fd00::42", 443); + let outside = sock("2001:db8::1", 443); + match cfg.decision("svc.internal", 443, &[outside, inside]) { + ProxyDecision::Direct(addrs) => assert_eq!(addrs, vec![inside]), + ProxyDecision::Proxy(ep) => panic!("expected direct dial, got proxy {ep:?}"), + } + } + + #[test] + fn no_proxy_ipv6_cidr_honors_port_qualifier() { + let cfg = no_proxy_cfg("fd00::/8:6443"); + assert!(bypasses_port(&cfg, "fd00::7", 6443)); + assert!(!bypasses_port(&cfg, "fd00::7", 443)); + match cfg.decision("svc.internal", 6443, &[sock("fd00::42", 6443)]) { + ProxyDecision::Direct(addrs) => assert_eq!(addrs, vec![sock("fd00::42", 6443)]), + ProxyDecision::Proxy(ep) => panic!("expected direct dial, got proxy {ep:?}"), + } + assert!(matches!( + cfg.decision("svc.internal", 443, &[sock("fd00::42", 443)]), + ProxyDecision::Proxy(_) + )); + } + + #[test] + fn no_proxy_hostname_match_authorizes_all_resolved_addresses() { + let cfg = no_proxy_cfg("internal.corp"); + let addrs = [sock("10.0.0.5", 443), sock("93.184.216.34", 443)]; + match cfg.decision("internal.corp", 443, &addrs) { + ProxyDecision::Direct(direct) => assert_eq!(direct, addrs.to_vec()), + ProxyDecision::Proxy(ep) => panic!("expected direct dial, got proxy {ep:?}"), + } + } + + #[test] + fn loopback_and_localhost_always_bypass() { + // No NO_PROXY at all: loopback still bypasses unconditionally. + let cfg = config_ok(&[(HTTPS_PROXY, "http://proxy:8080")]); + assert!(bypasses(&cfg, "localhost")); + assert!(bypasses(&cfg, "127.0.0.1")); + assert!(bypasses(&cfg, "::1")); + assert!(!bypasses(&cfg, "example.com")); + } + + // -- CONNECT handshake -- + + async fn fake_proxy(response: &'static str) -> (SocketAddr, tokio::task::JoinHandle) { + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let addr = listener.local_addr().unwrap(); + let handle = tokio::spawn(async move { + let (mut socket, _) = listener.accept().await.unwrap(); + let request = read_request(&mut socket).await; + socket.write_all(response.as_bytes()).await.unwrap(); + request + }); + (addr, handle) + } + + fn endpoint_for(addr: SocketAddr, auth: Option<&str>) -> ProxyEndpoint { + ProxyEndpoint { + host: addr.ip().to_string(), + port: addr.port(), + proxy_authorization: auth.map(str::to_string), + } + } + + #[tokio::test] + async fn connect_via_targets_validated_ip_by_default() { + // The default CONNECT target is a validated address, so the proxy + // performs no DNS resolution; the hostname only travels inside the + // tunnel (TLS SNI, application Host). + let (addr, handle) = fake_proxy("HTTP/1.1 200 Connection established\r\n\r\n").await; + let endpoint = endpoint_for(addr, Some("Basic dXNlcjpwYXNz")); + let stream = connect_via( + &endpoint, + "api.example.com", + 443, + ConnectTarget::Ip("93.184.216.34".parse().unwrap()), + ) + .await + .unwrap(); + drop(stream); + let request = handle.await.unwrap(); + assert!(request.starts_with("CONNECT 93.184.216.34:443 HTTP/1.1\r\n")); + assert!(request.contains("Host: 93.184.216.34:443\r\n")); + assert!( + !request.contains("api.example.com"), + "hostname must not leak into the CONNECT request in IP mode: {request}" + ); + assert!(request.contains("Proxy-Authorization: Basic dXNlcjpwYXNz\r\n")); + } + + /// Read one request's header block from a fake-proxy socket. + async fn read_request(socket: &mut TcpStream) -> String { + let mut buf = vec![0u8; 4096]; + let mut used = 0; + loop { + let n = socket.read(&mut buf[used..]).await.unwrap(); + used += n; + if n == 0 || buf[..used].windows(4).any(|w| w == b"\r\n\r\n") { + break; + } + } + String::from_utf8_lossy(&buf[..used]).into_owned() + } + + #[tokio::test] + async fn connect_via_validated_falls_back_across_validated_addresses() { + // The direct path tries every validated address; the proxied path + // must too, or a dual-stack destination whose first address is + // unreachable through the proxy fails needlessly. + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let addr = listener.local_addr().unwrap(); + let handle = tokio::spawn(async move { + // First attempt: refuse the tunnel. + let (mut refused, _) = listener.accept().await.unwrap(); + let first = read_request(&mut refused).await; + refused + .write_all(b"HTTP/1.1 502 Bad Gateway\r\n\r\n") + .await + .unwrap(); + drop(refused); + // Second attempt: establish it. + let (mut accepted, _) = listener.accept().await.unwrap(); + let second = read_request(&mut accepted).await; + accepted + .write_all(b"HTTP/1.1 200 Connection established\r\n\r\n") + .await + .unwrap(); + (first, second, accepted) + }); + + let endpoint = endpoint_for(addr, None); + let addrs = [sock("192.0.2.1", 443), sock("192.0.2.2", 443)]; + let stream = connect_via_validated(&endpoint, "api.example.com", 443, &addrs) + .await + .unwrap(); + let (first, second, _accepted) = handle.await.unwrap(); + drop(stream); + assert!( + first.starts_with("CONNECT 192.0.2.1:443 HTTP/1.1\r\n"), + "{first}" + ); + assert!( + second.starts_with("CONNECT 192.0.2.2:443 HTTP/1.1\r\n"), + "{second}" + ); + } + + #[tokio::test] + async fn connect_via_validated_survives_a_hanging_first_attempt() { + // A proxy that accepts the CONNECT but never answers must not + // consume the whole aggregate budget: the per-address cap moves on + // to the next validated address. + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let addr = listener.local_addr().unwrap(); + let handle = tokio::spawn(async move { + // First attempt: read the CONNECT, then hang with the socket + // held open until the client times the attempt out. + let (mut hung, _) = listener.accept().await.unwrap(); + let first = read_request(&mut hung).await; + // Second attempt: establish the tunnel. + let (mut accepted, _) = listener.accept().await.unwrap(); + let second = read_request(&mut accepted).await; + accepted + .write_all(b"HTTP/1.1 200 Connection established\r\n\r\n") + .await + .unwrap(); + drop(hung); + (first, second, accepted) + }); + + let endpoint = endpoint_for(addr, None); + let addrs = [sock("192.0.2.1", 443), sock("192.0.2.2", 443)]; + // Small real-time budget: the first attempt hangs for its fair + // share (half), then the second succeeds within the remainder. + let stream = connect_via_validated_with_budget( + &endpoint, + "api.example.com", + 443, + &addrs, + Duration::from_secs(2), + ) + .await + .unwrap(); + let (first, second, _accepted) = handle.await.unwrap(); + drop(stream); + assert!( + first.starts_with("CONNECT 192.0.2.1:443 HTTP/1.1\r\n"), + "{first}" + ); + assert!( + second.starts_with("CONNECT 192.0.2.2:443 HTTP/1.1\r\n"), + "{second}" + ); + } + + #[tokio::test] + async fn connect_via_validated_reports_failure_across_all_addresses() { + let (addr, _handle) = fake_proxy("HTTP/1.1 502 Bad Gateway\r\n\r\n").await; + let endpoint = endpoint_for(addr, None); + // Only the first attempt gets a response; the second finds the + // listener gone, and the aggregate error must say both were tried. + let addrs = [sock("192.0.2.1", 443), sock("192.0.2.2", 443)]; + let err = connect_via_validated(&endpoint, "api.example.com", 443, &addrs) + .await + .unwrap_err(); + assert!( + err.to_string().contains("all 2 validated addresses"), + "{err}" + ); + } + + #[tokio::test] + async fn connect_via_validated_rejects_empty_address_list() { + let endpoint = endpoint_for(sock("127.0.0.1", 3128), None); + let err = connect_via_validated(&endpoint, "api.example.com", 443, &[]) + .await + .unwrap_err(); + assert_eq!(err.kind(), ErrorKind::InvalidInput); + } + + #[tokio::test] + async fn connect_via_sends_hostname_on_operator_opt_in() { + let (addr, handle) = fake_proxy("HTTP/1.1 200 Connection established\r\n\r\n").await; + let endpoint = endpoint_for(addr, None); + let stream = connect_via(&endpoint, "api.example.com", 443, ConnectTarget::Hostname) + .await + .unwrap(); + drop(stream); + let request = handle.await.unwrap(); + assert!(request.starts_with("CONNECT api.example.com:443 HTTP/1.1\r\n")); + assert!(request.contains("Host: api.example.com:443\r\n")); + } + + #[tokio::test] + async fn connect_via_preserves_payload_after_connect_response() { + // The proxy's 200 response and the first tunneled bytes can arrive + // in a single read; the suffix belongs to the destination stream and + // must be replayed, not discarded. + let (addr, _handle) = + fake_proxy("HTTP/1.1 200 Connection established\r\n\r\nserver-first-payload").await; + let endpoint = endpoint_for(addr, None); + let mut stream = connect_via(&endpoint, "api.example.com", 443, ConnectTarget::Hostname) + .await + .unwrap(); + let mut payload = vec![0u8; "server-first-payload".len()]; + stream.read_exact(&mut payload).await.unwrap(); + assert_eq!(payload, b"server-first-payload"); + } + + #[tokio::test] + async fn prefixed_stream_replays_prefix_before_socket_bytes() { + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let addr = listener.local_addr().unwrap(); + let connect = tokio::spawn(async move { TcpStream::connect(addr).await.unwrap() }); + let (mut accepted, _) = listener.accept().await.unwrap(); + let dialed = connect.await.unwrap(); + + accepted.write_all(b" from-socket").await.unwrap(); + let mut stream = PrefixedStream::new(dialed, b"from-prefix".to_vec()); + let mut out = vec![0u8; "from-prefix from-socket".len()]; + stream.read_exact(&mut out).await.unwrap(); + assert_eq!(out, b"from-prefix from-socket"); + + // Writes pass through to the socket untouched by the prefix. + stream.write_all(b"reply").await.unwrap(); + let mut reply = vec![0u8; 5]; + accepted.read_exact(&mut reply).await.unwrap(); + assert_eq!(reply, b"reply"); + } + + #[tokio::test] + async fn connect_via_rejects_non_200() { + let (addr, _handle) = + fake_proxy("HTTP/1.1 407 Proxy Authentication Required\r\n\r\n").await; + let endpoint = endpoint_for(addr, None); + let err = connect_via(&endpoint, "api.example.com", 443, ConnectTarget::Hostname) + .await + .unwrap_err(); + assert!(err.to_string().contains("407"), "{err}"); + } + + #[tokio::test] + async fn connect_via_rejects_malformed_response() { + let (addr, _handle) = fake_proxy("garbage without status\r\n\r\n").await; + let endpoint = endpoint_for(addr, None); + let err = connect_via(&endpoint, "api.example.com", 443, ConnectTarget::Hostname) + .await + .unwrap_err(); + assert_eq!(err.kind(), ErrorKind::InvalidData); + } + + #[tokio::test] + async fn connect_via_ipv6_target_is_bracketed() { + // Both modes must bracket IPv6 authorities in the request line. + let (addr, handle) = fake_proxy("HTTP/1.1 200 OK\r\n\r\n").await; + let endpoint = endpoint_for(addr, None); + let _ = connect_via( + &endpoint, + "v6.example.com", + 443, + ConnectTarget::Ip("2001:db8::1".parse().unwrap()), + ) + .await + .unwrap(); + let request = handle.await.unwrap(); + assert!(request.starts_with("CONNECT [2001:db8::1]:443 HTTP/1.1\r\n")); + + let (addr, handle) = fake_proxy("HTTP/1.1 200 OK\r\n\r\n").await; + let endpoint = endpoint_for(addr, None); + let _ = connect_via(&endpoint, "2001:db8::1", 443, ConnectTarget::Hostname) + .await + .unwrap(); + let request = handle.await.unwrap(); + assert!(request.starts_with("CONNECT [2001:db8::1]:443 HTTP/1.1\r\n")); + } + + #[test] + fn connect_by_hostname_requires_exact_true() { + // Default is the validated-IP binding. + let cfg = config_ok(&[(HTTPS_PROXY, "http://proxy:8080")]); + assert!(!cfg.connect_by_hostname()); + + let cfg = config_ok(&[ + (HTTPS_PROXY, "http://proxy:8080"), + (PROXY_CONNECT_BY_HOSTNAME, "true"), + ]); + assert!(cfg.connect_by_hostname()); + + // Anything other than the exact opt-in value the driver writes is a + // misconfiguration, not a silent fallback to either mode. + for value in ["false", "yes", "1", "TRUE"] { + let err = config_from(&[ + (HTTPS_PROXY, "http://proxy:8080"), + (PROXY_CONNECT_BY_HOSTNAME, value), + ]) + .unwrap_err(); + assert!(err.contains(PROXY_CONNECT_BY_HOSTNAME), "{value}: {err}"); + } + } + + #[test] + fn connect_by_hostname_without_proxy_is_fatal() { + let err = config_from(&[(PROXY_CONNECT_BY_HOSTNAME, "true")]).unwrap_err(); + assert!(err.contains(PROXY_CONNECT_BY_HOSTNAME), "{err}"); + assert!(err.contains("no upstream proxy"), "{err}"); + } +} diff --git a/docs/reference/gateway-config.mdx b/docs/reference/gateway-config.mdx index 3dde7643d5..a644f3f602 100644 --- a/docs/reference/gateway-config.mdx +++ b/docs/reference/gateway-config.mdx @@ -389,6 +389,66 @@ sandbox_pids_limit = 2048 # but increase process churn (each check spawns a conmon subprocess). # Set to 0 to disable health checks entirely. Default: 10. health_check_interval_secs = 10 +# Corporate forward proxy for sandbox egress. When set, the in-container +# supervisor chains policy-approved TLS tunnels through this proxy with HTTP +# CONNECT instead of dialing destinations directly. Plain-HTTP requests are +# not proxied and always dial the destination directly. Only http:// proxy +# URLs in explicit http://host:port form are supported: the scheme and port +# are both required, and a URL carrying a path, query, or fragment is +# rejected rather than silently truncated. +# NO_PROXY entries (hostnames, domain suffixes, IPs, CIDRs, each with an +# optional :port qualifier) are dialed directly. A port-qualified entry only +# bypasses that destination port. IP and CIDR entries also match hostnames +# through their validated DNS resolution; such a match dials directly only +# the resolved addresses inside the entry. This is an operator-owned egress +# boundary: +# sandbox and template environment cannot override it, and the conventional +# HTTPS_PROXY/HTTP_PROXY/NO_PROXY variables a sandbox sets do not affect it. +# +# The CONNECT request sent to the proxy targets a validated resolved IP, +# not the hostname, so the proxy performs no DNS resolution of its own and +# the tunnel stays bound to the address that passed the sandbox's SSRF and +# allowed_ips validation. The hostname still travels inside the tunnel (TLS +# SNI, application Host), so destination servers behave normally. In +# split-horizon networks, point the gateway host at the corporate resolver +# so internal names validate to their internal addresses. If the proxy's +# ACLs filter on hostnames and reject IP CONNECT targets, set +# proxy_connect_by_hostname = true as a last resort: the proxy then +# resolves the name itself, so a name that resolves differently at the +# proxy (split-horizon DNS, rebinding) can reach destinations the sandbox +# policy never approved, and the proxy's own ACLs become the effective +# egress control for proxied TLS. +# +# Configuration is fail-closed: an invalid proxy URL is rejected at gateway +# startup, setting no_proxy, proxy_auth_file, or proxy_connect_by_hostname +# without a proxy URL is rejected as well, and a set-but-invalid value +# reaching a sandbox (for example an unreadable or malformed auth file) is +# fatal to that sandbox's supervisor instead of silently falling back to +# direct or unauthenticated egress. +# +# Credentials must NOT be embedded in the URL (an inline user:pass@ is +# rejected at startup, since it would be stored here and exposed in container +# metadata). Instead point proxy_auth_file at a file containing "user:pass"; +# the gateway delivers it to the supervisor through a root-only secret mount. +# The credential must use the user:pass form (non-empty user, no control +# characters); the same validation runs at sandbox-create time and in the +# supervisor, so a credential accepted here is never rejected in-container. +# Keep gateway.toml and the auth file owner-readable only (mode 0600). +# +# WARNING: the supervisor sends the credential as a Proxy-Authorization: +# Basic header over the plain-TCP connection to the http:// proxy. Basic +# auth is base64, not encryption: anyone on the network path between the +# sandbox host and the proxy can recover the credential. Because of that, +# proxy_auth_file requires the explicit acknowledgement +# proxy_auth_allow_insecure = true; without it the configuration is +# rejected at gateway startup. Only opt in when the path to the proxy is a +# trusted network segment. +# https_proxy = "http://proxy.corp.com:8080" +# no_proxy = "*.svc.cluster.local,10.0.0.0/8" +# proxy_auth_file = "/etc/openshell/secrets/proxy-auth" +# proxy_auth_allow_insecure = true +# Last resort for hostname-filtering proxy ACLs; see the warning above. +# proxy_connect_by_hostname = true ``` ### MicroVM diff --git a/tasks/scripts/gateway.sh b/tasks/scripts/gateway.sh index 11fbf9a059..3e94afe108 100644 --- a/tasks/scripts/gateway.sh +++ b/tasks/scripts/gateway.sh @@ -41,6 +41,28 @@ Environment: Podman supervisor sideload image. Defaults to openshell/supervisor:dev and is built on demand. +Corporate proxy (Podman driver only): + OPENSHELL_SANDBOX_HTTPS_PROXY + Corporate forward proxy URL for sandbox TLS + egress, in explicit http://host:port form. + OPENSHELL_SANDBOX_NO_PROXY + Comma-separated NO_PROXY list (hostnames, domain + suffixes, IPs, CIDRs, optional :port qualifiers) + dialed directly instead of through the proxy. + OPENSHELL_SANDBOX_PROXY_AUTH_FILE + Path to a file with proxy credentials as + user:pass. Requires the acknowledgement below — + the gateway refuses to start with one but not + the other. + OPENSHELL_SANDBOX_PROXY_AUTH_ALLOW_INSECURE + Set to true to acknowledge that the credential + is sent as cleartext Basic auth over the + plain-TCP connection to the http:// proxy. + OPENSHELL_SANDBOX_PROXY_CONNECT_BY_HOSTNAME + Set to true to send the destination hostname in + CONNECT instead of a validated IP. Last resort + for hostname-filtering proxy ACLs. + Docker and VM runs delegate to gateway:docker and gateway:vm setup scripts. EOF } @@ -110,6 +132,19 @@ detect_driver() { exit 2 } +# Escape a value for embedding in a double-quoted TOML basic string, so +# quotes, backslashes, or control characters in an environment value cannot +# corrupt gateway.toml or inject extra configuration keys. +toml_escape() { + local s=$1 + s=${s//\\/\\\\} + s=${s//\"/\\\"} + s=${s//$'\n'/\\n} + s=${s//$'\r'/\\r} + s=${s//$'\t'/\\t} + printf '%s' "${s}" +} + port_is_in_use() { local port=$1 if command_available lsof; then @@ -310,6 +345,9 @@ echo "Generating local gateway credentials..." mkdir -p "${STATE_DIR}" CONFIG_PATH="${STATE_DIR}/gateway.toml" +# The config may reference credential-bearing material (e.g. proxy_auth_file); +# keep it owner-only regardless of the ambient umask. +install -m 600 /dev/null "${CONFIG_PATH}" cat >"${CONFIG_PATH}" <>"${CONFIG_PATH}" fi + # ${VAR+x} distinguishes unset from set-but-empty: an unset variable + # writes nothing, but an explicitly empty one is written through so the + # gateway's fail-closed proxy validation rejects it at startup instead of + # this script silently dropping it. + if [[ -n "${OPENSHELL_SANDBOX_HTTPS_PROXY+x}" ]]; then + printf 'https_proxy = "%s"\n' "$(toml_escape "${OPENSHELL_SANDBOX_HTTPS_PROXY}")" >>"${CONFIG_PATH}" + fi + if [[ -n "${OPENSHELL_SANDBOX_NO_PROXY+x}" ]]; then + printf 'no_proxy = "%s"\n' "$(toml_escape "${OPENSHELL_SANDBOX_NO_PROXY}")" >>"${CONFIG_PATH}" + fi + if [[ -n "${OPENSHELL_SANDBOX_PROXY_AUTH_FILE+x}" ]]; then + printf 'proxy_auth_file = "%s"\n' "$(toml_escape "${OPENSHELL_SANDBOX_PROXY_AUTH_FILE}")" >>"${CONFIG_PATH}" + fi + if [[ -n "${OPENSHELL_SANDBOX_PROXY_AUTH_ALLOW_INSECURE+x}" ]]; then + case "${OPENSHELL_SANDBOX_PROXY_AUTH_ALLOW_INSECURE}" in + true|false) + printf 'proxy_auth_allow_insecure = %s\n' "${OPENSHELL_SANDBOX_PROXY_AUTH_ALLOW_INSECURE}" >>"${CONFIG_PATH}" + ;; + *) + # Not a TOML boolean: write it as a quoted string so the gateway's + # config parser rejects it at startup (fail closed, no injection). + printf 'proxy_auth_allow_insecure = "%s"\n' "$(toml_escape "${OPENSHELL_SANDBOX_PROXY_AUTH_ALLOW_INSECURE}")" >>"${CONFIG_PATH}" + ;; + esac + fi + if [[ -n "${OPENSHELL_SANDBOX_PROXY_CONNECT_BY_HOSTNAME+x}" ]]; then + case "${OPENSHELL_SANDBOX_PROXY_CONNECT_BY_HOSTNAME}" in + true|false) + printf 'proxy_connect_by_hostname = %s\n' "${OPENSHELL_SANDBOX_PROXY_CONNECT_BY_HOSTNAME}" >>"${CONFIG_PATH}" + ;; + *) + # Not a TOML boolean: write it as a quoted string so the gateway's + # config parser rejects it at startup (fail closed, no injection). + printf 'proxy_connect_by_hostname = "%s"\n' "$(toml_escape "${OPENSHELL_SANDBOX_PROXY_CONNECT_BY_HOSTNAME}")" >>"${CONFIG_PATH}" + ;; + esac + fi ;; esac