Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions architecture/compute-runtimes.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,12 @@ when a sandbox create request asks for GPU resources.

Per-sandbox CPU and memory values currently enter the driver layer through
template resource limits. Docker and Podman apply them as runtime limits.

Docker and Podman operators can configure `sandbox_uid` and `sandbox_gid` in
their driver tables. When unset, the drivers resolve `10001:10001` and inject
it for the agent child; the root supervisor remains the container entry process.
Rootless Podman requires that the chosen values are available in its subordinate
UID/GID mappings.
Kubernetes mirrors each limit into the matching request. VM accepts the fields
but currently ignores them.

Expand Down
7 changes: 7 additions & 0 deletions architecture/sandbox.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,13 @@ container-granted capabilities. This is fail-closed: the supervisor retains
aborts unless the bounding set ends up empty. A `setpcap` `EPERM` is tolerated
only when the set is already empty; any other outcome fails the spawn.

The compute runtime selects the agent's numeric UID and GID; sandbox creators
do not control it. Docker and Podman start the supervisor as root, inject the
driver-resolved identity, and the supervisor drops only the agent child to that
identity. A container image therefore need not contain a matching passwd or
group entry. Agent children receive a stable `HOME=/sandbox`; numeric identities
use the UID as their `USER` and `LOGNAME` presentation value.

## Startup Flow

1. The compute runtime starts the workload with sandbox identity, callback
Expand Down
2 changes: 1 addition & 1 deletion crates/openshell-core/src/sandbox_env.rs
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ pub const PROVIDER_SPIFFE_WORKLOAD_API_SOCKET: &str =
/// Resolved sandbox UID used to override `run_as_user` when the policy
/// specifies a numeric value instead of the hardcoded "sandbox" user name.
///
/// Set by compute drivers (Kubernetes, Docker, VM) from resolved config or
/// Set by compute drivers (Kubernetes, Docker, Podman, and VM) from resolved config or
/// cluster autodetection. The supervisor reads this at startup and uses it
/// directly with `setuid()` / `chown()` without requiring an `/etc/passwd`
/// entry in the sandbox image.
Expand Down
1 change: 1 addition & 0 deletions crates/openshell-driver-docker/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ repository.workspace = true

[dependencies]
openshell-core = { path = "../openshell-core", default-features = false }
openshell-policy = { path = "../openshell-policy" }

tokio = { workspace = true }
tonic = { workspace = true }
Expand Down
1 change: 1 addition & 0 deletions crates/openshell-driver-docker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ contract:
| `apparmor=unconfined` | Avoids Docker's default profile blocking required mount operations. |
| `restart_policy = unless-stopped` | Keeps managed sandboxes resumable across daemon or gateway restarts. |
| `PidsLimit` | Enforces the sandbox PID budget at the Docker cgroup layer. Set `[openshell.drivers.docker].sandbox_pids_limit = 0` to inherit the Docker/runtime default. |
| `sandbox_uid` / `sandbox_gid` | Operator-controlled numeric identity for the agent child. The supervisor still starts as root; the UID defaults to `10001` and the GID defaults to the resolved UID. |
| CDI GPU request | Uses opaque `driver_config.cdi_devices` values when set; otherwise selects the requested count of NVIDIA CDI GPUs in round-robin order when daemon CDI support is detected. Docker daemon `/info` can permit `nvidia.com/gpu=all` as a WSL2 all-only compatibility fallback, where it counts as one selectable device. Exact CDI device lists must not contain duplicates and must match the effective GPU count. |

The agent child process does not retain these supervisor privileges.
Expand Down
40 changes: 40 additions & 0 deletions crates/openshell-driver-docker/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ const SUPERVISOR_PATH: &str = "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
const HOST_OPENSHELL_INTERNAL: &str = "host.openshell.internal";
const HOST_DOCKER_INTERNAL: &str = "host.docker.internal";
const DOCKER_NETWORK_DRIVER: &str = "bridge";
const DEFAULT_SANDBOX_UID: u32 = 10_001;

/// Queried by the Docker driver to decide when a sandbox's supervisor
/// relay is live. Implementations return `true` once a sandbox has an
Expand Down Expand Up @@ -136,6 +137,14 @@ pub struct DockerComputeConfig {
/// Set to `0` to leave Docker's runtime/default PID limit unchanged.
pub sandbox_pids_limit: i64,

/// Numeric identity used for the agent process. The root supervisor
/// retains UID 0 while preparing the sandbox.
pub sandbox_uid: Option<u32>,

/// Numeric group used for the agent process. Defaults to the resolved
/// sandbox UID when unset.
pub sandbox_gid: Option<u32>,

/// Allow sandbox requests to attach host bind mounts through
/// `template.driver_config`.
#[serde(default)]
Expand All @@ -158,6 +167,8 @@ impl Default for DockerComputeConfig {
host_gateway_ip: String::new(),
ssh_socket_path: "/run/openshell/ssh.sock".to_string(),
sandbox_pids_limit: DEFAULT_SANDBOX_PIDS_LIMIT,
sandbox_uid: None,
sandbox_gid: None,
Comment on lines +170 to +171

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QUestion: Why is the default not DEFAULT_SANDBOX_UID?

enable_bind_mounts: false,
}
}
Expand Down Expand Up @@ -187,6 +198,8 @@ struct DockerDriverRuntimeConfig {
supports_gpu: bool,
allow_all_default_gpu: bool,
sandbox_pids_limit: i64,
sandbox_uid: u32,
sandbox_gid: u32,
enable_bind_mounts: bool,
}

Expand Down Expand Up @@ -307,6 +320,7 @@ impl DockerComputeDriver {
docker_config: &DockerComputeConfig,
supervisor_readiness: Arc<dyn SupervisorReadiness>,
) -> CoreResult<Self> {
let sandbox_identity = resolve_sandbox_identity(docker_config)?;
let docker = Docker::connect_with_local_defaults()
.map_err(|err| Error::execution(format!("failed to create Docker client: {err}")))?;
let version = docker.version().await.map_err(|err| {
Expand Down Expand Up @@ -370,6 +384,8 @@ impl DockerComputeDriver {
supports_gpu,
allow_all_default_gpu,
sandbox_pids_limit: docker_config.sandbox_pids_limit,
sandbox_uid: sandbox_identity.0,
sandbox_gid: sandbox_identity.1,
enable_bind_mounts: docker_config.enable_bind_mounts,
},
events: broadcast::channel(WATCH_BUFFER).0,
Expand Down Expand Up @@ -2162,6 +2178,14 @@ fn build_environment(sandbox: &DriverSandbox, config: &DockerDriverRuntimeConfig
openshell_core::sandbox_env::TELEMETRY_ENABLED.to_string(),
openshell_core::telemetry::enabled_env_value().to_string(),
);
environment.insert(
openshell_core::sandbox_env::SANDBOX_UID.to_string(),
config.sandbox_uid.to_string(),
);
environment.insert(
openshell_core::sandbox_env::SANDBOX_GID.to_string(),
config.sandbox_gid.to_string(),
);
// The root supervisor executes namespace helpers during bootstrap; keep
// their search path driver-owned even when the template/spec set PATH.
environment.insert("PATH".to_string(), SUPERVISOR_PATH.to_string());
Expand Down Expand Up @@ -2619,6 +2643,22 @@ fn validate_sandbox_pids_limit(value: i64) -> CoreResult<()> {
Ok(())
}

fn resolve_sandbox_identity(config: &DockerComputeConfig) -> CoreResult<(u32, u32)> {
let uid = config.sandbox_uid.unwrap_or(DEFAULT_SANDBOX_UID);
let gid = config.sandbox_gid.unwrap_or(uid);
for (field, value) in [("sandbox_uid", uid), ("sandbox_gid", gid)] {
if !(openshell_policy::MIN_SANDBOX_UID..=openshell_policy::MAX_SANDBOX_UID).contains(&value)
{
return Err(Error::config(format!(
"docker {field} must be in [{}, {}]",
openshell_policy::MIN_SANDBOX_UID,
openshell_policy::MAX_SANDBOX_UID,
)));
}
}
Ok((uid, gid))
}

fn docker_pids_limit(value: i64) -> Result<Option<i64>, Status> {
if value < 0 {
return Err(Status::failed_precondition(
Expand Down
68 changes: 68 additions & 0 deletions crates/openshell-driver-docker/src/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,8 @@ fn runtime_config() -> DockerDriverRuntimeConfig {
supports_gpu: false,
allow_all_default_gpu: false,
sandbox_pids_limit: DEFAULT_SANDBOX_PIDS_LIMIT,
sandbox_uid: 10_001,
sandbox_gid: 10_001,
enable_bind_mounts: false,
}
}
Expand Down Expand Up @@ -551,6 +553,50 @@ fn docker_compute_config_disables_bind_mounts_by_default() {
assert!(!cfg.enable_bind_mounts);
}

#[test]
fn docker_sandbox_identity_defaults_to_10001() {
assert_eq!(
resolve_sandbox_identity(&DockerComputeConfig::default()).unwrap(),
(10_001, 10_001)
);
}

#[test]
fn docker_sandbox_identity_uses_resolved_uid_for_default_gid() {
let config = DockerComputeConfig {
sandbox_uid: Some(12_345),
..DockerComputeConfig::default()
};
assert_eq!(resolve_sandbox_identity(&config).unwrap(), (12_345, 12_345));
}

#[test]
fn docker_sandbox_identity_allows_configured_gid_without_uid() {
let config = DockerComputeConfig {
sandbox_gid: Some(12_346),
..DockerComputeConfig::default()
};
assert_eq!(resolve_sandbox_identity(&config).unwrap(), (10_001, 12_346));
}

#[test]
fn docker_sandbox_identity_rejects_system_uid() {
let config = DockerComputeConfig {
sandbox_uid: Some(999),
..DockerComputeConfig::default()
};
assert!(resolve_sandbox_identity(&config).is_err());
}

#[test]
fn docker_sandbox_identity_rejects_gid_above_policy_range() {
let config = DockerComputeConfig {
sandbox_gid: Some(openshell_policy::MAX_SANDBOX_UID + 1),
..DockerComputeConfig::default()
};
assert!(resolve_sandbox_identity(&config).is_err());
}

#[test]
fn container_create_body_sets_driver_owned_pids_limit() {
let body = build_container_create_body(&test_sandbox(), &runtime_config()).unwrap();
Expand All @@ -567,6 +613,28 @@ fn build_environment_sets_docker_tls_paths() {
assert!(env.contains(&"TEMPLATE_ENV=template".to_string()));
assert!(env.contains(&"SPEC_ENV=spec".to_string()));
assert!(env.contains(&"OPENSHELL_SANDBOX_COMMAND=sleep infinity".to_string()));
assert!(env.contains(&"OPENSHELL_SANDBOX_UID=10001".to_string()));
assert!(env.contains(&"OPENSHELL_SANDBOX_GID=10001".to_string()));
}

#[test]
fn build_environment_keeps_sandbox_identity_driver_controlled() {
let mut sandbox = test_sandbox();
let spec = sandbox.spec.as_mut().unwrap();
spec.environment.insert(
openshell_core::sandbox_env::SANDBOX_UID.to_string(),
"1234".to_string(),
);
spec.template.as_mut().unwrap().environment.insert(
openshell_core::sandbox_env::SANDBOX_GID.to_string(),
"1234".to_string(),
);

let env = build_environment(&sandbox, &runtime_config());
assert!(env.contains(&"OPENSHELL_SANDBOX_UID=10001".to_string()));
assert!(env.contains(&"OPENSHELL_SANDBOX_GID=10001".to_string()));
assert!(!env.contains(&"OPENSHELL_SANDBOX_UID=1234".to_string()));
assert!(!env.contains(&"OPENSHELL_SANDBOX_GID=1234".to_string()));
}

#[test]
Expand Down
1 change: 1 addition & 0 deletions crates/openshell-driver-podman/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ path = "src/main.rs"

[dependencies]
openshell-core = { path = "../openshell-core", default-features = false }
openshell-policy = { path = "../openshell-policy" }

tokio = { workspace = true }
tonic = { workspace = true, features = ["transport"] }
Expand Down
6 changes: 6 additions & 0 deletions crates/openshell-driver-podman/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,12 @@ The container spec in `container.rs` sets these security-critical fields:

The restricted agent child does not retain these supervisor privileges.

The gateway operator may set `[openshell.drivers.podman].sandbox_uid` and
`sandbox_gid` for the agent child. The supervisor remains `0:0` during setup;
the UID defaults to `10001` and the GID defaults to the resolved UID. In
rootless mode, configure values that are available in Podman's subordinate
UID/GID mapping.

## Driver Config Mounts

The gateway forwards the `podman` block from `--driver-config-json` to this
Expand Down
74 changes: 74 additions & 0 deletions crates/openshell-driver-podman/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ use std::str::FromStr;
/// Default Podman bridge network name.
pub const DEFAULT_NETWORK_NAME: &str = "openshell";
pub const MACOS_PODMAN_MACHINE_HOST_GATEWAY_IP: &str = "192.168.127.254";
pub const DEFAULT_SANDBOX_UID: u32 = 10_001;

// Re-export the shared default so existing imports inside this crate keep working.
pub use openshell_core::config::DEFAULT_SANDBOX_PIDS_LIMIT;
Expand Down Expand Up @@ -122,6 +123,11 @@ pub struct PodmanComputeConfig {
///
/// Set to `0` to leave Podman's runtime/default PID limit unchanged.
pub sandbox_pids_limit: i64,
/// Numeric identity used for the agent process. The supervisor remains
/// root while it prepares the sandbox.
pub sandbox_uid: Option<u32>,
/// Numeric group used for the agent process. Defaults to the resolved UID.
pub sandbox_gid: Option<u32>,
/// Allow sandbox requests to attach host bind mounts through
/// `template.driver_config`.
#[serde(default)]
Expand Down Expand Up @@ -189,6 +195,24 @@ impl PodmanComputeConfig {
Ok(())
}

/// Resolve and validate the numeric identity injected into the supervisor.
pub fn resolve_sandbox_identity(&self) -> Result<(u32, u32), crate::client::PodmanApiError> {
let uid = self.sandbox_uid.unwrap_or(DEFAULT_SANDBOX_UID);
let gid = self.sandbox_gid.unwrap_or(uid);
for (field, value) in [("sandbox_uid", uid), ("sandbox_gid", gid)] {
if !(openshell_policy::MIN_SANDBOX_UID..=openshell_policy::MAX_SANDBOX_UID)
.contains(&value)
{
return Err(crate::client::PodmanApiError::InvalidInput(format!(
"{field} must be in [{}, {}]",
openshell_policy::MIN_SANDBOX_UID,
openshell_policy::MAX_SANDBOX_UID,
)));
}
}
Ok((uid, gid))
}

/// Validate optional host gateway override.
pub fn validate_host_gateway_ip(&self) -> Result<(), crate::client::PodmanApiError> {
let trimmed = self.host_gateway_ip.trim();
Expand Down Expand Up @@ -260,6 +284,8 @@ impl Default for PodmanComputeConfig {
guest_tls_cert: None,
guest_tls_key: None,
sandbox_pids_limit: DEFAULT_SANDBOX_PIDS_LIMIT,
sandbox_uid: None,
sandbox_gid: None,
enable_bind_mounts: false,
health_check_interval_secs: DEFAULT_HEALTH_CHECK_INTERVAL_SECS,
}
Expand All @@ -283,6 +309,8 @@ impl std::fmt::Debug for PodmanComputeConfig {
.field("guest_tls_cert", &self.guest_tls_cert)
.field("guest_tls_key", &self.guest_tls_key)
.field("sandbox_pids_limit", &self.sandbox_pids_limit)
.field("sandbox_uid", &self.sandbox_uid)
.field("sandbox_gid", &self.sandbox_gid)
.field("enable_bind_mounts", &self.enable_bind_mounts)
.field(
"health_check_interval_secs",
Expand Down Expand Up @@ -382,6 +410,52 @@ mod tests {
assert!(err.to_string().contains("sandbox_pids_limit"));
}

#[test]
fn sandbox_identity_defaults_to_10001() {
assert_eq!(
PodmanComputeConfig::default()
.resolve_sandbox_identity()
.unwrap(),
(10_001, 10_001)
);
}

#[test]
fn sandbox_identity_uses_uid_for_default_gid() {
let cfg = PodmanComputeConfig {
sandbox_uid: Some(12_345),
..PodmanComputeConfig::default()
};
assert_eq!(cfg.resolve_sandbox_identity().unwrap(), (12_345, 12_345));
}

#[test]
fn sandbox_identity_allows_configured_gid_without_uid() {
let cfg = PodmanComputeConfig {
sandbox_gid: Some(12_346),
..PodmanComputeConfig::default()
};
assert_eq!(cfg.resolve_sandbox_identity().unwrap(), (10_001, 12_346));
}

#[test]
fn sandbox_identity_rejects_system_uid() {
let cfg = PodmanComputeConfig {
sandbox_uid: Some(999),
..PodmanComputeConfig::default()
};
assert!(cfg.resolve_sandbox_identity().is_err());
}

#[test]
fn sandbox_identity_rejects_gid_above_policy_range() {
let cfg = PodmanComputeConfig {
sandbox_gid: Some(openshell_policy::MAX_SANDBOX_UID + 1),
..PodmanComputeConfig::default()
};
assert!(cfg.resolve_sandbox_identity().is_err());
}

#[test]
#[cfg(target_os = "macos")]
fn default_socket_path_uses_podman_machine_on_macos() {
Expand Down
Loading
Loading