Red team review! 🧑‍💻

[jeremymanning]

World Compute: Safety-First Modification Plan

Version: draft for collaborator review
Date: 2026-04-16

Executive recommendation

World Compute should not launch as an open, general-purpose, public compute fabric.

The safest credible path is to convert it into a federated, institution-backed, tightly constrained compute network with these properties:

1. No arbitrary public code execution
2. No arbitrary guest internet access
3. Strong institutional identity and accountability
4. Curated workload classes and signed artifacts only
5. Boring, testable control-plane logic before decentralization
6. Progressive rollout from trusted cohorts outward

This plan treats risk minimization as the primary design objective and utility maximization as secondary, even when that costs openness, convenience, and speed.

---

1. Design objective

Primary objective

Prevent World Compute from becoming any of the following:

• a malware distribution path
• a botnet or abuse platform
• a supply-chain compromise amplifier
• a privacy breach engine
• a fragile governance experiment with security consequences

Secondary objective

Preserve enough usefulness that the system is still worth deploying:

• distributed batch computation
• approved research workloads
• approved model serving / batch inference
• institutionally sponsored collaborative compute

---

2. The core design change

The main modification is conceptual:

Old posture: “A world-scale decentralized compute resource that anyone can use.”
New posture: “A federated academic/research compute network for approved workloads, operated under strong identity, constrained execution, and explicit accountability.”

That sounds less ambitious. It is also much safer.

---

3. Hard non-goals for v1

These should be explicitly out of scope for the first public-facing release.

Forbidden in v1

• open public job submission
• arbitrary user-uploaded binaries or containers
• arbitrary outbound network access from jobs
• fully autonomous mesh-LLM control of admission, scheduling, or deployment
• peer-to-peer guest networking
• direct donor-LAN access from jobs
• long-running public services
• self-updating workloads
• runtime package installation from the internet
• GPU passthrough for untrusted public workloads
• anonymous or pseudonymous use
• governance by public voting on safety-critical decisions

Deferred until far later, if ever

• public “world computer” branding in the literal sense
• generalized marketplace semantics
• permissionless federation
• user-defined custom job graphs with dynamic code download
• internet-facing agent swarms

---

4. Threat model

The system should be designed against at least these adversaries:

External adversaries

• people submitting malicious workloads
• people using the platform for scanning, spam, DDoS, credential attacks, or malware staging
• people probing the guest/host boundary for sandbox escapes
• people attempting model theft or data exfiltration

Internal / trusted-but-fallible adversaries

• compromised university accounts
• malicious insiders at participating institutions
• coordinators making unsafe policy changes
• developers shipping vulnerable or malicious updates
• compromised signing keys or CI/CD systems

Governance adversaries

• Sybil-style influence attacks
• policy drift toward convenience over safety
• institutional free-riding without accountability
• pressure to “just open it up a little more”

The plan below is designed so that no single failure should produce catastrophic system-wide abuse.

---

5. Safety design principles

1. Default deny. Every capability must be granted explicitly.
2. Least privilege. Users, jobs, hosts, coordinators, and services each get the minimum required access.
3. Secure by default. Unsafe modes must not be the easy modes.
4. No safety dependence on a language model. LLMs may assist, but deterministic controls must be authoritative.
5. Accountability over anonymity. Every job should be attributable to a real institutional identity.
6. Containment over optimism. Assume some reviews will fail; the runtime must still be safe.
7. Narrow first, broaden later. Start with a small safe envelope and expand only with evidence.
8. Institutional trust beats crowd trust. Early trust should come from participating organizations, not reputation markets.
9. One-way ratchet for risk. New capabilities require explicit review, testing, and governance approval.
10. The supply chain is part of the perimeter. Signed malware is still malware.

---

6. Identity, admission, and accountability

6.1 Replace “.edu email” with institutional federation

Do not rely on email suffixes as the main gate.

Use institutional SSO through research/education identity federations, with:

• current institutional affiliation
• required MFA
• institution-backed identity assertion
• per-user persistent unique identifiers
• revocation/deprovisioning tied to the home institution

6.2 Add assurance tiers

Every user account should have an assurance level, for example:

• Tier A: faculty/staff/research staff with institution-backed identity
• Tier B: students sponsored by a Tier A user
• Tier C: external collaborators sponsored by a participating institution
• Tier D: internal operators / coordinators

Higher-risk capabilities should require higher assurance.

6.3 Require sponsorship and institutional liability

Each participating institution should designate:

• a technical contact
• a security/abuse contact
• a policy approver
• a resource owner / donor admin

Each submitter should be traceable to:

• a home institution
• a sponsoring lab, PI, or department
• an abuse escalation path

6.4 Ban shared and throwaway identities

No shared lab accounts.
No “guest” submitter accounts.
No access based solely on verified email possession.

6.5 Use account controls beyond login

Require:

• MFA
• recent reauthentication for sensitive actions
• device/session risk checks where possible
• per-user submission quotas
• per-institution quotas
• cooldowns after policy violations

---

7. What users may submit

This is the most important product change.

7.1 v1 should not accept arbitrary code

In the first serious deployment, users should submit workloads from a reviewed catalog, not custom executables.

Examples:

• approved scientific pipelines
• approved simulation kernels
• approved model inference images
• approved fine-tuning adapters
• approved data transforms
• approved evaluation jobs

7.2 Shift from “upload code” to “choose a declared workload type”

A submission should be a declarative manifest, not a tarball of freedom.

The manifest should specify:

• workload class
• approved image/artifact digest
• input data locations
• expected outputs
• resource limits
• whether GPU is requested
• whether any network endpoint is needed
• data sensitivity classification
• sponsoring user and project
• justification text

7.3 New workload classes require review

Any new workload class should trigger:

• policy review
• technical review
• reproducibility check
• abuse assessment
• limited pilot
• canary run on trusted hosts first

Only after that should it become generally available to that trust tier.

7.4 Signed artifacts only

All images, binaries, and model artifacts must be:

• built in approved pipelines
• signed
• provenance-attested
• pinned by digest
• logged in a transparency system
• blocked if unsigned or policy-incompatible

7.5 No runtime code fetch

Jobs may not:

• pip install from the internet
• curl | bash
• fetch secondary payloads
• self-modify from remote endpoints
• download model weights from arbitrary sources at runtime

Everything needed to execute should be present in reviewed artifacts or approved object stores.

---

8. Review architecture

8.1 Deterministic policy engine is the real gate

Admission control should be implemented as explicit, testable policy logic.

A job should be rejected unless:

• the submitter identity is valid and authorized
• the workload class is approved
• the artifact digest is approved
• requested resources are within policy
• required endpoints are on an allowlist
• data classification is compatible with the target environment
• quotas are not exceeded
• no policy bans are triggered

8.2 LLM review is advisory only

An LLM may help with:

• summarizing manifest intent
• detecting suspicious text patterns
• clustering similar abusive submissions
• routing ambiguous jobs to humans
• identifying unusual combinations of permissions

An LLM should not be the authoritative final decision-maker for:

• admission
• release signing
• safety policy changes
• containment actions with large blast radius

8.3 Human review path

Create explicit review queues:

• Green: auto-admit approved low-risk jobs
• Yellow: human review required
• Red: deny by policy and alert operator

Triggers for human review should include:

• new workload class
• request for GPU on less-trusted hardware
• any requested network exception
• elevated data sensitivity
• unusual output patterns
• repeated near-policy-violation behavior by a user/project
• cross-institution routing requests
• jobs whose intent cannot be clearly explained

---

9. Networking and egress policy

This is the single highest-value technical change.

9.1 Default-deny egress

By default, jobs should have no internet access.

They should not be able to:

• resolve arbitrary DNS
• open arbitrary outbound sockets
• contact cloud metadata endpoints
• contact Tor/VPN endpoints
• reach arbitrary APIs
• send email
• scan hosts
• hit social platforms, pastebins, or code forges

9.2 Also block local/network abuse paths

Jobs should not reach:

• donor LAN resources
• RFC1918 private ranges
• link-local addresses
• loopback on the host
• multicast/broadcast discovery
• management interfaces
• storage/control-plane systems except via explicit brokers

9.3 Allowlist only where absolutely necessary

If a workload truly needs network access, it should use:

• declared endpoints
• approved protocols
• narrow destination allowlists
• mediated gateways/proxies
• full logging
• separate trust tier

Network exceptions should be rare and reviewable.

9.4 No guest-to-guest peer mesh for v1

Do not let untrusted workloads create their own overlay network.
Any inter-job communication should go through controlled orchestration services.

---

10. Host and runtime isolation

10.1 Treat each job as hostile

Even reviewed jobs should execute as though compromise is possible.

10.2 Minimum runtime controls

Every job should have:

• hardware-backed VM or similarly strong isolation
• read-only base image
• ephemeral writable scratch space
• no host file mounts by default
• no host credentials
• no host device passthrough by default
• strict CPU/memory/disk quotas
• strict wall-clock timeouts
• process count limits
• filesystem and syscall restrictions
• no persistence after completion except declared outputs

10.3 Separate trust tiers for hardware

Use different pools for different risk levels, for example:

• Tier 1 hosts: dedicated institutional nodes for low-risk CPU jobs
• Tier 2 hosts: dedicated institutional GPU nodes for approved model workloads
• Tier 3 hosts: especially trusted or confidential-compute-capable nodes for sensitive jobs

Do not start with arbitrary personal laptops as first-class execution targets.

10.4 Be conservative on GPU exposure

GPU passthrough and low-level accelerator access should be treated as elevated risk.
For early phases:

• prefer CPU or tightly brokered GPU serving
• reserve GPU execution for approved workload classes
• use dedicated trusted pools
• keep kernels short and bounded
• maintain rapid eviction / containment options

---

11. Data policy

11.1 Classify data at submission time

Every submission should declare a data class such as:

• public
• internal research data
• controlled but non-regulated data
• sensitive / restricted data

11.2 Match data class to environment

Sensitive data should not be routable to ordinary volunteer hardware.

A simple starting rule:

• public data may run on broader trusted pools
• internal data runs only on institution-approved pools
• sensitive data runs only on specially approved infrastructure, or not at all

11.3 Minimize retention

Retain only:

• required audit metadata
• declared outputs
• minimal security logs
• short-lived scratch data during execution

Purge intermediate data aggressively.

11.4 Protect prompts, activations, and model inputs

If the platform later supports distributed LLM inference, treat:

• prompts
• retrieved documents
• embeddings
• intermediate activations
• generated outputs

as potentially sensitive data.
Do not route them over untrusted peers by default.

---

12. Control plane and governance

12.1 Do not decentralize the safety boundary in v1

Early governance should be boring and legible.

For the first serious phases:

• coordinators are operated by a small set of vetted institutions
• membership is explicit
• safety policy is versioned and reviewable
• deployment rights are tightly separated
• emergency controls are centralized enough to work

12.2 Remove public voting from safety-critical paths

Do not use public or broad member voting for:

• release approvals
• safety policy changes
• admission policy relaxations
• incident response actions
• coordinator election in the first phases

Use a formal review board instead.

12.3 Separation of duties

No single actor should be able to:

• approve a risky workload class
• sign artifacts
• deploy policy changes
• override admission
• silence alerts

without independent review.

12.4 Keep the mesh LLM out of the control loop at first

The mesh LLM, if kept at all, should be limited to:

• advisory analysis
• operator-facing summaries
• anomaly triage suggestions

It should not autonomously:

• change policy
• approve jobs
• update the platform
• reroute trust relationships
• deploy major changes

---

13. Supply-chain and release security

This area deserves the same seriousness as sandboxing.

13.1 Treat build/sign/release as crown-jewel systems

The following must be hardened before broad rollout:

• source control protections
• CI/CD isolation
• provenance generation
• signing key storage
• release approval workflow
• registry security
• artifact retention and rollback

13.2 Minimum required practices

Require:

• reproducible or at least highly deterministic builds where feasible
• SBOM generation
• provenance attestations
• signed commits / protected branches
• mandatory reviews
• hardware-backed or equivalent protected signing
• transparency logging for signatures/releases
• rapid revocation and key rotation procedures
• emergency stop-ship capability

13.3 Concrete maturity target

A reasonable target is:

• adopt NIST SSDF practices as a baseline
• target meaningful SLSA maturity for build provenance and release integrity
• use a signing/transparency ecosystem such as Sigstore or an equivalent internal system

13.4 Multiple release channels

Use:

• dev
• staging
• canary
• production

Do not promote anything directly from development to production.

---

14. Detection, monitoring, and audit

14.1 Log for security, not voyeurism

Collect:

• who submitted what
• artifact digests
• policy decisions
• resource use
• attempted denied capabilities
• network exception requests
• runtime crashes / anomalies
• output hashes and destinations
• host assignment
• policy version used

14.2 Detection rules should focus on constrained environments

Look for:

• repeated denied syscalls
• attempts to open unexpected sockets
• repeated forbidden filesystem probes
• crash loops
• unusual GPU behavior
• sudden cross-user similarity suggesting templated abuse
• anomalous output sizes or patterns
• repeated jobs whose declared purpose and actual behavior diverge

14.3 Every kill should leave a trail

Containment actions should be:

• logged
• attributable
• reviewable after the fact
• reversible only by authorized roles

14.4 Transparency reporting

Publish periodic security reports to participants:

• number of denied jobs
• number of manual reviews
• incidents and near-misses
• policy changes
• deprecations and newly allowed capabilities

That helps keep convenience from quietly outrunning safety.

---

15. Incident response

15.1 Predefine incident classes

At minimum:

• attempted abuse without host compromise
• successful abuse of network policy
• guest escape / suspected host compromise
• supply-chain compromise
• identity abuse / account takeover
• data handling breach
• coordinator compromise

15.2 Predefine responses

For severe incidents, the platform should be able to:

• freeze new submissions
• quarantine a workload class
• block a user/project/institution
• revoke an artifact or signing root
• drain a host pool
• rotate secrets and certificates
• force reauthentication
• pause federation links between institutions

15.3 Practice this before launch

Run tabletop exercises for:

• malicious approved artifact
• compromised university account
• policy bug allowing egress
• suspected sandbox escape
• leaked signing key
• malicious insider on the review team

No broad launch before those exercises produce actionable fixes.

---

16. Rollout plan

Phase 0: internal-only prototype

Scope:

• one institution
• dedicated nodes only
• approved workloads only
• no arbitrary code
• no guest internet
• no mesh LLM authority
• no public access

Success condition:

• stable policy engine
• stable audit pipeline
• reproducible reviewed workload path
• successful incident drills

Phase 1: trusted multi-institution pilot

Scope:

• a small set of participating institutions
• institutional SSO required
• sponsored users only
• dedicated trusted hardware pools
• mostly CPU jobs, tightly constrained GPU jobs if any

Success condition:

• low false-positive/false-negative review rates
• no policy-bypass incidents
• documented abuse handling with institutions

Phase 2: curated broader service

Scope:

• larger set of institutions
• broader approved workload catalog
• more automation in review
• selective network exceptions through brokered services
• possibly institution-approved model inference pools

Success condition:

• evidence that broader utility does not erode the safety posture

Phase 3: limited expansion of programmability

Only after extensive evidence.
Potentially allow:

• more flexible manifests
• institution-scoped custom images
• wider GPU use
• stronger federation semantics

This phase should require a fresh threat review, not just incremental optimism.

---

17. Go / no-go criteria

World Compute should not move to a broader phase unless all of the following are true:

1. Identity: submitters are institution-backed and revocable
2. Admission: only approved workload classes are runnable
3. Egress: default-deny network policy is proven in practice
4. Isolation: runtime containment has been externally reviewed
5. Supply chain: signing, provenance, and release controls are in place
6. Detection: logs and alerts are actionable and tested
7. Response: emergency pause/quarantine mechanisms work
8. Governance: decision rights are explicit and separated
9. Abuse handling: institutions accept accountability and response obligations
10. Red-team results: major findings are closed or consciously accepted by leadership

If any of these are missing, the safe answer is delay.

---

18. What utility survives under this plan

Even with these restrictions, the system can still be useful.

Good early fits

• batch inference for approved models
• evaluation jobs
• data preprocessing pipelines
• simulation workloads with fixed kernels
• institutionally curated ML experimentation
• distributed background agents over reviewed code paths
• campus or consortium compute sharing

Bad early fits

• arbitrary public cloud replacement
• interactive public chat at hyperscale
• frontier model training
• internet-connected autonomous agents
• regulated or highly sensitive data on heterogeneous volunteer machines
• unrestricted “run my code anywhere” workflows

---

19. The safest high-utility specialization

If the project wants maximum utility with minimum additional risk, the best specialization is probably:

A federated, batch-oriented, institutionally trusted model-serving and approved-inference network

rather than a general-purpose public compute market.

That version still needs strong controls, but it removes the most dangerous feature: arbitrary user-supplied execution.

---

20. Concrete actions for the next 90 days

P0 — do immediately

• remove open-public-submission language from the design
• remove or disable any guest internet access in prototypes
• define approved workload classes
• define job manifest schema
• define trust tiers for users, workloads, and hardware
• define the human review process
• define incident classes and emergency controls
• move mesh-LLM safety authority to advisory-only

P1 — before any external pilot

• integrate institutional SSO
• require MFA
• stand up signed artifact registry
• implement provenance and release logging
• implement policy engine
• implement per-user/per-institution quotas
• implement audit logging and alerting
• run red-team and tabletop exercises

P2 — before any multi-institution scaling

• finalize governance charter
• finalize abuse response agreements with institutions
• obtain external security review
• validate canary / rollback procedures
• validate host quarantine and federation pause procedures
• define a public transparency and disclosure policy

---

21. Bottom line

To make World Compute as safe as possible, the project should stop trying to be a permissionless world computer in its early life.

The safest viable version is:

• institution-backed
• strongly identified
• curated
• network-constrained
• artifact-signed
• policy-driven
• auditable
• boring at the control plane
• incremental in rollout

That version is less flashy. It is also far more likely to survive contact with the real world.

---

Selected references

• InCommon Federation overview
• eduGAIN overview
• NIST Secure Software Development Framework (SP 800-218)
• NIST guidance augmenting SSDF for AI model development (SP 800-218A draft)
• NIST Zero Trust Architecture (SP 800-207)
• SLSA overview and levels
• Sigstore / Rekor transparency logging
• CISA Secure by Design / secure defaults
• MANRS anti-spoofing guidance