Skip to content

README.md

Latest commit

 

History

History
180 lines (133 loc) · 4.21 KB

README.md

File metadata and controls

180 lines (133 loc) · 4.21 KB

CPU Worker: Serverless CPU Computing

Simple example demonstrating CPU-based serverless workers with automatic scaling on Runpod's infrastructure.

Quick Start

1. Install Dependencies

uv sync

2. Authenticate

uv run flash login

Or create a .env file with RUNPOD_API_KEY=your_api_key_here.

3. Run the Example

python cpu_worker.py

The function executes on a Runpod CPU worker and prints the result directly:

Testing CPU worker with payload: {'name': 'Testing CPU worker'}
Result: {'status': 'success', 'message': 'Hello, Testing CPU worker!', 'worker_type': 'CPU', ...}

First run takes 30-60 seconds (provisioning). Subsequent runs take 2-3 seconds.

Alternative: HTTP API Testing

To test via HTTP endpoints instead:

uv run flash run

Visit http://localhost:8888/docs for interactive API documentation.

curl -X POST http://localhost:8888/cpu_worker/runsync \
  -H "Content-Type: application/json" \
  -d '{"name": "Flash User"}'

What This Demonstrates

CPU Worker (cpu_worker.py)

Simple CPU-based serverless function that:

  • Processes requests without GPU overhead
  • Returns system and platform information
  • Scales from 0-3 workers automatically
  • Runs on general-purpose CPU instances

The worker demonstrates:

  • Remote execution with the @Endpoint decorator
  • CPU resource configuration via cpu= parameter
  • Automatic scaling via workers= parameter
  • Lightweight API request handling

API Endpoints

QB (queue-based) endpoints are auto-generated from @Endpoint functions. Visit /docs for the full API schema.

cpu_hello

Executes a simple CPU worker and returns a greeting with system information.

Request:

{
  "name": "Flash User"
}

Response:

{
  "status": "success",
  "message": "Hello, Flash User!",
  "worker_type": "CPU",
  "timestamp": "2024-01-24T10:30:45.123456",
  "platform": "Linux",
  "python_version": "3.11.0"
}

Project Structure

02_cpu_worker/
├── cpu_worker.py        # CPU worker with @Endpoint decorator
├── pyproject.toml       # Project metadata
├── requirements.txt     # Dependencies
├── .env.example         # Environment variables template
└── README.md            # This file

Key Concepts

Remote Execution

The @Endpoint decorator transparently executes functions on serverless infrastructure:

  • Code runs locally during development
  • Automatically deploys to Runpod when configured
  • Handles serialization and resource management
from runpod_flash import Endpoint

@Endpoint(name="my-worker", cpu="cpu3c-1-2", workers=(0, 3))
async def my_function(data: dict) -> dict:
    return {"result": "processed"}

CPU Instance Types

Available CPU configurations:

  • CpuInstanceType.CPU3G_2_8: 2 vCPU, 8GB RAM (General Purpose)
  • CpuInstanceType.CPU3C_4_8: 4 vCPU, 8GB RAM (Compute Optimized)
  • CpuInstanceType.CPU5G_4_16: 4 vCPU, 16GB RAM (Latest Gen)

CPU type can be specified as an enum or a string shorthand:

# enum
@Endpoint(name="worker", cpu=CpuInstanceType.CPU3C_1_2)

# string shorthand
@Endpoint(name="worker", cpu="cpu3c-1-2")

Resource Scaling

The CPU worker scales to zero when idle:

  • workers=(0, 3): Scale from 0 to 3 workers
  • idle_timeout=5: 5 minutes before scaling down

Development

Run the Worker

python cpu_worker.py

HTTP API Testing (Optional)

uv run flash run

When to Use CPU Workers

Choose CPU workers for:

  • API request handling
  • Data processing and transformation
  • Lightweight compute tasks
  • Cost-sensitive workloads
  • No GPU requirements

Compare with GPU workers when you need:

  • Machine learning inference
  • Image/video processing
  • CUDA acceleration
  • GPU-specific libraries (PyTorch, TensorFlow)

Next Steps

  • Customize CPU type: Change "cpu3c-1-2" to a different instance type
  • Add request validation and error handling
  • Integrate with databases or external APIs
  • Deploy to production with flash deploy

Resources