feat: Real-World Terrain Generator (UK Prototype)

[MichaelFisher1997]

Overview

Implement a real-world terrain generator that uses NASA SRTM 90m elevation data, ESA land-cover rasters, and OSM water features to generate accurate UK terrain as an optional world type.

This replaces the investigation in #128 with a concrete implementation plan.

Architecture

New subsystem: src/world/realworld/ — completely separate from existing noise generators.

Key Design Decisions

• DEM Source: NASA SRTM 90m behind a DEMSource interface (swappable to Copernicus later)
• Cache: 10GB LRU disk cache for tiles
• Water: Pre-processed UK water mask (100m raster, build step)
• Vertical Stretch: Configurable per-world (1.0x / 1.5x / 2.0x)
• Scope: UK only for prototype (~49°N to 61°N, 8°W to 2°E)
• Threading: Async I/O on dedicated thread pool; gen workers never block

Implementation Phases

Phase 1: Geospatial Infrastructure

• projection.zig — WGS84 ↔ block(x,z), equirectangular projection
• tile_cache.zig — 10GB LRU disk cache (file-based or SQLite)
• dem_source.zig — DEMSource interface with SRTM90Source implementation
• tile_prefetcher.zig — Predicts needed tiles from player position, queues fetches

Phase 2: Data Pipeline

• scripts/generate_uk_water_mask.py — Pre-process OSM water polygons to 100m raster mask
• biome_mapper.zig — ESA land-cover class → BiomeId mapping table
• realworld_data.zig — Aggregator: samples DEM, water mask, ESA raster per chunk

Phase 3: Generator Implementation

• realworld_generator.zig — Implements existing Generator.VTable
  • generate() — Samples data, fills blocks, applies vertical stretch + noise detail
  • Cache miss → returns "not ready", chunk retries next frame
  • Slope-based material selection + water placement

Phase 4: Integration

• Register in registry.zig as 4th generator option
• UI selector for stretch factor in singleplayer.zig
• Store config in level.dat (generator_index, stretch_factor, world_bounds)

Phase 5 (Future): Round World

• Deferred until after real-world gen is stable
• Will require chunk coordinate wrapping, spherical distance math, polar projection

Threading Model

[Main Thread]          [Gen Workers]              [I/O Workers]
     |                       |                           |
WorldStreamer ──► chunk.gen_queue ──► generator.generate()
(predicts tiles)    (4 workers)         │ reads tile_cache
     │                                   │ (hit: generate)
     │                                   │ (miss: return NOT_READY)
     ▼                                   ▼
TilePrefetcher ◄──────── queue fetch ───┘
(queues async fetches)              [I/O pool downloads
     │                               DEM/ESA/OSM tiles]
     ▼
tile_cache (disk/SQLite)

Critical: Worker threads never block on I/O. They either generate immediately (cache hit) or yield (cache miss).

Block Fill Logic

1. Sample DEM elevation → blockY = (elevation - sea_level) / scale + water_level
2. Apply noise detail (±2 blocks from existing noise system)
3. Slope-based material selection:
   • Slope < 0.2: grass/dirt
   • 0.2–0.6: stone

   • 0.6: stone + gravel

4. Water: if OSM lake/river polygon covers block, set water blocks up to sea level

Risks & Mitigations

• ESA raster availability: Verify UK coverage and license. Fallback: derived biomes from elevation/slope.
• SRTM server reliability: Retry logic + backoff in SRTM90Source.
• Water mask complexity: If OSM→raster is too complex, fallback to dynamic rasterization (slower but simpler).

Acceptance Criteria

• Can create a new "Real World (UK)" world from singleplayer menu
• Terrain heights match SRTM 90m data within 1 block
• ESA biomes map correctly to existing BiomeId values
• Water features (lakes/rivers) render at correct elevations
• Chunk generation doesn't block on I/O (no frame drops during exploration)
• 10GB cache eviction works correctly (LRU)
• Falls back gracefully when offline (cached tiles only)

[github-actions]

This is a well-structured feature request with clear architecture. A few observations:

Scope Concern: This single issue spans 5 phases across 4 subsystems (geospatial infrastructure, data pipeline, generator implementation, UI integration). I'd recommend splitting into separate issues:

• Issue A: Geospatial infrastructure (projection.zig, tile_cache.zig, dem_source.zig, tile_prefetcher.zig) — this is prerequisite work
• Issue B: Data pipeline (scripts/generate_uk_water_mask.py, biome_mapper.zig, realworld_data.zig) — data processing
• Issue C: Generator core (realworld_generator.zig) — the actual Generator implementation
• Issue D: Integration (registry.zig, singleplayer.zig, level.dat) — UI and config

Architectural Notes:

• Generator.VTable interface is the right hook point — good design choice
• The "return NOT_READY on cache miss" pattern will cause chunk retry churn; consider a small pending queue with bounded retry to avoid frame-spike popping when exploring into uncached areas
• No mention of LOD interaction — distant terrain with real-world data at 90m resolution may need separate LOD treatment

Missing Info:

• What's the memory budget for the I/O workers? 10GB disk cache + ESA rasters could exceed reasonable limits
• Is there a fallback if SRTM tiles are missing for a region (e.g., some sea areas return null)?

Overall the plan is solid. Breaking into sub-issues would make progress tracking easier and allow parallel implementation where dependencies allow.

opencode session  |  github run

[MichaelFisher1997]

Future Multiplayer/Server Idea (way down the line):

When we eventually tackle multiplayer, the real-world data pipeline could be even more powerful if moved to a dedicated server:

• Dockerized tile server: Runs the DEM/ESA data fetching, projection, and caching logic in a container
• In-memory + disk cache: Server keeps hot tiles in RAM (fast), cold tiles on disk (persistent)
• Stream to clients: Players connect and receive pre-generated/cached chunk data rather than generating locally
• Benefits:
  • Clients don't need 10GB local cache or async I/O thread pool
  • Single authoritative terrain source (no seed/desync issues)
  • Server can pre-generate popular areas (spawn, cities) ahead of time
  • Easy to scale: multiple world instances (UK, US, Alps) each with their own tile server

This would essentially turn the realworld/ subsystem into a client-side renderer while the heavy data work happens server-side.

[github-actions]

This issue has had no activity for 60 days. It will be closed in 14 days unless there is new activity or an exemption label is added.

[github-actions]

Closing this issue because it remained stale for 14 days without new activity.