Log profiles, destination registry, and queued Cloud Logging transport

[anth-volk]

Fixes #22

What this adds

The restart of the bounded/async log emission work after PR #23 was abandoned (see its closing comment). The design is deliberately smaller and strategy-shaped, in six commits:

1. Bounded Google writes — the gapic write method (the single choke point under log_struct) is rebound on the destination-owned client with a retry whose transient-error budget and per-call timeout are both capped by OBSERVABILITY_GOOGLE_WRITE_TIMEOUT_SECONDS (default 10s, clamped — 0/inf/nan cannot disable or unbound it), replacing the 60s transport defaults. The write path stays log_struct, so field mapping is unchanged.
2. Stdout formatter registry — formatters register by name; the core knows only plain. The google formatter (agent-native severity/trace/spanId/labels special keys, no time key since emission is synchronous) lives in the google module. A broken formatter degrades to the unformatted line.
3. Destination registry + generic queued transport — destinations register as inline (stdout) or remote; the manager builds by lookup (no destination names in the manager) and wraps every remote strategy in QueuedLogDestination, built on stdlib queue.Queue + logging.handlers.QueueListener so CPython owns the thread/shutdown machinery. Mutable state is exactly four pieces (queue, listener, drop counter, closed flag — documented in the module docstring with the accepted races). emit() snapshots + stamps enqueue time and never blocks or raises; a full queue drops-and-counts with throttled reports; close() is idempotent and deadline-bounded (one monotonic deadline covers the sentinel put and the join; an undrained listener is abandoned with a report). No breaker, no retry queue, no fork hooks, no generations — delivery is best-effort by contract because stdout is always the durable sibling record.
4. Bounded shutdown + restart_observability() — shutdown closes log destinations first, inline, with a deadline derived from the shutdown budget (half when OTel providers need flushing, all of it otherwise); the provider flush thread only exists when providers do and is joined with the remaining budget. Restart closes and rebuilds destinations from config, documented for single-threaded lifecycle moments only (post-snapshot, post-fork, before traffic).
5. Log profiles — consumers set one env var: OBSERVABILITY_LOG_PROFILE = gcp-agent (google-format stdout only; Cloud Run), gcp-direct (stdout + queued Cloud Logging; Modal), plain-sync (plain stdout, guaranteed zero threads — the kill switch), auto (default: detects via OBSERVABILITY_PLATFORM > K_SERVICE > Modal markers, preserving caller defaults when nothing matches). Explicit OBSERVABILITY_LOG_DESTINATIONS/OBSERVABILITY_STDOUT_FORMAT override the expansion; gcp-direct without a resolvable project downgrades to plain-sync with a warning.
6. Docs — README profiles/semantics/lifecycle sections; runbook switched to profile-based config with the plain-sync kill switch.

Semantics changes to be aware of

• The google destination is always queued (maintainer-confirmed): no remote write ever runs on a request thread, making the 60–180s request stalls from the sync log_struct path structurally impossible. Delivery becomes explicitly best-effort; a hard crash loses what was still queued. No production consumer uses sync google writes today (household pins stdout).
• auto detection partially activates on deployed platforms: verified against household main's actual deploy config — on a lock refresh before the adoption PR, Cloud Run (which sets OBSERVABILITY_PLATFORM=google_cloud_run) starts emitting google-format stdout (additive keys the agent promotes; payload fields unchanged), while Modal does not activate: its secret sync on main carries no OBSERVABILITY_GOOGLE_CLOUD_PROJECT, so gcp-direct downgrades to plain-sync with a one-time warning report. Queued direct writes from Modal begin only when the adoption PR adds the project/WIF vars. If even the Cloud Run format change is unwanted early, deploys can pin OBSERVABILITY_LOG_PROFILE=plain-sync until adoption.

Validation

188 tests (was 126 on main), branch coverage 91% (gate ≥90), ruff clean. The queued transport is tested against fake inner destinations only (drain determinism via sentinel ordering, snapshot-at-enqueue, overflow + throttling, bounded close with stuck writes and jammed queues, listener survival across write failures, emit-never-raises matrix, atexit lifecycle, breaker non-interaction, reconfigure-closes-previous). Sequencing uses threading.Event only — no unconditional sleeps.

Merging auto-releases as 1.4.0 (.added fragments → minor).

Review fixes (ee055cf)

A five-angle deep review (bugs, plan conformance, test coverage, DRY, strategy purity) ran against this branch; every finding is fixed in the follow-up commit:

• Lifecycle bugs: restart_observability() now honors OBSERVABILITY_ENABLED; reconfigure installs new destinations before closing the old ones so close-phase reports keep a sink; the Google library's one-time instrumentation diagnostic entry is suppressed (the README had claimed it already was); the google destination closes its client after a drained close; manager close shares one deadline across destinations.
• Strategy purity: strategy requirements are declarative (register_destination(required_config=...), checked generically during profile expansion — no google conditional left in core config); backend knobs are strategy-owned (OBSERVABILITY_GOOGLE_WRITE_TIMEOUT_SECONDS is parsed by the google factory, the per-backend config field is gone); register_destination/register_stdout_formatter are exported top-level; google_credentials moved under destinations/; the old module path is removed (top-level re-exports unchanged; verified no consumer imports the deep path).
• DRY: one shared guarded-report helper + throttled counter, one duck-typed close contract, one name normalizer (destinations, formatters, and profiles now all forgive hyphen/underscore/case variance; unknown stdout formats are reported instead of silently degrading), queue defaults hoisted to config, stdout construction unified.
• Tests: shared fakes + conftest; 216 tests (was 188), 92% branch coverage; new coverage for close-failure containment, the configure-crash stdout fallback, gcp-agent end-to-end formatting through the manager, a broken reporting channel at construction, timed-out closes leaving the inner destination alone, clamp boundary pins, and restart delivery resumption.

Remaining before merge: a live smoke against real Cloud Logging (gcp-direct with local ADC) — not yet run.

🤖 Generated with Claude Code