Skip to content

Route observability logs to Google Cloud Logging - #142

Merged
anth-volk merged 1 commit into
mainfrom
google-cloud-logging-consumer
Jul 3, 2026
Merged

Route observability logs to Google Cloud Logging#142
anth-volk merged 1 commit into
mainfrom
google-cloud-logging-consumer

Conversation

@anth-volk

@anth-volk anth-volk commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Fixes #141

Summary

Routes deployed observability logs to Google Cloud Logging, using the destination features of policyengine-observability 1.1–1.3 (now pinned on main as ==1.3.1):

  • Installs policyengine-observability with the [google] extra ([fastapi,httpx,google]==1.3.1).
  • Deployed platforms default to the google_cloud_logging destination; local/dev stays on stdout. Platform detection gains Cloud Run (K_SERVICE/K_REVISION) alongside the existing Modal detection, with an OBSERVABILITY_PLATFORM override, and OBSERVABILITY_LOG_DESTINATIONS remains the escape hatch.
  • Static log metadata gains cloud_run_service / cloud_run_revision / cloud_run_configuration / google_cloud_project, and those keys join the metric attribute allowlist.
  • Modal Workload Identity Federation config (OBSERVABILITY_GOOGLE_CLOUD_PROJECT, OBSERVABILITY_GOOGLE_WORKLOAD_IDENTITY_PROVIDER, OBSERVABILITY_GOOGLE_SERVICE_ACCOUNT_EMAIL — repo vars with defaults) flows into the production and preview Modal secrets in deploy.yml / pr-beta-deploy.yml; no service-account JSON.
  • Tests: destination defaults per platform (init-only, so no destination is constructed), Cloud Run metadata on request logs, env override behaviour. Emission-path tests force OBSERVABILITY_LOG_DESTINATIONS=stdout so unit tests never touch the Google network seam — including the pre-existing modal metadata test, which the new deployed-platform default would otherwise have turned into a live gRPC call.

History note

An earlier revision carried a second commit ("Use dataset-free compiled parameter export") gated on PolicyEngine/policyengine-uk-v2#112. That commit was dropped in the rebase: #175 solved the same problem with explicit dataset="frs" selection, and the upstream dataset-free export API remains unreleased. If uk-v2#112 ships, the simplification can return as a small follow-up PR.

Verification

Full backend suite in a fresh Python 3.13 venv from this branch's requirements (observability 1.3.1 + google-cloud-logging 3.16.0): 241 passed, 10 skipped. The suite also caught and fixed a live-network hazard: with GCL as the deployed default, emission-path tests attempted real Cloud Logging writes (observed as a gRPC 404 against the placeholder project locally); those now pin stdout per the testing skill's network-seam rule.

Deploy prerequisites

  • The WIF pool/provider and observability-writer service account referenced by the workflow defaults must exist and trust the Modal identity (see the observability repo's Stage 3 runbook).
  • Repo vars OBSERVABILITY_GOOGLE_* optionally override the defaults.

🤖 Generated with Claude Code

@vercel

vercel Bot commented Jun 29, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
policyengine-uk-chat Ready Ready Preview, Comment Jul 3, 2026 10:38pm

Request Review

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown

Beta preview has been cleaned up because this PR was closed.

@anth-volk
anth-volk marked this pull request as ready for review July 3, 2026 22:48
@anth-volk
anth-volk merged commit 8ba5e36 into main Jul 3, 2026
7 of 8 checks passed
@anth-volk
anth-volk deleted the google-cloud-logging-consumer branch July 3, 2026 22:50
anth-volk added a commit that referenced this pull request Jul 3, 2026
…g-consumer"

This reverts commit 8ba5e36, reversing
changes made to c2f8f01.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Route UK Chat observability logs to Google Cloud Logging

1 participant