Add waterlevel-trends data product (one feature per well)#84
Merged
Conversation
New ogc_waterlevel_trend product: nm_waterlevel_trends. One GeoJSON feature per well with a depth-to-water trend computed from the well's water-level timeseries, across all NM sources. Algorithm ported from the Ocotillo API ogc_depth_to_water_trend_wells materialized view: - least-squares slope (REGR_SLOPE) of depth-to-water (ft) vs observation time, scaled to ft/year (365.25-day year); - classify only with >=10 records, or >=4 records spanning >=2 years; slope > 0.25 ft/yr "increasing" (deeper/declining), < -0.25 "decreasing", else "stable"; otherwise "not enough data". The product includes a trend_method description of how the trend was calculated (TREND_METHOD_DESCRIPTION). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Your pull request is automatically being deployed to Dagster Cloud.
|
Replace the OLS slope/threshold classification with the non-parametric Mann-Kendall trend test (pymannkendall.original_test, alpha=0.05) for direction/significance, and the Theil-Sen slope (scipy, time-aware) for the ft/year rate. More robust to outliers and irregular sampling than ordinary least squares. - trend_category: significant increasing/decreasing, else 'stable'; 'not enough data' below the record/span gate (unchanged gate). - new per-feature mk_p_value and mk_tau properties. - add pymannkendall dependency (pulls scipy); serverless gets it via the root pyproject backend deps. - updated TREND_METHOD_DESCRIPTION. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The four dump_* functions each repeated the FeatureCollection envelope + file write, the coords/geometry build, the feature-id rule, and ISO-UTC formatting. Extract: - _dump_collection(path, id, features, meta, extra) — envelope + write, with extra for collection-level keys (trend_method). - _point_geometry(lat, lon, elev) — Point geometry. - _feature_id(source, rid) — "source:id" rule. - _iso_utc(epoch) — shared ISO-8601 UTC formatting. Behavior-identical (-58 lines); all 13 persister tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
New
ogc_waterlevel_trendproduct —nm_waterlevel_trends. One GeoJSON feature per well carrying a depth-to-water trend computed from that well's water-level timeseries, across all NM sources. The collection includes atrend_methoddescription of how the trend was calculated.Algorithm (ported from Ocotillo)
Replicates the Ocotillo API
ogc_depth_to_water_trend_wellsmaterialized view (OcotilloAPI alembic migration):REGR_SLOPEequivalent) of depth-to-water (ft) vs observation time, scaled to ft/year (×31,557,600 s/yr, a 365.25-day year);> 0.25 ft/yr→ increasing (water level deeper / declining table),< -0.25→ decreasing, else stable; otherwise not enough data.DIE water-level values are already depth-to-water below ground surface (feet), so no measuring-point adjustment is applied (Ocotillo subtracts
measuring_point_heightfrom its raw values).Per-feature properties
record_count,first_observation_datetime,last_observation_datetime,span_years,slope_ft_per_year,trend_category,well_depth(+units), geometry. Collection-leveltrend_methoddocuments the method.Changes
backend/persisters/ogc_features.py:dump_waterlevel_trend_collection+_regr_slope/_classify_trend/_parse_epoch_secondsand the ported thresholds +TREND_METHOD_DESCRIPTION.orchestration/assets/products.py: combine asset dumps the trend collection forogc_waterlevel_trend(per-well timeseries, index-aligned with sites).orchestration/definitions.py: register the output type (auto-gets a job + schedule).orchestration/config/products.yaml:nm_waterlevel_trends(all NM sources, daily 12:00).tests/test_persisters/test_ogc_features.py: trend classification (increasing / stable / decreasing / not-enough-data) + required-field coverage.Verification
nm_waterlevel_trends_job.🤖 Generated with Claude Code