Add hardness, water-type, data-density, and water-level-change products#100
Merged
Conversation
Four new OGC GeoJSON products, all reusing existing cohort source fetches (zero new API pulls): - nm_hardness (ogc_hardness): total hardness as CaCO3 from latest calcium + magnesium, with soft/moderate/hard/very-hard class. - nm_water_type (ogc_water_type): hydrochemical (Piper) water type from the major-ion suite — meq% of cations (Ca/Mg/Na+K) and anions (HCO3+CO3/Cl/SO4), dominant ion >50% else mixed, plus charge-balance QC. - nm_waterlevel_data_density (ogc_data_density): per-well measurement coverage (raw/daily counts, span, mean interval, observations per year). - nm_waterlevel_change (ogc_waterlevel_change): depth-to-water change over a trailing window (default 5y); positive = water table declined; start must be within half the window of the target else status=insufficient. Hardness and water-type pivot per-well summaries (analytes cohort); data-density and water-level-change consume sites/timeseries (waterlevels cohort). All four add no new shared source assets — they ride fetches the existing products already perform. Adds the dumpers + shared pivot/feature helpers in ogc_features.py, the combine dispatch and multi-analyte param handling in products.py, the output types in definitions.py, the product entries in products.yaml, and dumper tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Your pull request is automatically being deployed to Dagster Cloud.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Four new OGC GeoJSON products, all reusing existing cohort source fetches (zero new API pulls):
output_typeogc_hardnessogc_water_typeogc_data_densityogc_waterlevel_changeHardness + water-type pivot per-well summaries (analytes cohort). Data-density + WL-change consume sites/timeseries (waterlevels cohort). Source assets: 68 before, 68 after — the four ride fetches the existing products already perform.
Design decisions
2.497·Ca + 4.118·Mg(mg/L as CaCO₃). Latest-each pairing; Ca/Mg dates may differ, both emitted. Missing ion →insufficient.mixed;charge_balance_pctflags suspect analyses. No cations or no anions →insufficient.record_count= distinct days,observation_count= raw valid readings.window_yearsbefore, accepted only within ½ window of target elsestatus=insufficient. Positivechange_ft= depth-to-water increased = water table declined.window_yearsconfigurable in YAML.Files
backend/persisters/ogc_features.py— 4 dumpers + shared_num/_pivot_by_well/_well_feature/_site_feature.orchestration/assets/products.py— combine dispatch + multi-analyte param handling.orchestration/definitions.py— 4 added to_SUPPORTED_OUTPUT_TYPES.orchestration/config/products.yaml— 4 product entries.tests/test_persisters/test_ogc_features.py— 14 new dumper tests.Verification
dg.Definitionsbuilds clean (assets + jobs + schedules); new products land in the correct cohorts; graph adds 0 new source assets.🤖 Generated with Claude Code