You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on May 10, 2026. It is now read-only.
MobilityDB Discussion #861 proposes a portable SQL dialect that runs the same query unchanged on MobilityDB/PostgreSQL, MobilityDuck/DuckDB, and MobilitySpark/Spark (see also MobilitySpark RFC #3).
MobilityPySpark has two gaps that block portability:
1 — Function name mismatches
MobilityPySpark uses Python-style names that differ from the MEOS/MobilityDB canonical names. The portable dialect requires canonical names so the same SQL runs on all three platforms:
nearestApproachDistance / min_distance currently returns -1.0 on failure. The MEOS C layer uses DBL_MAX (established in MobilityDB PR #846); at the SQL level the canonical value is NULL.
Any UDF wrapper must convert the internal failure sentinel to None (→ SQL NULL) before returning. The -1.0 convention pre-dates PR #846 and should be retired.
Proposed fix
Rename UDFs to canonical names in pysparkmeos/UDF/udf.py and fix the NAD sentinel. Aliases for backward compatibility are optional but the canonical names must be the primary registered names.
This is the concrete change that unblocks BerlinMOD Q5 on MobilityPySpark.
Background
MobilityDB Discussion #861 proposes a portable SQL dialect that runs the same query unchanged on MobilityDB/PostgreSQL, MobilityDuck/DuckDB, and MobilitySpark/Spark (see also MobilitySpark RFC #3).
MobilityPySpark has two gaps that block portability:
1 — Function name mismatches
MobilityPySpark uses Python-style names that differ from the MEOS/MobilityDB canonical names. The portable dialect requires canonical names so the same SQL runs on all three platforms:
tpoint_at(trip, instant)atTime(trip, instant)ever_intersects(trip, geom)eIntersects(trip, geom)min_distance(t1, t2)/nearest_approach_distance(t1, t2)nearestApproachDistance(t1, t2)at_period(trip, period)atTime(trip, tstzspan)at_geom(trip, geom)atGeometry(trip, geom)2 — NAD failure sentinel (blocks Q5 portability)
nearestApproachDistance/min_distancecurrently returns-1.0on failure. The MEOS C layer usesDBL_MAX(established in MobilityDB PR #846); at the SQL level the canonical value isNULL.Any UDF wrapper must convert the internal failure sentinel to
None(→ SQLNULL) before returning. The-1.0convention pre-dates PR #846 and should be retired.Proposed fix
Rename UDFs to canonical names in
pysparkmeos/UDF/udf.pyand fix the NAD sentinel. Aliases for backward compatibility are optional but the canonical names must be the primary registered names.This is the concrete change that unblocks BerlinMOD Q5 on MobilityPySpark.
cc @Action52