Skip to content

Phase 8 / Task C1: Issue #166 — Part 2 @link fallback in cross-reference fields #166

@Sam-Bolling

Description

@Sam-Bolling

Phase 8 Task

Task ID: C1
Title: Issue #166 — Part 2 @link Fallback in Cross-Reference Fields
Source: Repo issue #166; spec authority OGC 23-002 §16.1
Severity: P1-Major (server-interop bug — connected-systems-go non-conformance)
Category: Bug Fix / Server Interop / Spec Conformance
Ownership: Ours (CSAPI module — csapi/formats/part2.ts + spec)
Phase 8 phase: C — Server-Interop Bug Fix


Goal

All Part 2 parsers extract cross-reference IDs from either the @id (scalar string) form OR the @link (object with href) form, per OGC 23-002 §16.1. The library is conformant for connected-systems-go servers (and any future server emitting the object form) without breaking interop with OpenSensorHub or Toolbox4OGC, which emit the scalar form.

Acceptance criterion (from P8-contribution-goal-and-definition.md): B1 — Part 2 parsers accept @link object form alongside @id scalar form; @id always wins when both are present.

Locked Decision

⚠️ This decision is locked. Do not re-litigate in this issue. Surface deviations to the user; do not silently re-decide.

Decision: Add a single extractCrossReferenceId(obj, fieldName) helper in csapi/formats/part2.ts (or its _helpers.ts) that:

  1. Returns the scalar @id value when present and non-empty (authoritative form — @id always wins).
  2. Otherwise returns the last path segment of @link.href when the @link object has a string href property (handles both full URLs and bare identifiers).
  3. Returns undefined otherwise.

Replace every inline obj['<field>@id'] extraction in the affected parsers with a call to this helper. Do NOT change the scalar-form precedence; existing @id-form fixtures and tests must continue to pass without modification.

Locked in:


Problem Statement

OGC 23-002 §16.1 (paraphrased): a resource may reference another resource by either:

  • "system@id": "0o123" — scalar string identifier, OR
  • "system@link": { "href": "https://api.example.com/systems/0o123", "title": "...", ... } — object form with href carrying the identifier (last URL segment) or a bare ID.

Servers MAY emit either form; clients MUST accept either. The library currently only accepts @id. Against connected-systems-go, which emits the object form for cross-references, the affected parsers silently drop the cross-reference ID — Datastream.systemId, Observation.datastreamId/foiId/samplingFeatureId, Command.controlStreamId, CommandStatus.commandId all come back undefined even when the server fully advertised the relationship.

Affected parsers / fields:

Parser Cross-reference fields
parseDatastream system@id / system@link.href
parseControlStream system@id / system@link.href
parseObservation datastream@id / datastream@link.href; foi@id / foi@link.href; samplingFeature@id / samplingFeature@link.href
parseCommand controlstream@id / controlstream@link.href
parseCommandStatus command@id / command@link.href

Affected code: src/ogc-api/csapi/formats/part2.ts — at least 5 inline obj['<field>@id'] extractions across these parsers.

Impact: Silent data loss for any consumer talking to a connected-systems-go (or future spec-conformant) server. The smoke test against cs-go will surface the divergence end-to-end.

Files to Modify

File Action Est. Lines Purpose
src/ogc-api/csapi/formats/part2.ts Modify ~50 Add extractCrossReferenceId(obj, fieldName) helper; replace 5+ inline obj['<field>@id'] extractions in parseDatastream, parseControlStream, parseObservation (3 fields), parseCommand, parseCommandStatus
src/ogc-api/csapi/formats/part2.spec.ts Modify ~250 Add the 5-test pattern (scalar, object, both, bare-href, neither) for each affected parser/field — ~25–30 new test cases total
fixtures/csapi/part2/<resource>-link-form.json Add (optional) ~variable Per-parser fixtures exercising the @link form to lock the contract end-to-end

Cross-reference P8-implementation-guide §5.1 for the canonical files-modified list. If you find yourself adding a file that isn't in the implementation guide, stop — surface the question.

Implementation Approach

Pull the canonical helper, replacement pattern, and test pattern from P8-implementation-guide §5.1.

New helper in src/ogc-api/csapi/formats/part2.ts (or _helpers.ts):

/**
 * Extracts a cross-reference ID from either the scalar `@id` form or the
 * object `@link` form, per OGC 23-002 §16.1.
 *
 * The `@link` form carries an `href` that may be a full URL or just an
 * identifier; the last path segment is used as the ID. When both forms
 * are present, `@id` wins (it is the authoritative scalar form).
 */
function extractCrossReferenceId(
  obj: Record<string, unknown>,
  fieldName: string
): string | undefined {
  // Form 1: scalar @id (authoritative; wins if both forms present)
  const idValue = obj[`${fieldName}@id`];
  if (typeof idValue === 'string' && idValue.length > 0) {
    return idValue;
  }
  // Form 2: object @link with href
  const linkValue = obj[`${fieldName}@link`];
  if (
    typeof linkValue === 'object' &&
    linkValue !== null &&
    typeof (linkValue as Record<string, unknown>).href === 'string'
  ) {
    const href = (linkValue as Record<string, unknown>).href as string;
    const lastSegment = href.split('/').filter(Boolean).pop();
    if (lastSegment && lastSegment.length > 0) {
      return lastSegment;
    }
  }
  return undefined;
}

Replacement pattern (apply at every cross-reference call site):

-      ...(typeof obj['system@id'] === 'string'
-        ? { systemId: obj['system@id'] as string }
-        : {}),
+      ...((() => {
+        const systemId = extractCrossReferenceId(obj, 'system');
+        return systemId !== undefined ? { systemId } : {};
+      })()),

Test pattern (5 cases per parser/field — multiply by ~5–6 fields = ~25–30 tests):

For each affected parser/field, add the canonical 5-test pattern verbatim from P8-implementation-guide §5.1:

  1. Parses ID from <field>@id (scalar form) — preserves existing behavior
  2. Parses ID from <field>@link.href (object form, OGC 23-002 §16.1) — new behavior
  3. Prefers @id over @link when both are present — locks the precedence invariant
  4. Handles @link.href as a bare identifier with no path — covers minimal-href edge case
  5. Returns undefined when neither @id nor @link is present — covers the empty-cross-reference case

Fixtures (optional): existing fixtures use the @id form and continue to pass unchanged. Adding one new fixture per parser exercising the @link form is allowed for end-to-end lock-in but is not required for the acceptance gate.

Scope — What NOT to Touch

Acceptance Criteria

  • extractCrossReferenceId(obj, fieldName) helper added to src/ogc-api/csapi/formats/part2.ts (or its _helpers.ts) with the canonical signature and JSDoc
  • Helper prefers @id (scalar) over @link (object) when both are present
  • Helper returns the last path segment of @link.href (handles full URLs and bare IDs)
  • Helper returns undefined when neither form is present or well-formed
  • All 5+ inline obj['<field>@id'] extractions in parseDatastream, parseControlStream, parseObservation (3 fields), parseCommand, and parseCommandStatus replaced with helper calls
  • 5-test pattern added for each affected parser/field — ~25–30 new test cases total
  • Existing @id-form fixtures and tests pass unchanged (regression-safe)
  • No CSAPIError subclass introduced; any helper-internal throw uses EndpointError (per B2's locked decision)
  • All modified files pass npx prettier --check
  • npm run typecheck exits 0
  • npm run lint exits 0
  • npm run test:browser exits 0 (specifically src/ogc-api/csapi/formats/part2.spec.ts — all new @link tests green; all existing tests still green)
  • npm run test:node exits 0

Acceptance Gate (verification command)

The Phase 8 roadmap defines a specific verification command for this task (P8-ROADMAP §Phase C Task C1). Paste the output of these commands on the issue before closing:

# 1. Targeted spec — all new @link tests green; all existing tests green
npm run test:browser -- src/ogc-api/csapi/formats/part2.spec.ts

# 2. Confirm helper exists and is wired in at every cross-reference site
git grep -n "extractCrossReferenceId" -- src/ogc-api/csapi/formats/part2.ts
# Expected: 1 definition + ≥5 call sites

# 3. Confirm no inline obj['<field>@id'] extractions remain in the affected parsers
git grep -n "obj\['.*@id'\]" -- src/ogc-api/csapi/formats/part2.ts
# Expected: zero matches in the cross-reference extraction sites
#   (some unrelated @id usages may remain in resource-self-id extraction;
#    inspect each result before passing)

# 4. Full upstream QA suite (must mirror .github/workflows/qa.yml)
npm run format:check
npm run typecheck
npm run lint
npm run test:browser
npm run test:node

Expected output:

  • part2.spec.ts run: every new test case green; every pre-existing test case still green
  • git grep confirms helper definition + ≥ 5 call sites; no remaining inline obj['...@id'] cross-reference extractions
  • All five upstream QA commands exit 0

Smoke-test marquee: Phase 8's smoke test (smoke-test-prompt-template-phase-8.md) Step 16 exercises the @link fallback against the live cs-go server. That step is the end-to-end interop validation; a green part2.spec.ts does not satisfy the smoke test, only the unit-level acceptance gate.

Dependencies

Mandatory. Walk the P8-ROADMAP dependency graph and fill in every applicable row.

Blocked by: Task B2 (EndpointError standardization). Recommended sequencing per the roadmap: B2 → C1, so any helper-internal EndpointError throw lands on the standardized type and avoids one revisit. Mechanically, C1 could run before B2; the chosen order saves work.
Blocks: Nothing directly. Task D1 is sequenced last regardless.
Related: Smoke-test Step 16 (live cs-go @link validation); deferred issue #171 (SWE Common-aware result extraction — explicitly NOT this task)

Roadmap dependency row: P8-ROADMAP §Phase C Task C1: "Effort: Small-Medium (~3 hours: helper + 5 call-site swaps + 25 tests + optional fixtures). Risk: Low (@id is the preferred branch; @link is purely additive fallback). Dependencies: Task B2 (so any EndpointError throws inside the new helper are already on the standardized type). Independent of Tasks A1–A4 and B1."

References

# Document What It Provides
1 P8-contribution-goal-and-definition.md Phase 8 goal, scope, acceptance criteria, locked decisions
2 P8-implementation-guide.md §5.1 Authoritative execution-level guide (this task)
3 P8-ROADMAP.md §Phase C Task C1 Task ordering, dependencies, acceptance gate
4 Issue #166 Authoritative "why" for this task
5 OGC 23-002 §16.1 Spec authority for @link / @id cross-reference forms
6 src/ogc-api/csapi/formats/part2.ts Receives the helper + 5 call-site swaps
7 src/ogc-api/csapi/formats/part2.spec.ts Receives ~25–30 new test cases
8 smoke-test-prompt-template-phase-8.md Step 16 Live cs-go @link validation
9 Issue #171 Deferred — SWE Common result extraction (NOT this task)

Original Finding (preserved from pre-Phase-8 issue body)

Below is the original investigation/assessment that motivated this Phase 8 task. Preserved verbatim — Phase 8 absorbs this issue in place rather than creating a wrapper.

⚠️ Status Update — 2026-04-28 (Phase 8 review)

This issue has been validated against the live source tree and accepted for Phase 8 scope. During validation, two material errors in the original report and one design recommendation were corrected. Read this update first; the original report below is preserved for audit history but is partially superseded by the points here.

Corrections to original report

  1. Affected-site count is 6, not 5. The original report stated parseBaseStream() was already fixed for system@id / system@link in commit 2f0869a. That is incorrect:

    • Commit 2f0869a does not exist in this repository's history.
    • 2f0869a is a commit in the consumer demo app OS4CSAPI/ogc-csapi-explorer, where a tryLinkFallback() workaround was added in application code — not a library fix.
    • Current parseBaseStream() (src/ogc-api/csapi/formats/part2.ts lines 97-99) extracts only obj['system@id'] with no system@link fallback.
    • The doc comments at lines 197 and 282 describe both system@id and system@link as cross-reference fields — implying both are handled — but the implementation only handles @id. This documentation/implementation drift likely originated in commit f25acf0 ("refactor(part2): extract parseBaseStream helper") and was not caught in subsequent reviews because no test fixture exercised @link-only input.
    • system@id in parseBaseStream must be included as the 6th affected site. Treat all 6 sites as a single atomic patch.
  2. No prior precedent exists in the library. Because correction Phase 1.1: Create Type System (csapi/model.ts) #1 is true, the original report's claim that we should "follow the existing parseBaseStream() pattern" is unfounded — there is no existing pattern. The pattern is being established by this fix.

  3. Recommended approach: Option B (file-local helper), not Option A.

Updated file-modification estimate

File Action Est. Lines Purpose
src/ogc-api/csapi/formats/part2.ts Modify ~35-45 Add file-local extractIdFromRef() helper + call from all 6 cross-reference extraction sites (system@id in parseBaseStream, plus the 5 originally listed)
src/ogc-api/csapi/formats/part2.spec.ts Modify ~50 Add @link-only fixture cases for each of the 6 sites; add the "both @id and @link present → @id wins" precedence test
docs/governance/known-server-quirks.md Modify ~5 One-line entry documenting the Go/PostGIS CSAPI server emits @link-only for cross-reference fields
docs/research/references.md Modify (optional) ~10 If the Go/PostGIS server becomes a recurring test target, add it under "Live Infrastructure"

Updated acceptance criteria

Add to the original list:

  • parseDatastream({ ..., 'system@link': { href: 'systems/abc' } }) produces systemId: 'abc' (the 6th site)
  • parseControlStream({ ..., 'system@link': { href: 'systems/abc' } }) produces systemId: 'abc' (via parseBaseStream)
  • extractIdFromRef() correctly handles href values containing query strings, trailing slashes, and fragments
  • extractIdFromRef() is not exported from part2.ts (file-local only)
  • docs/governance/known-server-quirks.md is updated with a one-line entry for the Go/PostGIS server's @link-only behavior

Why the burden is ours (clarification)

camptocamp/ogc-client upstream has zero CSAPI parser code; 100% of part2.ts was authored as part of this contribution. OGC 23-001 §16 and 23-002 §16.1 define both @id (scalar string) and @link (object) as valid encoding forms — servers may legitimately emit either. The Go/PostGIS server is spec-compliant, not non-compliant. A spec-compliant client must accept both forms; we currently do not. Per AI_OPERATIONAL_CONSTRAINTS.md precedence (OGC specifications → everything else), accepting both forms is mandatory, and the gap is ours to fix.

How this gap escaped 6 months of review

This is documented for institutional learning — see the post-update analysis in the comment thread below.


Finding

Part 2 parsers extract cross-reference IDs only from flat @id fields and silently discard servers that provide @link objects instead — causing datastreamId, featureOfInterestId, controlStreamId, and commandId to be undefined when connected to compliant servers that use @link-only responses.

Review Source: Live integration testing against a Go CSAPI server (PostGIS-backed, OGC-compliant) from the ogc-csapi-explorer project
Severity: P1-Critical
Category: Type Safety / API Design
Ownership: Ours


Problem Statement

Five cross-reference extraction sites in part2.ts only check for flat @id string fields. The OGC 23-001/23-002 specs define two encoding forms for cross-resource references:

  1. @id — flat string containing the referenced resource's local ID (e.g., "datastream@id": "abc123")
  2. @link — object with href, optional uid, title, rt (e.g., "datastream@link": { "href": "datastreams/abc123" })

Servers may provide one or both. The Go CSAPI server (and potentially other compliant implementations) provides only @link for several cross-reference fields. Our parsers currently ignore @link entirely for these fields, producing undefined where a valid ID should exist.

Note: parseBaseStream() (used by parseDatastream and parseControlStream) was already fixed for system@id / system@link in commit 2f0869a. This issue covers the remaining 5 unfixed sites. (Superseded by Status Update above — there is no prior fix; all 6 sites are unfixed.)

Affected code — parseObservation() (lines ~510-518):

// src/ogc-api/csapi/formats/part2.ts — parseObservation()
...(typeof obj['datastream@id'] === 'string'
  ? { datastreamId: obj['datastream@id'] as string }
  : {}),                                              // ← no @link fallback
...(typeof obj['samplingFeature@id'] === 'string'
  ? { samplingFeatureId: obj['samplingFeature@id'] as string }
  : {}),                                              // ← no @link fallback
...(typeof obj['foi@id'] === 'string'
  ? { featureOfInterestId: obj['foi@id'] as string }
  : {}),                                              // ← no @link fallback

Affected code — parseCommand() (lines ~440-442):

// src/ogc-api/csapi/formats/part2.ts — parseCommand()
...(typeof obj['controlstream@id'] === 'string'
  ? { controlStreamId: obj['controlstream@id'] as string }
  : {}),                                              // ← no @link fallback

Affected code — parseCommandStatus() (lines ~587-589):

// src/ogc-api/csapi/formats/part2.ts — parseCommandStatus()
...(typeof obj['command@id'] === 'string'
  ? { commandId: obj['command@id'] as string }
  : {}),                                              // ← no @link fallback

Affected code — parseBaseStream() (lines ~97-99) — added by Status Update 2026-04-28

// src/ogc-api/csapi/formats/part2.ts — parseBaseStream()
...(typeof obj['system@id'] === 'string'
  ? { systemId: obj['system@id'] as string }
  : {}),                                              // ← no @link fallback

Scenario:

// Go CSAPI server returns this for an observation:
const raw = {
  id: "obs-001",
  resultTime: "2026-04-17T00:00:00Z",
  result: { magnitude: 4.2 },
  "datastream@link": { href: "datastreams/8872b356-..." },
  // Note: NO "datastream@id" field
};

const obs = parseObservation(raw);
// Expected: obs.datastreamId === "8872b356-..."
// Actual:   obs.datastreamId === undefined

Impact: Any consumer that relies on datastreamId, samplingFeatureId, featureOfInterestId, controlStreamId, or commandId to associate parsed resources will silently lose those associations when connected to a server that uses @link-only encoding. The ogc-csapi-explorer map view showed 0 datastreams/observations until the system@link fallback was added — the same class of failure will occur for these 5 remaining fields.

Ownership Verification

All affected code was written entirely as part of our CSAPI contribution. The camptocamp/ogc-client upstream has zero CSAPI parser code. The parseObservation(), parseCommand(), and parseCommandStatus() functions were authored in Phase 5.

Conclusion: This code is ours.

Files to Modify

(Original table — see Status Update for revised count and helper-doc additions.)

File Action Est. Lines Purpose
src/ogc-api/csapi/formats/part2.ts Modify ~25 Add @link fallback for all 5 cross-reference extraction sites
src/ogc-api/csapi/formats/part2.spec.ts Modify ~40 Add test cases for @link-only inputs in each affected parser

Proposed Solutions

Option A: Inline fallback at each site (Recommended) (Superseded — see Status Update; Option B is now recommended)

Follow the exact pattern already established in parseBaseStream() for system@link:

// Example for datastream@id in parseObservation():
...(typeof obj['datastream@id'] === 'string'
  ? { datastreamId: obj['datastream@id'] as string }
  : typeof obj['datastream@link']?.href === 'string'
    ? { datastreamId: (obj['datastream@link'].href as string).split('/').pop()! }
    : {}),

Apply this pattern to all 5 sites:

  • datastream@id → fallback to datastream@link.href
  • samplingFeature@id → fallback to samplingFeature@link.href
  • foi@id → fallback to foi@link.href
  • controlstream@id → fallback to controlstream@link.href
  • command@id → fallback to command@link.href

Pros: Consistent with existing parseBaseStream() pattern; minimal diff; no new helpers needed; each site is self-contained
Cons: Repeated pattern at 5 sites; .split('/').pop()! is fragile if href has query params
Effort: Small | Risk: Low

Option B: Shared extractIdFromRef() helper — RECOMMENDED (per Status Update 2026-04-28)

function extractIdFromRef(
  obj: Record<string, unknown>,
  field: string
): string | undefined {
  const idKey = `${field}@id`;
  const linkKey = `${field}@link`;
  if (typeof obj[idKey] === 'string') return obj[idKey] as string;
  const link = obj[linkKey];
  if (typeof link === 'object' && link !== null) {
    const href = (link as Record<string, unknown>).href;
    if (typeof href === 'string') {
      const url = new URL(href, 'http://placeholder');
      return url.pathname.replace(/\/+$/, '').split('/').pop() || undefined;
    }
  }
  return undefined;
}

Pros: DRY; handles query params/trailing slashes/fragments correctly via URL constructor; single point of maintenance; @id-priority rule encoded once
Cons: Slightly larger diff; may be premature abstraction if #110 will provide a general-purpose utility (N/A — #110 is DEFERRED)
Effort: Small | Risk: Low
Scope clarification (per Status Update): Helper must be file-local (not exported from part2.ts) to avoid premature public-API expansion.

Scope — What NOT to Touch

  • ❌ Do NOT modify parseBaseStream() — it already has the system@link fallback (commit 2f0869a) (Superseded — parseBaseStream IS to be modified; see Status Update.)
  • ❌ Do NOT modify model.ts interfaces — the existing datastreamId, controlStreamId, commandId fields are sufficient
  • ❌ Do NOT add new model fields (e.g., systemId on Observation) — that's a separate concern
  • ❌ Do NOT implement @link resolution/fetching utilities — that's tracked in DEFERRED — No @link / @id resolution utilities for cross-resource reference following #110 (DEFERRED)
  • ❌ Do NOT modify files outside the "Files to Modify" table above (as revised in Status Update)
  • ❌ Do NOT refactor adjacent code that isn't part of this finding
  • ❌ Do NOT export extractIdFromRef() — file-local only

Acceptance Criteria (original)

  • parseObservation({ ..., "datastream@link": { href: "datastreams/abc" } }) produces datastreamId: "abc"
  • parseObservation({ ..., "samplingFeature@link": { href: "samplingFeatures/def" } }) produces samplingFeatureId: "def"
  • parseObservation({ ..., "foi@link": { href: "fois/ghi" } }) produces featureOfInterestId: "ghi"
  • parseCommand({ ..., "controlstream@link": { href: "controlstreams/jkl" } }) produces controlStreamId: "jkl"
  • parseCommandStatus({ ..., "command@link": { href: "commands/mno" } }) produces commandId: "mno"
  • parseDatastream({ ..., "system@link": { href: "systems/pqr" } }) produces systemId: "pqr" (added in Status Update — 6th site)
  • parseControlStream({ ..., "system@link": { href: "systems/pqr" } }) produces systemId: "pqr" (added in Status Update — same parseBaseStream site)
  • When both @id and @link are present, @id takes priority (consistent across all 6 sites; previously claimed consistent with parseBaseStream — now this fix establishes the pattern)
  • extractIdFromRef() correctly handles href values containing query strings, trailing slashes, and fragments (added in Status Update)
  • extractIdFromRef() is not exported from part2.ts (added in Status Update)
  • docs/governance/known-server-quirks.md updated with one-line entry for Go/PostGIS server @link-only behavior (added in Status Update)
  • Existing tests still pass (npm test)
  • No lint errors (npm run lint)
  • All modified files pass npx prettier --check

Dependencies (original)

Blocked by: Nothing
Blocks: Nothing
Related: #103 (Part 2 parsers — preserves @id/@link fields); #108 (Part 1 interfaces — CSAPIResourceRef type); #110 (@link resolution utilities — DEFERRED, higher-level concern)


Operational Constraints

⚠️ MANDATORY: Before starting work on this issue, review docs/governance/AI_OPERATIONAL_CONSTRAINTS.md.

Key constraints:

  • Precedence: OGC specifications → AI Collaboration Agreement → This issue description → Existing code → Conversational context
  • No scope expansion: Fix the finding, nothing more
  • Minimal diffs: Prefer the smallest change that satisfies the acceptance criteria
  • Ask when unclear: If intent is ambiguous, stop and ask for clarification
  • Respect ownership: This code is ours — fix on the current working branch

References (original)

# Document What It Provides
1 src/ogc-api/csapi/formats/part2.ts L97-99 (parseBaseStream), L440-445 (parseCommand), L500-520 (parseObservation), L585-592 (parseCommandStatus) Affected code to modify — 6 sites total (Status Update correction)
2 OGC 23-001 §16 — JSON encoding for Part 1 resources Defines @link inline property format: { href, uid?, title?, rt? }
3 OGC 23-002 §16.1 — JSON encoding for Part 2 resources Defines @id inline property format (scalar string)
4 ogc-csapi-explorer commit 2f0869a Explorer-side workaround (tryLinkFallback()) for the gap this issue closes — not a prior library fix
5 #110@link resolution utilities (DEFERRED) Higher-level concern; this issue is the parser-level prerequisite
6 docs/governance/known-server-quirks.md To be updated with Go/PostGIS server @link-only encoding entry

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions