Skip to content

Spec has no way to declare a non-additive metric #290

Description

@MikeNitsenko

The gap

Ossie can't say that a metric is non-additive along a dimension. Metrics are static SQL strings, and the spec doesn't define the join semantics they run under.

So two conforming consumers can get different numbers from the same model.

Example

datasets:
  - name: orders          # many
    primary_key: [id]
  - name: customers       # one
    primary_key: [id]

relationships:
  - name: orders_to_customers
    from: orders
    to: customers
    from_columns: [customer_id]
    to_columns: [id]

metrics:
  - name: total_lifetime_value
    expression:
      dialects:
        - dialect: ANSI_SQL
          expression: SUM(customers.lifetime_value)

Group total_lifetime_value by orders.status. The join repeats each customer row once per order, so the SUM over-counts by the customer's order count.

The model isn't wrong. The spec just doesn't say which answer is correct.

Two converters already hit this

dbt. MetricFlow has non_additive_dimension. Ossie has no field for it, so converters/dbt/src/ossie_dbt/osi_to_msi.py sets non_additive_dimension=None on export, and import records a CUMULATIVE_SEMANTICS_LOSS issue.

Cube. Cube fixes this at query time. When a cube is on the multiplied side of a join, it selects the distinct primary keys, joins them back to the measure's own cube, and aggregates there. Each row counts once. It refuses the query when the measures themselves span cubes that fan out. That's a runtime rewrite driven by declared primary keys, so a static expression can't inherit it.

That makes Cube → Ossie → Databricks unsafe for these metrics. A Databricks metric view has join cardinality and rely.at_most_one_match, but at_most_one_match asserts there is no fan-out. It isn't a dedup instruction. A SUM over a one_to_many join is computed on the flattened rows.

Options

Ossie already has enough to detect this: primary_key plus directional from(many)/to(one) relationships. What's missing is a way to declare it.

  1. Spec language only. Require consumers to deduplicate on the aggregated dataset's primary_key when a relationship fans it out. No schema change, but it changes what existing models mean.
  2. A metric-level field. non_additive_dimension (following dbt), or an additivity / grain marker.
  3. Leave it to custom_extensions and accept these metrics don't round-trip portably.

Is this worth pursuing, and which direction? I can write up a concrete proposal for whichever the list prefers.

Context

Came up building the Ossie ↔ Cube converter (#289), where it's the biggest fidelity issue. Possibly adjacent to #279, though that's about dialect drift rather than join semantics.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions