Skip to content

Aggregated channel values bypass characteristic typing and base-unit conversion #10

Description

@simontaurus

Version

opensemantic.base 0.42.8.post1000002004006,
opensemantic.lab 0.8.1.post1000002004003.

Summary

A data channel can aggregate sibling channels into a single value, for
sources that only expose flat variables and have no composite type of
their own. The aggregate is assembled as a plain dict and stays a
plain dict all the way into storage: it is never wrapped in a
characteristic, and its members are never converted to base units.

The same physical quantity is therefore stored in two different units
depending on whether it arrived as a standalone channel or as a member
of an aggregate.

Where it happens

opensemantic/lab/_controller_mixin.py, _handle_datachange_notification
builds the aggregate keyed by channel name:

val = {channel.name: _val}
for sr in srs:
    val[sr.channel.name] = _v

opensemantic/base/_controller_mixin.py, _wrap_raw_value refuses to
type it:

if hasattr(value, "to_json") or isinstance(value, dict):
    return value          # dict passes through untyped

and _value_to_store_data serialises it verbatim:

if isinstance(value, dict):
    return json.loads(json.dumps(value, default=str))
# scalars below this line get _wrap_raw_value(...).to_base()

So a scalar channel is normalised to its base unit, while the same
quantity inside an aggregate keeps whatever unit the source produced.
Server-side aggregation over the resulting column is wrong, and a
consumer has to know which path a row came from to interpret it.

What is needed

Adding a characteristic type is necessary but not sufficient. Four
things have to line up:

  1. An aggregate characteristic exists. A minimal one is enough to
    model the general case:

    class Event(Characteristic):
        timestamp: Optional[Time]
        description: Optional[Text]

    Members that are quantities then convert like any other value.

  2. The aggregating channel declares it. The channel that carries the
    subchannel list must have characteristic set to that type, so
    _resolve_characteristic_class can find it.

  3. _wrap_raw_value must construct it from a dict. Today the
    isinstance(value, dict) early return makes 1 and 2 have no effect.
    It needs to become something like

    cls = self._resolve_characteristic_class(channel)
    if isinstance(value, dict) and cls is not None:
        try:
            return cls(**value)
        except Exception:
            return value

    Once the aggregate is a model, _value_to_store_data takes the
    existing typed branch and to_base() handles the members. No change
    needed there.

  4. The member keys must map to the characteristic's properties. The
    keys are currently channel.name, i.e. source-side names. Either the
    subchannel names are required to match the property names of the
    characteristic, which is simple but couples source naming to the
    schema, or the mapping is made explicit on the subchannel entry.
    Worth deciding deliberately, since option one turns a rename on the
    device into silent data loss.

Sequencing

The live writer and any history migration have to change together,
otherwise old and new rows disagree. Convert history only after the
writer emits base units for aggregate members, and share one conversion
function between them so they cannot drift.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions