Version
opensemantic.base 0.42.8.post1000002004006,
opensemantic.lab 0.8.1.post1000002004003.
Summary
A data channel can aggregate sibling channels into a single value, for
sources that only expose flat variables and have no composite type of
their own. The aggregate is assembled as a plain dict and stays a
plain dict all the way into storage: it is never wrapped in a
characteristic, and its members are never converted to base units.
The same physical quantity is therefore stored in two different units
depending on whether it arrived as a standalone channel or as a member
of an aggregate.
Where it happens
opensemantic/lab/_controller_mixin.py, _handle_datachange_notification
builds the aggregate keyed by channel name:
val = {channel.name: _val}
for sr in srs:
val[sr.channel.name] = _v
opensemantic/base/_controller_mixin.py, _wrap_raw_value refuses to
type it:
if hasattr(value, "to_json") or isinstance(value, dict):
return value # dict passes through untyped
and _value_to_store_data serialises it verbatim:
if isinstance(value, dict):
return json.loads(json.dumps(value, default=str))
# scalars below this line get _wrap_raw_value(...).to_base()
So a scalar channel is normalised to its base unit, while the same
quantity inside an aggregate keeps whatever unit the source produced.
Server-side aggregation over the resulting column is wrong, and a
consumer has to know which path a row came from to interpret it.
What is needed
Adding a characteristic type is necessary but not sufficient. Four
things have to line up:
-
An aggregate characteristic exists. A minimal one is enough to
model the general case:
class Event(Characteristic):
timestamp: Optional[Time]
description: Optional[Text]
Members that are quantities then convert like any other value.
-
The aggregating channel declares it. The channel that carries the
subchannel list must have characteristic set to that type, so
_resolve_characteristic_class can find it.
-
_wrap_raw_value must construct it from a dict. Today the
isinstance(value, dict) early return makes 1 and 2 have no effect.
It needs to become something like
cls = self._resolve_characteristic_class(channel)
if isinstance(value, dict) and cls is not None:
try:
return cls(**value)
except Exception:
return value
Once the aggregate is a model, _value_to_store_data takes the
existing typed branch and to_base() handles the members. No change
needed there.
-
The member keys must map to the characteristic's properties. The
keys are currently channel.name, i.e. source-side names. Either the
subchannel names are required to match the property names of the
characteristic, which is simple but couples source naming to the
schema, or the mapping is made explicit on the subchannel entry.
Worth deciding deliberately, since option one turns a rename on the
device into silent data loss.
Sequencing
The live writer and any history migration have to change together,
otherwise old and new rows disagree. Convert history only after the
writer emits base units for aggregate members, and share one conversion
function between them so they cannot drift.
Version
opensemantic.base 0.42.8.post1000002004006,
opensemantic.lab 0.8.1.post1000002004003.
Summary
A data channel can aggregate sibling channels into a single value, for
sources that only expose flat variables and have no composite type of
their own. The aggregate is assembled as a plain
dictand stays aplain
dictall the way into storage: it is never wrapped in acharacteristic, and its members are never converted to base units.
The same physical quantity is therefore stored in two different units
depending on whether it arrived as a standalone channel or as a member
of an aggregate.
Where it happens
opensemantic/lab/_controller_mixin.py,_handle_datachange_notificationbuilds the aggregate keyed by channel name:
opensemantic/base/_controller_mixin.py,_wrap_raw_valuerefuses totype it:
and
_value_to_store_dataserialises it verbatim:So a scalar channel is normalised to its base unit, while the same
quantity inside an aggregate keeps whatever unit the source produced.
Server-side aggregation over the resulting column is wrong, and a
consumer has to know which path a row came from to interpret it.
What is needed
Adding a characteristic type is necessary but not sufficient. Four
things have to line up:
An aggregate characteristic exists. A minimal one is enough to
model the general case:
Members that are quantities then convert like any other value.
The aggregating channel declares it. The channel that carries the
subchannel list must have
characteristicset to that type, so_resolve_characteristic_classcan find it._wrap_raw_valuemust construct it from a dict. Today theisinstance(value, dict)early return makes 1 and 2 have no effect.It needs to become something like
Once the aggregate is a model,
_value_to_store_datatakes theexisting typed branch and
to_base()handles the members. No changeneeded there.
The member keys must map to the characteristic's properties. The
keys are currently
channel.name, i.e. source-side names. Either thesubchannel names are required to match the property names of the
characteristic, which is simple but couples source naming to the
schema, or the mapping is made explicit on the subchannel entry.
Worth deciding deliberately, since option one turns a rename on the
device into silent data loss.
Sequencing
The live writer and any history migration have to change together,
otherwise old and new rows disagree. Convert history only after the
writer emits base units for aggregate members, and share one conversion
function between them so they cannot drift.