Transform derived integration and derived validation

[jacopocinaark]

transform derived integration and derived validation

ref: AB#21929

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (6)

README.md:731

• TimeSerieData.rows is defined as a dict (and is serialized as a list of Key/Value objects). The README example passes a list of tuples, which won’t work with the SDK as written.

request = DerivedTransformQueryValidation(
    data=TimeSerieData(
        rows=[
            (datetime(2018, 10, 1, 0, 0), 100),
            (datetime(2018, 10, 1, 1, 0), 100)
        ],
        type=MarketDataType.ActualTimeSerie,
    ),

src/Artesian/MarketData/_Dto/DerivedTransformQueryValidation.py:24

• Docstring contains HTML-escaped operators (<, >=), which renders incorrectly in Python help/docs and is inconsistent with the other SQL examples.

                   SELECT Time, CASE WHEN EXTRACT(HOUR FROM (Time AT TIME ZONE 'UTC') AT TIME ZONE 'Europe/Rome')
                   < 10 THEN Value + 1 ELSE Value END AS Value FROM $table WHERE Time IS NOT NULL
                   SELECT Time, Value FROM $table WHERE Version IS NOT NULL AND((EXTRACT(hour FROM Version)
                   < 10 AND Time >= date_trunc('day', Version + interval '1 day')) OR(EXTRACT(hour FROM Version)
                   >= 10 AND Time >= date_trunc('day', Version + interval '2 day')))

src/Artesian/MarketData/_Dto/MarketDataEntityInput.py:114

• This exception message has a grammar issue and is slightly ambiguous (list vs single id). Clarify that Transform requires exactly one referenced id (length == 1).

        if (
            self.derivedCfg is not None
            and self.derivedCfg.derivedAlgorithm is DerivedAlgorithm.Transform
            and (
                self.derivedCfg.orderedReferencedMarketDataIds is None
                or len(self.derivedCfg.orderedReferencedMarketDataIds) != 1
            )
        ):
            raise Exception(
                f"DerivedCfg with {self.derivedCfg.derivedAlgorithm} algorithm "
                "must have exactly one orderedReferencedMarketDataIds."
            )

src/Artesian/MarketData/_Dto/init.py:40

• __all__ should include the new derived-transform DTOs to match the rest of the _Dto package exports.

    ArtesianMetadataFacetCount.__name__,
    DerivedCfg.__name__,
    CheckConversionResult.__name__,
    UnitOfMeasure.__name__,
    TimeSerieData.__name__,
]  # type: ignore

src/Artesian/MarketData/_Dto/DerivedTransformQueryValidationResponse.py:23

• Typo in docstring: transfored -> transformed.

        data: The time series data transfored by the query.

README.md:720

• MarketDataType is not exported from the top-level Artesian package (see src/Artesian/__init__.py); this import will fail. Import it from Artesian.MarketData (or the enum module) instead.

from Artesian.MarketData import MarketDataService
from Artesian.MarketData._Dto.DerivedTransformQueryValidation import DerivedTransformQueryValidation
from Artesian.MarketData._Dto.TimeSerieData import TimeSerieData
from Artesian import MarketDataType
from datetime import datetime

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (5)

README.md:719

• README sample imports MarketDataType from the root Artesian package, but it is not exported there (only under Artesian.MarketData). This example will raise ImportError as written.

from Artesian import MarketDataType

README.md:729

• README sample passes rows as a list of tuples, but TimeSerieData.rows is defined as a Dict[datetime, Optional[float]] and serializes to [{Key, Value}, ...]. The example should use a dict literal to match the SDK and API payload shape.

        rows=[
            (datetime(2018, 10, 1, 0, 0), 100),
            (datetime(2018, 10, 1, 1, 0), 100)
        ],

src/Artesian/MarketData/_Dto/DerivedTransformQueryValidationResponse.py:25

• Typo in docstring: transfored → transformed.

    Attributes:
        data: The time series data transformed by the query.
        error: The Error in case of invalid query validation.
        valid: The transformation is valid or invalid.

src/Artesian/MarketData/_Dto/init.py:5

• DerivedTransformQueryValidation / DerivedTransformQueryValidationResponse are new DTOs in this package but are not re-exported from Artesian.MarketData._Dto, unlike the other DTOs in this folder. This makes from Artesian.MarketData._Dto import DerivedTransformQueryValidation fail even though it’s part of the public MarketDataService surface.

from .MarketDataEntityInput import MarketDataEntityInput
from .MarketDataEntityOutput import MarketDataEntityOutput
from .CheckConversionResult import CheckConversionResult
from .UnitOfMeasure import UnitOfMeasure
from .TimeSerieData import TimeSerieData

src/Artesian/MarketData/_Dto/init.py:40

• The _Dto package defines __all__ to control star-imports, but the new DerivedTransform validation DTOs are not included. This makes them inconsistent with the rest of the DTOs in this folder.

    CheckConversionResult.__name__,
    UnitOfMeasure.__name__,
    TimeSerieData.__name__,
]  # type: ignore

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (6)

README.md:719

• The README example imports MarketDataType from Artesian, but MarketDataType is exposed from Artesian.MarketData (not the package root). As written, the snippet will raise ImportError for users.

from Artesian import MarketDataType

README.md:729

• TimeSerieData.rows is a dict (serialized as a list of {Key, Value} objects); the README example passes a list of tuples, which won't serialize to the expected request shape.

        rows=[
            (datetime(2018, 10, 1, 0, 0), 100),
            (datetime(2018, 10, 1, 1, 0), 100)
        ],

src/Artesian/MarketData/_Dto/init.py:5

• DerivedTransformQueryValidation and DerivedTransformQueryValidationResponse are new DTOs but they are not imported/exported from Artesian.MarketData._Dto.__init__, unlike the other DTOs. This makes from Artesian.MarketData._Dto import ... inconsistent for these types.

from .TimeSerieData import TimeSerieData

src/Artesian/MarketData/_Dto/init.py:39

• DerivedTransformQueryValidation / DerivedTransformQueryValidationResponse should be added to __all__ to match how other DTOs are exposed from this package.

    TimeSerieData.__name__,

src/Artesian/MarketData/_Dto/DerivedTransformQueryValidation.py:33

• transform is effectively required (enforced in __post_init__), but the type is declared as Optional[str] with a default of None. This creates a misleading API surface and redundant runtime validation.

    transform: Optional[str] = None

    def __post_init__(self) -> None:
        if self.transform is None:
            raise ValueError("transform must be provided for query validation.")

samples/TestDerivedDerivedTransformQueryValidation.py:1

• The sample filename TestDerivedDerivedTransformQueryValidation.py appears to have a duplicated "Derived" segment; this makes the sample harder to discover/align with the feature name.

from datetime import datetime

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (4)

README.md:731

• The README example imports MarketDataType from the top-level Artesian package, but it is exported from Artesian.MarketData. Also TimeSerieData.rows is a dict of datetime -> value (serialized as Rows: [{Key,Value}, ...]), so the example rows=[(datetime, value), ...] won't serialize to the expected payload.

from Artesian.MarketData import MarketDataService
from Artesian.MarketData._Dto.DerivedTransformQueryValidation import DerivedTransformQueryValidation
from Artesian.MarketData._Dto.TimeSerieData import TimeSerieData
from Artesian import MarketDataType
from datetime import datetime

mds = MarketDataService(cfg)

request = DerivedTransformQueryValidation(
    data=TimeSerieData(
        rows=[
            (datetime(2018, 10, 1, 0, 0), 100),
            (datetime(2018, 10, 1, 1, 0), 100)
        ],
        type=MarketDataType.ActualTimeSerie,
    ),

src/Artesian/MarketData/_Dto/DerivedTransformQueryValidation.py:33

• transform is validated as required in __post_init__, but the dataclass field is currently optional with a default of None, which makes it easy to accidentally construct an invalid request and also misleads type checkers. Make it a required constructor argument and validate for non-empty strings.

    data: TimeSerieData
    transform: Optional[str] = None

    def __post_init__(self) -> None:
        if self.transform is None:
            raise ValueError("transform must be provided for query validation.")

src/Artesian/MarketData/_Dto/init.py:8

• New DTOs for derived transform query validation are not re-exported from Artesian.MarketData._Dto, so from Artesian.MarketData._Dto import DerivedTransformQueryValidation won't work even though other DTOs are exported here. Consider exporting them alongside TimeSerieData for a consistent public surface.

from .MarketDataEntityInput import MarketDataEntityInput
from .MarketDataEntityOutput import MarketDataEntityOutput
from .CheckConversionResult import CheckConversionResult
from .UnitOfMeasure import UnitOfMeasure
from .TimeSerieData import TimeSerieData
from .CurveRangeEntity import CurveRangeEntity
from .PagedResult import PagedResultCurveRangeEntity
from .ArtesianSearchResults import ArtesianSearchResults

src/Artesian/MarketData/_Dto/init.py:40

• If you add DerivedTransformQueryValidation / DerivedTransformQueryValidationResponse to the module imports, they should also be included in __all__ to make the re-export effective and keep the export list accurate.

    DerivedCfg.__name__,
    CheckConversionResult.__name__,
    UnitOfMeasure.__name__,
    TimeSerieData.__name__,
]  # type: ignore

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (5)

src/Artesian/MarketData/_Dto/init.py:6

• DerivedTransformQueryValidation and DerivedTransformQueryValidationResponse are new MarketData DTOs, but they are not re-exported from Artesian.MarketData._Dto like the other DTOs. This makes imports inconsistent for SDK users.

from .CheckConversionResult import CheckConversionResult
from .UnitOfMeasure import UnitOfMeasure
from .TimeSerieData import TimeSerieData
from .CurveRangeEntity import CurveRangeEntity

README.md:719

• The README example imports MarketDataType from the top-level Artesian package, but Artesian.__init__ does not export it. This example will raise ImportError; import it from Artesian.MarketData instead.

from Artesian import MarketDataType

README.md:729

• The README example passes rows as a list of tuples, but TimeSerieData.rows is a dict keyed by datetime (and serializes to a list of {Key, Value} objects). The current snippet won’t type-check and won’t serialize to the expected API shape.

        rows=[
            (datetime(2018, 10, 1, 0, 0), 100),
            (datetime(2018, 10, 1, 1, 0), 100)
        ],

src/Artesian/MarketData/_Dto/init.py:40

• After adding the derived-transform DTOs to the module imports, they should also be included in __all__ so from Artesian.MarketData._Dto import * and documentation tooling behave consistently.

    DerivedCfg.__name__,
    CheckConversionResult.__name__,
    UnitOfMeasure.__name__,
    TimeSerieData.__name__,
]  # type: ignore

src/Artesian/MarketData/_Dto/DerivedCfg.py:16

• DerivedCfg gained a new transform field, but the docstring’s Attributes list doesn’t mention it, so generated docs will be incomplete/misleading.

        derivedAlgorithm: the derived configuration algorithm (MUV, Coalesce, Sum, Transform)
        version: the derived configuration version
        orderedReferencedMarketDataIds: the ordered reference MarketData Ids
        used in the computation

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (4)

README.md:719

• README example imports MarketDataType from the top-level Artesian package, but MarketDataType is exported from Artesian.MarketData (not Artesian.init). As written, from Artesian import MarketDataType will fail.

from Artesian import MarketDataType

README.md:729

• README example passes rows as a list of tuples, but TimeSerieData.rows is a Dict[datetime, float] and the SDK serializer expects dict-like input to produce the {Key,Value} list shape. The example should use a dict keyed by datetime.

        rows=[
            (datetime(2018, 10, 1, 0, 0), 100),
            (datetime(2018, 10, 1, 1, 0), 100)
        ],

src/Artesian/MarketData/_Dto/DerivedCfg.py:16

• DerivedCfg now supports the Transform algorithm, but the docstring Attributes section doesn’t document the new transform field, which makes the public API harder to understand.

        derivedAlgorithm: the derived configuration algorithm (MUV, Coalesce, Sum, Transform)
        version: the derived configuration version
        orderedReferencedMarketDataIds: the ordered reference MarketData Ids
        used in the computation

src/Artesian/MarketData/_Dto/DerivedTransformQueryValidation.py:33

• DerivedTransformQueryValidation treats transform as required (it raises in post_init), but the type is declared Optional[str]. This makes type hints misleading and forces runtime validation for something that can be enforced by the signature.

    transform: Optional[str] = None

    def __post_init__(self: "DerivedTransformQueryValidation") -> None:
        if self.transform is None:
            raise ValueError("transform must be provided for query validation.")

[Copilot]

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (5)

README.md:719

• The README example imports MarketDataType from the top-level Artesian package, but Artesian/__init__.py does not export it. This snippet will raise ImportError; import MarketDataType from Artesian.MarketData instead.

from Artesian import MarketDataType

README.md:729

• TimeSerieData.rows is a Dict[datetime, Optional[float]], but the README example passes a list of tuples. This won’t serialize to the expected [{Key, Value}, ...] payload used by the SDK; use a dict like other time-series examples in the README.

        rows=[
            (datetime(2018, 10, 1, 0, 0), 100),
            (datetime(2018, 10, 1, 1, 0), 100)
        ],

src/Artesian/MarketData/_Dto/DerivedTransformQueryValidation.py:33

• transform is required (enforced by __post_init__), but it is typed as Optional[str] with a default of None. This makes the public API/type hints inaccurate; make it a required str and drop the runtime check.

    transform: Optional[str] = None

    def __post_init__(self: "DerivedTransformQueryValidation") -> None:
        if self.transform is None:
            raise ValueError("transform must be provided for query validation.")

src/Artesian/MarketData/_Dto/init.py:5

• New DTOs for derived transform validation are not re-exported from Artesian.MarketData._Dto, while other DTOs are. This makes from Artesian.MarketData._Dto import DerivedTransformQueryValidation fail and is inconsistent with the module’s public surface; add the missing imports here.

from .MarketDataEntityInput import MarketDataEntityInput
from .MarketDataEntityOutput import MarketDataEntityOutput
from .CheckConversionResult import CheckConversionResult
from .UnitOfMeasure import UnitOfMeasure
from .TimeSerieData import TimeSerieData

src/Artesian/MarketData/_Dto/init.py:40

• __all__ should include the newly added derived transform validation DTOs so they are importable from Artesian.MarketData._Dto like the other DTOs.

    DerivedCfg.__name__,
    CheckConversionResult.__name__,
    UnitOfMeasure.__name__,
    TimeSerieData.__name__,
]  # type: ignore