Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Apache Ossie provides a single JSON- and YAML-based specification that any tool

## What's in this repository

- [`core-spec/`](core-spec/) — The Ossie core specification (`spec.md`), the machine-readable schema (`spec.yaml`, `osi-schema.json`), and accompanying documentation.
- [`core-spec/`](core-spec/) — The Ossie core specification (`spec.md`), the machine-readable schema (`spec.yaml`, `ossie-schema.json`), and accompanying documentation.
- [`converters/`](converters/) — Reference converters that translate between Ossie and other semantic formats (e.g., dbt, GoodData, Polaris, Salesforce).
- [`examples/`](examples/) — Example semantic models, including a complete TPC-DS model.
- [`validation/`](validation/) — Tooling for validating semantic models against the Ossie schema.
Expand Down
4 changes: 2 additions & 2 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,7 @@ Enterprise adoption requires consistent identifiers, validation, and governance

**Related Issues:**

- [Issue #102 — Add semantic versioning and Git releases for core-spec/osi-schema.json](https://github.com/apache/ossie/issues/102)
- [Issue #102 — Add semantic versioning and Git releases for core-spec/ossie-schema.json](https://github.com/apache/ossie/issues/102)
- [Issue #92 — Community Implementation: Trust Control Center — Ossie-compatible governance & reconciliation platform](https://github.com/apache/ossie/issues/92)
- [Issue #87 — New Flags: Restricted and Internal only indicators](https://github.com/apache/ossie/issues/87)

Expand Down Expand Up @@ -428,7 +428,7 @@ Broad ecosystem adoption depends on practical tools that let teams validate thei

**Existing Artifacts:**

- [JSON Schema (osi-schema.json)](core-spec/osi-schema.json) — schema for structural validation
- [JSON Schema (ossie-schema.json)](core-spec/ossie-schema.json) — schema for structural validation
- [Validation Script (validate.py)](validation/validate.py) — validates Ossie YAML against JSON Schema, unique names, references, and SQL syntax
- [Snowflake Converter](converters/snowflake/) — Ossie → Snowflake Cortex Analyst YAML converter
- [GoodData Converter](converters/gooddata/) — bidirectional Ossie ↔ GoodData LDM converter
Expand Down
2 changes: 1 addition & 1 deletion converters/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -228,7 +228,7 @@ A converter should map `ai_context` when the target vendor supports equivalent c

### Step-by-Step Guide

1. **Validate input**: Use the [Ossie JSON Schema](../core-spec/osi-schema.json) and the [validation script](../validation/validate.py) to ensure the source Ossie model is valid before conversion.
1. **Validate input**: Use the [Ossie JSON Schema](../core-spec/ossie-schema.json) and the [validation script](../validation/validate.py) to ensure the source Ossie model is valid before conversion.

2. **Parse the Ossie model**: Load the YAML file and iterate over the top-level `semantic_model` entries.

Expand Down
2 changes: 1 addition & 1 deletion converters/databricks/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ version = "0.2.0.dev0"
description = "Databricks Unity Catalog Metric View <> Apache Ossie converter"
requires-python = ">=3.11"
classifiers = [
"License :: OSI Approved :: Apache Software License",
"License :: Ossie Approved :: Apache Software License",
"Programming Language :: Python :: 3",
]
dependencies = [
Expand Down
2 changes: 1 addition & 1 deletion converters/databricks/tests/fixtures/fixtureA_ossie.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
# specific language governing permissions and limitations
# under the License.

# yaml-language-server: $schema=../../../../core-spec/osi-schema.json
# yaml-language-server: $schema=../../../../core-spec/ossie-schema.json
#
# Fixture A -- all-native round-trip (Apache Ossie -> MV -> Apache Ossie).
# Star schema; every field maps to a native MV field. Documented losses on the
Expand Down
2 changes: 1 addition & 1 deletion converters/databricks/tests/fixtures/fixtureB_ossie.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
# specific language governing permissions and limitations
# under the License.

# yaml-language-server: $schema=../../../../core-spec/osi-schema.json
# yaml-language-server: $schema=../../../../core-spec/ossie-schema.json
#
# Fixture B -- expected Apache Ossie produced from fixtureB_metric_view.yaml. MV-only
# features are stashed in custom_extensions[DATABRICKS], keyed by their exact v1.1
Expand Down
2 changes: 1 addition & 1 deletion converters/databricks/tests/test_roundtrip_properties.py
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ def test_mv_to_ossie_to_mv(self, mv):
assert_mv_roundtrip(mv)


class TestOSIRoundTrip:
class TestOssieRoundTrip:
@given(ossie=ossie_models())
@_SETTINGS
def test_ossie_to_mv_to_ossie(self, ossie):
Expand Down
26 changes: 13 additions & 13 deletions converters/dbt/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@ Converts between dbt's [MetricFlow Semantic Interface](https://docs.getdbt.com/d

Both conversion directions are supported:

- `msi-to-osi` — `semantic_manifest.json` (dbt output) → Ossie YAML
- `osi-to-msi` — Ossie YAML → `semantic_manifest.json`
- `msi-to-ossie` — `semantic_manifest.json` (dbt output) → Ossie YAML
- `ossie-to-msi` — Ossie YAML → `semantic_manifest.json`

## Requirements

Expand Down Expand Up @@ -57,21 +57,21 @@ dbt parse
Then convert to Ossie YAML:

```bash
ossie-dbt msi-to-osi -i target/semantic_manifest.json -o semantic_model.yaml
ossie-dbt msi-to-ossie -i target/semantic_manifest.json -o semantic_model.yaml
```

By default the Ossie semantic model is named `semantic_model`. Override it with `--model-name`:

```bash
ossie-dbt msi-to-osi -i target/semantic_manifest.json -o semantic_model.yaml --model-name my_project
ossie-dbt msi-to-ossie -i target/semantic_manifest.json -o semantic_model.yaml --model-name my_project
```

Conversion issues (e.g. dropped CONVERSION or PRIVATE metrics) are printed as warnings to stderr. The output file is still written.

### Apache Ossie → dbt

```bash
ossie-dbt osi-to-msi -i semantic_model.yaml -o semantic_manifest.json
ossie-dbt ossie-to-msi -i semantic_model.yaml -o semantic_manifest.json
```

Produces a `semantic_manifest.json` that metricflow can load.
Expand All @@ -80,31 +80,31 @@ Produces a `semantic_manifest.json` that metricflow can load.

```bash
ossie-dbt --help
ossie-dbt msi-to-osi --help
ossie-dbt osi-to-msi --help
ossie-dbt msi-to-ossie --help
ossie-dbt ossie-to-msi --help
```

## Python API

```python
from ossie_dbt import MSIToOSIConverter, OSIToMSIConverter
from ossie_dbt import MSIToOssieConverter, OssieToMSIConverter
from metricflow_semantics.model.dbt_manifest_parser import parse_manifest_from_dbt_generated_manifest

# dbt → Apache Ossie
manifest = parse_manifest_from_dbt_generated_manifest(Path("target/semantic_manifest.json").read_text())
result = MSIToOSIConverter().convert(manifest, osi_model_name="my_project")
result = MSIToOssieConverter().convert(manifest, ossie_model_name="my_project")

for issue in result.issues:
print(f"[warning] {issue.issue_type.value}: {issue.element_name}")

osi_yaml = result.output.to_osi_yaml()
ossie_yaml = result.output.to_ossie_yaml()

# Apache Ossie → dbt
import yaml
from ossie import OSIDocument
from ossie import OssieDocument

document = OSIDocument.model_validate(yaml.safe_load(Path("semantic_model.yaml").read_text()))
result = OSIToMSIConverter().convert(document)
document = OssieDocument.model_validate(yaml.safe_load(Path("semantic_model.yaml").read_text()))
result = OssieToMSIConverter().convert(document)
manifest_json = result.output.model_dump_json(by_alias=True, exclude_none=True, indent=2)
```

Expand Down
8 changes: 4 additions & 4 deletions converters/dbt/src/ossie_dbt/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,13 @@
# under the License.

from ossie_dbt.converter_issues import ConverterIssue, ConverterIssueType, ConverterResult
from ossie_dbt.msi_to_osi import MSIToOSIConverter
from ossie_dbt.osi_to_msi import OSIToMSIConverter
from ossie_dbt.msi_to_ossie import MSIToOssieConverter
from ossie_dbt.ossie_to_msi import OssieToMSIConverter

__all__ = [
"ConverterIssue",
"ConverterIssueType",
"ConverterResult",
"MSIToOSIConverter",
"OSIToMSIConverter",
"MSIToOssieConverter",
"OssieToMSIConverter",
]
44 changes: 22 additions & 22 deletions converters/dbt/src/ossie_dbt/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@
"""CLI entry point for the ossie-dbt converter.

Usage:
ossie-dbt msi-to-osi -i semantic_manifest.json -o output.yaml
ossie-dbt osi-to-msi -i input.yaml -o semantic_manifest.json
ossie-dbt msi-to-ossie -i semantic_manifest.json -o output.yaml
ossie-dbt ossie-to-msi -i input.yaml -o semantic_manifest.json
"""

import argparse
Expand All @@ -29,10 +29,10 @@

import yaml

from ossie import OSIDocument
from ossie import OssieDocument
from ossie_dbt.converter_issues import ConverterIssueType
from ossie_dbt.msi_to_osi import MSIToOSIConverter
from ossie_dbt.osi_to_msi import OSIToMSIConverter
from ossie_dbt.msi_to_ossie import MSIToOssieConverter
from ossie_dbt.ossie_to_msi import OssieToMSIConverter

from metricflow_semantics.model.dbt_manifest_parser import parse_manifest_from_dbt_generated_manifest

Expand All @@ -50,30 +50,30 @@
}


def _cmd_msi_to_osi(args: argparse.Namespace) -> None:
def _cmd_msi_to_ossie(args: argparse.Namespace) -> None:
input_path = Path(args.input)
output_path = Path(args.output)

manifest = parse_manifest_from_dbt_generated_manifest(input_path.read_text())
result = MSIToOSIConverter().convert(manifest, osi_model_name=args.model_name)
result = MSIToOssieConverter().convert(manifest, ossie_model_name=args.model_name)

if result.issues:
for issue in result.issues:
verb = "was dropped" if issue.issue_type in _DROPPED_ISSUE_TYPES else "was converted with loss"
reason = _ISSUE_REASON[issue.issue_type]
print(f"[WARNING] {issue.issue_type.value}: {issue.element_name} {verb} during conversion because {reason}", file=sys.stderr)

output_path.write_text(result.output.to_osi_yaml())
output_path.write_text(result.output.to_ossie_yaml())
print(f"Written to {output_path}", file=sys.stderr)


def _cmd_osi_to_msi(args: argparse.Namespace) -> None:
def _cmd_ossie_to_msi(args: argparse.Namespace) -> None:
input_path = Path(args.input)
output_path = Path(args.output)

raw = yaml.safe_load(input_path.read_text())
document = OSIDocument.model_validate(raw)
result = OSIToMSIConverter().convert(document)
document = OssieDocument.model_validate(raw)
result = OssieToMSIConverter().convert(document)

output_path.write_text(result.output.model_dump_json(by_alias=True, exclude_none=True, indent=2))
print(f"Written to {output_path}", file=sys.stderr)
Expand All @@ -86,22 +86,22 @@ def main() -> None:
)
subparsers = parser.add_subparsers(dest="command", required=True)

msi_to_osi = subparsers.add_parser("msi-to-osi", help="Convert semantic_manifest.json → Ossie YAML")
msi_to_osi.add_argument("-i", "--input", required=True, metavar="FILE", help="Path to semantic_manifest.json")
msi_to_osi.add_argument("-o", "--output", required=True, metavar="FILE", help="Path for output Ossie YAML")
msi_to_osi.add_argument(
msi_to_ossie = subparsers.add_parser("msi-to-ossie", help="Convert semantic_manifest.json → Ossie YAML")
msi_to_ossie.add_argument("-i", "--input", required=True, metavar="FILE", help="Path to semantic_manifest.json")
msi_to_ossie.add_argument("-o", "--output", required=True, metavar="FILE", help="Path for output Ossie YAML")
msi_to_ossie.add_argument(
"--model-name", default="semantic_model", metavar="NAME", help="Ossie semantic model name (default: semantic_model)"
)

osi_to_msi = subparsers.add_parser("osi-to-msi", help="Convert Ossie YAML → semantic_manifest.json")
osi_to_msi.add_argument("-i", "--input", required=True, metavar="FILE", help="Path to Ossie YAML")
osi_to_msi.add_argument("-o", "--output", required=True, metavar="FILE", help="Path for output semantic_manifest.json")
ossie_to_msi = subparsers.add_parser("ossie-to-msi", help="Convert Ossie YAML → semantic_manifest.json")
ossie_to_msi.add_argument("-i", "--input", required=True, metavar="FILE", help="Path to Ossie YAML")
ossie_to_msi.add_argument("-o", "--output", required=True, metavar="FILE", help="Path for output semantic_manifest.json")

args = parser.parse_args()
if args.command == "msi-to-osi":
_cmd_msi_to_osi(args)
elif args.command == "osi-to-msi":
_cmd_osi_to_msi(args)
if args.command == "msi-to-ossie":
_cmd_msi_to_ossie(args)
elif args.command == "ossie-to-msi":
_cmd_ossie_to_msi(args)


if __name__ == "__main__":
Expand Down
Loading