Adding ontology converters

.github/workflows/converter-ontology-ci.yml

@@ -0,0 +1,60 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Converters Ontology CI
+
+on:
+  push:
+    branches: [ "main" ]
+    paths:
+      - 'converters/ontology/**'
+      - '.github/workflows/converter-ontology-ci.yml'
+  pull_request:
+    branches: [ "main" ]
+    paths:
+      - 'converters/ontology/**'
+      - '.github/workflows/converter-ontology-ci.yml'
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.11", "3.12", "3.13", "3.14"]
+
+    steps:
+      - name: Checkout project
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install dependencies
+        working-directory: converters/ontology
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements.lock
+          pip install -e ".[dev]"
+
+      - name: Unit Tests
+        working-directory: converters/ontology
+        run: |
+          pytest

converters/ontology/.gitignore

@@ -0,0 +1,59 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Python
+__pycache__/
+*.py[cod]
+*.pyo
+*.pyd
+
+# Virtual environments
+venv/
+.venv/
+env/
+
+# pyenv
+.python-version
+
+# Build / packaging
+dist/
+build/
+*.egg-info/
+*.egg
+.eggs/
+
+# Pytest
+.pytest_cache/
+.coverage
+htmlcov/
+
+# Mypy
+.mypy_cache/
+
+# Ruff
+.ruff_cache/
+
+# VS Code
+.vscode/
+*.code-workspace
+.history/
+
+# JetBrains (PyCharm, IntelliJ, etc.)
+.idea/
+*.iml
+*.iws
+*.ipr

converters/ontology/README.md

@@ -0,0 +1,114 @@
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one
+  or more contributor license agreements.  See the NOTICE file
+  distributed with this work for additional information
+  regarding copyright ownership.  The ASF licenses this file
+  to you under the Apache License, Version 2.0 (the
+  "License"); you may not use this file except in compliance
+  with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing,
+  software distributed under the License is distributed on an
+  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+  KIND, either express or implied.  See the License for the
+  specific language governing permissions and limitations
+  under the License.
+-->
+
+# Ossie Ontology Converters
+
+Converters between Ossie, Palantir, and Spec ontology formats.
+
+| Converter           | Direction |
+|---------------------|-----------|
+| `palantir_to_ossie` | Palantir ontology → Ossie model |
+| `ossie_to_spec`     | Ossie model → Spec YAML |
+| `spec_to_ossie`     | Spec YAML → Ossie model |
+
+## Prerequisites
+
+- [pyenv](https://github.com/pyenv/pyenv) — manages the Python version
+
+Install pyenv if you don't have it:
+
+```bash
+brew install pyenv
+```
+
+Add to your shell profile (`~/.zshrc` or `~/.bashrc`) and restart the shell:
+
+```bash
+export PYENV_ROOT="$HOME/.pyenv"
+export PATH="$PYENV_ROOT/bin:$PATH"
+eval "$(pyenv init -)"
+```
+
+## Setup
+
+```bash
+pyenv install 3.11
+pyenv local 3.11
+pip install --upgrade pip
+pip install virtualenv
+python -m virtualenv venv
+source ./venv/bin/activate
+pip install -r requirements.lock
+pip install -e ".[dev]"
+```
+
+## Generating / updating the lock file
+
+`requirements.lock` is produced by [pip-tools](https://github.com/jazzband/pip-tools) from `pyproject.toml`.
+Run this whenever you add or change a dependency:
+
+```bash
+pip-compile --output-file requirements.lock pyproject.toml
+```
+
+## Usage
+
+The package is importable as `ossie_ontology` after installation:
+
+```python
+from ossie_ontology.converter.palantir_to_ossie.converter import PalantirToOssieConverter
+from ossie_ontology.converter.ossie_to_spec.converter import OssieToSpecConverter
+from ossie_ontology.converter.spec_to_ossie.converter import SpecToOssieConverter
+```
+
+## Scripts
+
+### `scripts/palantir_to_ossie.py`
+
+Converts a Palantir ontology export (a `.zip` archive or an already extracted folder containing a Palantir ontology JSON and a `data_sets` folder of one or more dataset spec JSON files) into an Ossie-compliant YAML representation, printed to stdout.
+
+**Usage:**
+
+```bash
+python scripts/palantir_to_ossie.py path/to/palantir_export.zip
+# or an extracted folder:
+python scripts/palantir_to_ossie.py path/to/palantir_export/
+```
+
+Warnings are written to stderr; the Ossie YAML is written to stdout.
+
+**Environment variables (optional):**
+
+| Variable                  | Default    | Description                                              |
+|---------------------------|------------|----------------------------------------------------------|
+| `SNOWFLAKE_DATABASE_NAME` | `PALANTIR` | Snowflake database name used to qualify table references |
+| `SNOWFLAKE_SCHEMA_NAME`   | `PALANTIR` | Snowflake schema name used to qualify table references   |
+
+If already set in your environment they will be picked up automatically. To override them for a single run:
+
+```bash
+SNOWFLAKE_DATABASE_NAME=MY_DB SNOWFLAKE_SCHEMA_NAME=MY_SCHEMA \
+  python scripts/palantir_to_ossie.py path/to/palantir_export.zip
+```
+
+## Deactivating the environment
+
+```bash
+deactivate
+```

converters/ontology/pyproject.toml

@@ -0,0 +1,47 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+[project]
+name = "apache-ossie-ontology"
+version = "0.1.0"
+description = "Ossie ontology converters — Palantir → Ossie, Ossie → Spec, Spec → Ossie"
+readme = "README.md"
+authors = [
+    { name = "RelationalAI", email = "support@relational.ai" },
+]
+requires-python = ">= 3.11"
+dependencies = [
+    "pydantic",
+    "pyyaml",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest==9.0.3",
+    "pytest-snapshot",
+    "pip-tools",
+]
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+pythonpath = ["src"]
+
+[tool.pyright]
+pythonVersion = "3.11"

converters/ontology/requirements.lock

@@ -0,0 +1,21 @@
+#
+# This file is autogenerated by pip-compile with Python 3.11
+# by the following command:
+#
+#    pip-compile --output-file=requirements.lock pyproject.toml
+#
+annotated-types==0.7.0
+    # via pydantic
+pydantic==2.13.4
+    # via apache-ossie-ontology (pyproject.toml)
+pydantic-core==2.46.4
+    # via pydantic
+pyyaml==6.0.3
+    # via apache-ossie-ontology (pyproject.toml)
+typing-extensions==4.15.0
+    # via
+    #   pydantic
+    #   pydantic-core
+    #   typing-inspection
+typing-inspection==0.4.2
+    # via pydantic

converters/ontology/scripts/palantir_to_ossie.py

@@ -0,0 +1,69 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Description:
+#
+#   This script converts a Palantir ontology export into an Ossie compliant YAML
+#   representation of that ontology, using environment variables to configure the
+#   Snowflake database and schema names. The export may be supplied either as a
+#   zip archive or as an already extracted folder, and must contain:
+#     1. A Palantir ontology (JSON file) and
+#     2. A 'data_sets' folder containing one or more Palantir dataset specs (JSON files)
+#
+# Usage:
+#
+#   $ python palantir_to_ossie.py <path_to_zip_or_folder>
+# 
+# Environment variables used:
+#
+#   - SNOWFLAKE_DATABASE_NAME
+#   - SNOWFLAKE_SCHEMA_NAME
+#
+#   The tables that populate the ontology are named
+#   "{SNOWFLAKE_DATABASE_NAME}.{SNOWFLAKE_SCHEMA_NAME}.{TABLE_NAME}"
+#   where TABLE_NAME is the name of a data set that is referenced in
+#   the Palantir ontology.
+#
+# Outputs:
+#
+#   - stderr: Warnings
+#
+import os
+import sys
+from pathlib import Path
+
+from ossie_ontology.converter.palantir_to_ossie.converter import PalantirToOssieConverter
+from ossie_ontology.converter.ossie_to_spec.converter import OssieToSpecConverter
+
+from ossie_ontology.external.palantir.parser import PalantirParser
+
+if __name__ == "__main__":
+    db_name = os.environ.get("SNOWFLAKE_DATABASE_NAME", "PALANTIR")
+    schema_name = os.environ.get("SNOWFLAKE_SCHEMA_NAME", "PALANTIR")
+
+    if len(sys.argv) != 2:
+        sys.exit(f"Usage: {sys.argv[0]} <path to Palantir sources (.zip or folder)>")
+
+    path = Path(sys.argv[1])
+
+    parser = PalantirParser()
+    palantir_model = parser.parse(path)
+
+    ontology_model = PalantirToOssieConverter().convert(palantir_model, db_name, schema_name)
+
+    ossie_spec = OssieToSpecConverter.convert(ontology_model)
+    print(ossie_spec.dump_yaml())