Skip to content

Add OSI-Salesforce converter module #118

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
khush-bhatia merged 1 commit into from
May 1, 2026
Merged

Add OSI-Salesforce converter module #118

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
221 changes: 221 additions & 0 deletions converters/salesforce/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,221 @@
# OSI Salesforce Converter

A two-way converter between [OSI semantic models](../../core-spec/spec.md) and [Salesforce Semantic Model](https://developer.salesforce.com/docs/data/semantic-layer/guide/salesforce-semantic-model-schema.html).

This converter provides lossless, bidirectional conversion between OSI YAML format and Salesforce Semantic Model JSON format.

## Requirements

- **Java 17+**
- **Maven 3.6+** — required to build the jar

## Building

Build the executable jar from source:

```bash
mvn clean package
```

This produces a self-contained executable jar at `target/osi-salesforce-converter-0.1.0-SNAPSHOT.jar` with all dependencies bundled.

## Setup

Both schemas must be obtained and placed under `src/main/resources/schemas/` before building, so they get bundled into the jar.

### Salesforce Semantic Model Schema

1. Visit the [Salesforce Semantic Model Schema documentation](https://developer.salesforce.com/docs/data/semantic-layer/guide/salesforce-semantic-model-schema.html)
2. Copy the JSON schema content from the page
3. Save it to `src/main/resources/schemas/salesforce-semantic-model-schema.json`

### OSI Schema

1. Visit the [OSI schema on GitHub](https://github.com/open-semantic-interchange/OSI/blob/main/core-spec/osi-schema.json)
2. Copy the raw JSON contents
3. Save it to `src/main/resources/schemas/osi-schema.json`

## Usage

### Command Line

#### Import (Salesforce → OSI)

Convert a Salesforce Semantic Model JSON file to OSI YAML format:

```bash
java -jar target/osi-salesforce-converter-0.1.0-SNAPSHOT.jar toOSI input.json
# Output: Customer_Orders_Model.yaml (named after model's 'name' field)
# Created in the same directory as the input file
```

Example:
```bash
java -jar target/osi-salesforce-converter-0.1.0-SNAPSHOT.jar toOSI \
src/test/resources/examples/salesforceToOsi.json
# Output: src/test/resources/examples/Customer_Orders_Model.yaml
```

#### Export (OSI → Salesforce)

Convert an OSI YAML file to Salesforce Semantic Model JSON format:

```bash
java -jar target/osi-salesforce-converter-0.1.0-SNAPSHOT.jar toSF input.yaml
# Output: Customer_Orders_Model.json (named after model's 'apiName' field)
# Created in the same directory as the input file
```

Example:
```bash
java -jar target/osi-salesforce-converter-0.1.0-SNAPSHOT.jar toSF \
src/test/resources/examples/osiToSalesforce.yaml
# Output: src/test/resources/examples/Customer_Orders_Model.json
```

### Programmatic API

#### String Conversion

```java
import org.osi.converter.Converter;
import org.osi.converter.ConverterFactory;
import org.osi.converter.ConversionDirection;

Converter sfToOsi = ConverterFactory.getConverter(ConversionDirection.SALESFORCE_TO_OSI);
List<String> osiYamlList = sfToOsi.convert(salesforceJsonString);
String osiYaml = osiYamlList.get(0);

Converter osiToSf = ConverterFactory.getConverter(ConversionDirection.OSI_TO_SALESFORCE);
List<String> salesforceJsonList = osiToSf.convert(osiYamlString);
```

#### File Conversion

```java
import org.osi.converter.Converter;
import org.osi.converter.ConverterFactory;
import org.osi.converter.ConversionDirection;

import java.nio.file.Paths;

Converter sfToOsi = ConverterFactory.getConverter(ConversionDirection.SALESFORCE_TO_OSI);
sfToOsi.convert(Paths.get("input/model.json"), Paths.get("output/"));

Converter osiToSf = ConverterFactory.getConverter(ConversionDirection.OSI_TO_SALESFORCE);
osiToSf.convert(Paths.get("input/model.yaml"), Paths.get("output/"));
```

### Features

- **Schema-validated** - Input is validated against JSON Schema before processing
- **Lossless conversion** - Unmapped properties are preserved in `custom_extensions`
- **Bidirectional** - Full bi-directional conversion without data loss
- **Supports OSI Specification v0.1.1**

## Mapping Reference

### Import (Salesforce → OSI)

| Salesforce | OSI |
|------------|-----|
| `apiName` | `name` |
| `semanticDataObjects[]` | `datasets[]` |
| `semanticDataObjects[].apiName` | `datasets[].name` |
| `semanticDataObjects[].dataObjectName` | `datasets[].source` |
| `semanticDimensions[]` + `semanticMeasurements[]` | `fields[]` |
| `dataObjectFieldName` | `expression.dialects[].expression` |
| `semanticRelationships[]` | `relationships[]` |
| `criteria[]` | `from_columns` + `to_columns` |
| `semanticCalculatedMeasurements[]` | `metrics[]` |
| `semanticCalculatedDimensions[]` | Converted to `fields[]` if single data object dependency, otherwise stored in `custom_extensions` |
| `businessPreferences` | `ai_context` |
| Unmapped properties | `custom_extensions` (vendor: `SALESFORCE`) |

### Export (OSI → Salesforce)

| OSI | Salesforce |
|-----|------------|
| `name` | `apiName` |
| `datasets[]` | `semanticDataObjects[]` |
| `datasets[].name` | `semanticDataObjects[].apiName` |
| `datasets[].source` | `semanticDataObjects[].dataObjectName` |
| `fields[]` | Split into `semanticDimensions[]` and `semanticMeasurements[]` based on `expression` analysis |
| `expression.dialects[].expression` | `dataObjectFieldName` |
| `relationships[]` | `semanticRelationships[]` |
| `from_columns` + `to_columns` | `criteria[]` |
| `metrics[]` | `semanticCalculatedMeasurements[]` |
| `ai_context` | `businessPreferences` |
| `custom_extensions` (vendor: `SALESFORCE`) | Restored properties |

### Type Detection (Export)

Fields are automatically classified as dimensions or measurements based on expression analysis:

- **Measurements** — expressions containing SQL aggregation functions (`SUM`, `COUNT`, `AVG`, etc.)
- **Dimensions** — all other fields
- **Time dimensions** — Date/DateTime types set `dimension.is_time: true`

### Relationship Handling

**Unsupported relationships** (containing Formula or SemanticField types) are stored in `custom_extensions` at the model level rather than being converted to OSI relationships.

## Architecture

```
┌───────────────────────┐
│ OsiSalesforceConverter│
│ (CLI App) │
└───────────┬───────────┘
│
┌───────┴────────┐
│ ConverterFactory│
└───────┬────────┘
│
┌─────────────┴─────────────┐
│ ConverterImpl │
│ (Pipeline-based) │
│ │
│ • Configurable pipeline │
│ • Bidirectional mapping │
└─────────────┬─────────────┘
│
┌─────────────┴─────────────┐
│ Pipeline Handlers │
├───────────────────────────┤
│ • DatasetMappingHandler │
│ • FieldMappingHandler │
│ • RelationshipHandler │
│ • MetricMappingHandler │
│ • SemanticModelHandler │
└─────────────┬─────────────┘
│
┌─────────────┴─────────────┐
│ Support Components │
├───────────────────────────┤
│ • GenericMappingEngine │
│ • CustomExtensionHandler │
│ • SchemaValidator │
└───────────────────────────┘
```

**ConverterFactory** — Creates converter instances for specified direction

**Pipeline Configuration** — Handlers and direction-specific settings defined in `osi-salesforce-converter-config.yaml`

**GenericMappingEngine** — Path-based property mapping using `mappings.yaml` configuration

**CustomExtensionHandler** — Preserves unmapped Salesforce properties in OSI's `custom_extensions` for lossless bi-directional conversion

**SchemaValidator** — Validates input against JSON schemas before conversion

## Examples

See the test suite for sample models demonstrating various features:
- `src/test/resources/examples/osiToSalesforce.yaml` - OSI model example
- `src/test/java/org/osi/OsiToSalesforceConverterTest.java` - OSI to Salesforce conversion tests
- `src/test/java/org/osi/SalesforceToOsiConverterTest.java` - Salesforce to OSI conversion tests

## License

Apache License 2.0 — see [LICENSE](../../LICENSE).
107 changes: 107 additions & 0 deletions converters/salesforce/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>org.osi</groupId>
<artifactId>osi-salesforce-converter</artifactId>
<version>0.1.0-SNAPSHOT</version>
<packaging>jar</packaging>

<name>OSI Salesforce Converter</name>
<description>OSI Salesforce bidirectional semantic model converter</description>

<properties>
<maven.compiler.source>17</maven.compiler.source>
<maven.compiler.target>17</maven.compiler.target>
<maven.compiler.release>17</maven.compiler.release>
<java.version>17</java.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

<jackson.version>2.18.6</jackson.version>
<junit.version>5.11.4</junit.version>
<slf4j.version>2.0.16</slf4j.version>
<logback.version>1.5.25</logback.version>
<json-schema-validator.version>1.5.5</json-schema-validator.version>
<maven-shade-plugin.version>3.6.0</maven-shade-plugin.version>
</properties>

<dependencies>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>${jackson.version}</version>
</dependency>

<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-yaml</artifactId>
<version>${jackson.version}</version>
</dependency>

<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${slf4j.version}</version>
</dependency>

<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>${logback.version}</version>
<scope>runtime</scope>
</dependency>

<dependency>
<groupId>com.networknt</groupId>
<artifactId>json-schema-validator</artifactId>
<version>${json-schema-validator.version}</version>
</dependency>

<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>${junit.version}</version>
<scope>test</scope>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>${maven-shade-plugin.version}</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<createDependencyReducedPom>false</createDependencyReducedPom>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>org.osi.app.OsiSalesforceConverter</mainClass>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
</transformers>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>

</project>
Loading