Skip to content

Story 17.3 — Single-cluster quickstart MD twins (JDBC + ADBC + Arrow Flight SQL) — elasticsql #145

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
fupelaqu merged 4 commits into from
Jun 26, 2026
Merged

Story 17.3 — Single-cluster quickstart MD twins (JDBC + ADBC + Arrow Flight SQL) — elasticsql #145

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
867d2b3
docs: single-cluster quickstart MD twins for JDBC, ADBC, Arrow Flight…
fupelaqu Jun 25, 2026
f20f9d2
docs: remove private-repo source-path/module references from public docs
fupelaqu Jun 25, 2026
b96dc6e
docs(17.3): driver version sweep to 0.2.0 (tables, JARs, JFrog URLs),…
fupelaqu Jun 25, 2026
090bb94
Merge branch 'release-r1' into feature/17.3
fupelaqu Jun 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,10 +179,10 @@ Download the self-contained fat JAR for your Elasticsearch version:

| Elasticsearch Version | Artifact |
|-----------------------|----------------------------------------|
| ES 6.x | `softclient4es6-jdbc-driver-0.1.4.jar` |
| ES 7.x | `softclient4es7-jdbc-driver-0.1.4.jar` |
| ES 8.x | `softclient4es8-jdbc-driver-0.1.4.jar` |
| ES 9.x | `softclient4es9-jdbc-driver-0.1.4.jar` |
| ES 6.x | `softclient4es6-jdbc-driver-0.2.0.jar` |
| ES 7.x | `softclient4es7-jdbc-driver-0.2.0.jar` |
| ES 8.x | `softclient4es8-jdbc-driver-0.2.0.jar` |
| ES 9.x | `softclient4es9-jdbc-driver-0.2.0.jar` |

```text
JDBC URL: jdbc:elastic://localhost:9200
Expand All @@ -197,20 +197,20 @@ Driver class: app.softnetwork.elastic.jdbc.ElasticDriver
<dependency>
<groupId>app.softnetwork.elastic</groupId>
<artifactId>softclient4es8-jdbc-driver</artifactId>
<version>0.1.4</version>
<version>0.2.0</version>
</dependency>
```

**Gradle:**

```groovy
implementation 'app.softnetwork.elastic:softclient4es8-jdbc-driver:0.1.4'
implementation 'app.softnetwork.elastic:softclient4es8-jdbc-driver:0.2.0'
```

**sbt:**

```scala
libraryDependencies += "app.softnetwork.elastic" % "softclient4es8-jdbc-driver" % "0.1.4"
libraryDependencies += "app.softnetwork.elastic" % "softclient4es8-jdbc-driver" % "0.2.0"
```

The JDBC driver JARs are Scala-version-independent (no `_2.12` or `_2.13` suffix) and include all required dependencies.
Expand Down Expand Up @@ -298,9 +298,9 @@ resolvers += "Softnetwork" at "https://softnetwork.jfrog.io/artifactory/releases
// Choose your Elasticsearch version
libraryDependencies += "app.softnetwork.elastic" %% "softclient4es8-java-client" % "0.20.0"
// Add the community extensions for materialized views (optional)
libraryDependencies += "app.softnetwork.elastic" %% "softclient4es-community-extensions" % "0.1.4"
libraryDependencies += "app.softnetwork.elastic" %% "softclient4es-community-extensions" % "0.2.0"
// Add the JDBC driver if you want to use it from Scala (optional)
libraryDependencies += "app.softnetwork.elastic" %% "softclient4es-jdbc-driver" % "0.1.4"
libraryDependencies += "app.softnetwork.elastic" %% "softclient4es-jdbc-driver" % "0.2.0"
```

```scala
Expand Down
3 changes: 2 additions & 1 deletion documentation/client/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,9 @@ Welcome to the Client Engine Documentation. Navigate through the sections below:
- [Template Management](templates.md)
- [SQL Gateway Usage](gateway.md)
- [REPL Client](repl.md)
- [Arrow Flight SQL](arrow_flight_sql.md)
- [JDBC Driver](jdbc.md)
- [ADBC Driver](adbc_driver.md)
- [Arrow Flight SQL](arrow_flight_sql.md)
- [BI Tool Integration](bi_tools.md) — DBeaver, Superset, Grafana, Tableau, Power BI, Metabase (full guides on the website)
- [Telemetry & Privacy](telemetry.md)
- [Download Analytics](download_analytics.md)
Expand Down
69 changes: 59 additions & 10 deletions documentation/client/adbc_driver.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@ In-process columnar access to Elasticsearch — direct Arrow format, no separate

Download the self-contained fat JAR for your Elasticsearch version:

| Elasticsearch | Artifact |
|----------------|----------------------------------------|
| ES 6.x | `softclient4es6-adbc-driver-0.1.5.jar` |
| ES 7.x | `softclient4es7-adbc-driver-0.1.5.jar` |
| ES 8.x | `softclient4es8-adbc-driver-0.1.5.jar` |
| ES 9.x | `softclient4es9-adbc-driver-0.1.5.jar` |
| Elasticsearch | Artifact |
|----------------|-----------------------------------------------------|
| ES 6.x | `softclient4es6-adbc-driver-0.2.0.jar` |
| ES 7.x | `softclient4es7-adbc-driver-0.2.0.jar` |
| ES 8.x | `softclient4es8-adbc-driver-0.2.0.jar` |
| ES 9.x | `softclient4es9-adbc-driver-0.2.0.jar` |

### Maven / Gradle / sbt

Expand All @@ -36,20 +36,20 @@ Download the self-contained fat JAR for your Elasticsearch version:
<dependency>
<groupId>app.softnetwork.elastic</groupId>
<artifactId>softclient4es8-adbc-driver</artifactId>
<version>0.1.5</version>
<version>0.2.0</version>
</dependency>
```

**Gradle:**

```groovy
implementation 'app.softnetwork.elastic:softclient4es8-adbc-driver:0.1.5'
implementation 'app.softnetwork.elastic:softclient4es8-adbc-driver:0.2.0'
```

**sbt:**

```scala
libraryDependencies += "app.softnetwork.elastic" % "softclient4es8-adbc-driver" % "0.1.5"
libraryDependencies += "app.softnetwork.elastic" % "softclient4es8-adbc-driver" % "0.2.0"
```

---
Expand Down Expand Up @@ -98,6 +98,21 @@ adbc:elastic://<host>:<port>?bearer=<token>

---

## Your first JOIN

Elasticsearch can't JOIN across indices — SoftClient4ES does. **Free in Community: up to 2 cross-index JOINs per query** (a 3-table JOIN). ADBC routes the JOIN through the same embedded DuckDB engine as the JDBC driver, so the SQL is identical — only the connection setup differs:

```sql
SELECT e.name, e.salary, d.dept_name
FROM jdbc_join_emp e
JOIN jdbc_join_dept d ON e.dept_id = d.dept_id;
-- 5 rows (the orphan employee with dept_id = 99 is dropped by the INNER JOIN)
```

Set this as the statement's SQL (`stmt.setSqlQuery(...)`) and read the Arrow stream as shown in [Usage](#usage). For the full JOIN matrix, see the Cross-Index JOIN walkthrough (`../sql/joins.md`).

---

## Configuration

```hocon
Expand Down Expand Up @@ -131,9 +146,43 @@ elastic.credentials {

---

## Version compatibility

| Driver | Scala | ES versions | Clients | Process model |
|--------|-------|-------------|---------|---------------|
| ADBC | cross-built Scala 2.12 + 2.13 | ES 6.x / 7.x / 8.x / 9.x | Java/JVM only (polyglot → Flight SQL) | In-process |

The fat JARs are **Scala-version-independent for consumers** — they bundle their own Scala runtime, so you almost never need to think about the Scala axis. For non-JVM languages (Python, Go, DuckDB, C++), use the [Arrow Flight SQL server](arrow_flight_sql.md) with the Arrow project's standard `adbc_driver_flightsql` client.

---

## Licensing & self-selection

All client drivers (JDBC, ADBC, and the REPL) plus the Arrow Flight SQL sidecar are **free in Community**, including up to **2 cross-index JOINs per query** (a 3-table JOIN). A 4-table JOIN (a 3rd cross-index JOIN in one query) is rejected by the planner with a message ending `… Upgrade to Pro … See: https://portal.softclient4es.com/pricing`.

Multi-cluster **federation** — joining across *separate* ES clusters — is **Pro+** (`maxClusters` 1 / 5 / ∞). This quickstart covers the **single-cluster** shape, which is free.

---

## What does NOT work yet

Subqueries (`IN (SELECT …)`, `EXISTS`, scalar, derived tables) and CTEs (`WITH`) are not supported in the current release — they arrive in a later release. Write the JOIN explicitly instead. See the Known Limitations & Roadmap (`../sql/known_limitations.md`) for the full list.

---

## Going further

- [JDBC Driver](jdbc.md) — the row-based in-process alternative for any JDBC tool.
- [Arrow Flight SQL](arrow_flight_sql.md) — the gRPC server for polyglot and networked clients.
- Cross-Index JOIN walkthrough (`../sql/joins.md`) — the full JOIN matrix (rows 1/2/3) with worked examples.
- Multi-cluster federation operator guide (`federation_operator_guide.md`) — the Pro+ path: JOIN across separate ES clusters.
- Known Limitations & Roadmap (`../sql/known_limitations.md`) — what works in the current release vs what's coming.

---

## Telemetry

The ADBC driver sends one anonymous usage ping per day (no IP, no SQL, no PII). Opt out with `softclient4es.telemetry.enabled = false` in your HOCON config. See [Telemetry & Privacy](telemetry.md) for details.
The ADBC driver sends one anonymous usage ping per day (no IP, no SQL, no PII). Opt out with `softclient4es.telemetry.enabled = false` in your HOCON config, the `SOFTCLIENT4ES_TELEMETRY_ENABLED=false` environment variable, or `-Dsoftclient4es.telemetry.enabled=false`. ADBC has no connection-string opt-out — the `adbc:elastic://` URI carries no `telemetry` option. See [Telemetry & Privacy](telemetry.md) for details.

---

Expand Down
78 changes: 72 additions & 6 deletions documentation/client/arrow_flight_sql.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,15 @@ Zero-copy columnar access to Elasticsearch over gRPC — for DuckDB, Python, Apa

---

## Sidecar vs federation

There are two Flight SQL servers, and these quickstarts cover only the first:

- **Single-cluster sidecar** (port `32010`) — one server in front of **one** ES cluster. Cross-index JOINs run in the sidecar's embedded DuckDB engine. **This is what this page covers, and it is free in Community.**
- **Multi-cluster federation server** — a coordinator that fans out across **multiple** ES clusters to JOIN data living in different clusters. This is the **Pro+** path; see the federation operator guide (`federation_operator_guide.md`).

---

## Features

- **gRPC Protocol** — High-performance columnar data access over HTTP/2
Expand Down Expand Up @@ -40,15 +49,15 @@ Available images per ES version:
### Fat JAR

```bash
java -jar softclient4es8-arrow-flight-sql-<version>.jar
java -jar softclient4es8-arrow-flight-sql-0.2.0.jar
```

| Elasticsearch | Artifact |
|---------------|----------|
| ES 6.x | `softclient4es6-arrow-flight-sql-<version>.jar` |
| ES 7.x | `softclient4es7-arrow-flight-sql-<version>.jar` |
| ES 8.x | `softclient4es8-arrow-flight-sql-<version>.jar` |
| ES 9.x | `softclient4es9-arrow-flight-sql-<version>.jar` |
| ES 6.x | `softclient4es6-arrow-flight-sql-0.2.0.jar` |
| ES 7.x | `softclient4es7-arrow-flight-sql-0.2.0.jar` |
| ES 8.x | `softclient4es8-arrow-flight-sql-0.2.0.jar` |
| ES 9.x | `softclient4es9-arrow-flight-sql-0.2.0.jar` |

---

Expand All @@ -71,6 +80,24 @@ duckdb.sql("""
""")
```

## Your first JOIN

Elasticsearch can't JOIN across indices — the Flight SQL sidecar does. **Free in Community: up to 2 cross-index JOINs per query** (a 3-table JOIN). Both indices live in the **one** ES cluster the sidecar fronts; the JOIN runs in the sidecar's embedded DuckDB engine:

```python
cursor.execute("""
SELECT e.name, e.salary, d.dept_name
FROM jdbc_join_emp e
JOIN jdbc_join_dept d ON e.dept_id = d.dept_id
""")
table = cursor.fetch_arrow_table()
# 5 rows (the orphan employee with dept_id = 99 is dropped by the INNER JOIN)
```

To JOIN across *separate* ES clusters, deploy the federation server instead — see [Going further](#going-further).

---

## Apache Superset

Connect Superset via the Arrow Flight SQL dialect. Then query Elasticsearch directly from any dashboard:
Expand Down Expand Up @@ -138,9 +165,48 @@ elastic.credentials {

---

## In-process alternative: the ADBC driver

If you need columnar Arrow access **without** running a separate server, the in-process **ADBC driver** delivers the same SQL (including cross-index JOIN) inside your JVM — see the [ADBC Driver](adbc_driver.md) page. ADBC is Java/JVM in-process; polyglot clients (Python, Go, DuckDB, C++) connect to *this* Flight SQL server.

---

## Version compatibility

| Driver | Scala | ES versions | Clients | Process model |
|--------|-------|-------------|---------|---------------|
| Arrow Flight SQL | cross-built Scala 2.12 + 2.13 (server) | ES 6.x / 7.x / 8.x / 9.x | Any ADBC/Flight SQL client — Python, Go, DuckDB, C++, Grafana, Superset | Separate server (gRPC) |

The fat JARs are **Scala-version-independent for consumers** — you almost never need to think about the Scala axis.

---

## Licensing & self-selection

All client drivers (JDBC, ADBC, and the REPL) plus the Arrow Flight SQL sidecar are **free in Community**, including up to **2 cross-index JOINs per query** (a 3-table JOIN). A 4-table JOIN (a 3rd cross-index JOIN in one query) is rejected by the planner with a message ending `… Upgrade to Pro … See: https://portal.softclient4es.com/pricing`.

The single-cluster sidecar on this page is the free shape. Multi-cluster **federation** — joining across *separate* ES clusters — is **Pro+** (`maxClusters` 1 / 5 / ∞).

---

## What does NOT work yet

Subqueries (`IN (SELECT …)`, `EXISTS`, scalar, derived tables) and CTEs (`WITH`) are not supported in the current release — they arrive in a later release. Write the JOIN explicitly instead. See the Known Limitations & Roadmap (`../sql/known_limitations.md`) for the full list.

---

## Going further

- [ADBC Driver](adbc_driver.md) — the in-process columnar alternative (no separate server).
- Cross-Index JOIN walkthrough (`../sql/joins.md`) — the full JOIN matrix (rows 1/2/3) with worked examples.
- Multi-cluster federation operator guide (`federation_operator_guide.md`) — the Pro+ path: JOIN across separate ES clusters.
- Known Limitations & Roadmap (`../sql/known_limitations.md`) — what works in the current release vs what's coming.

---

## Telemetry

The Arrow Flight SQL sidecar sends one anonymous usage ping per day (no IP, no SQL, no PII). Opt out with `softclient4es.telemetry.enabled = false` in your server HOCON config. See [Telemetry & Privacy](telemetry.md) for details.
The Arrow Flight SQL sidecar sends one anonymous usage ping per day (no IP, no SQL, no PII). Opt out with `softclient4es.telemetry.enabled = false` in the server HOCON config, the `SOFTCLIENT4ES_TELEMETRY_ENABLED=false` environment variable, or `-Dsoftclient4es.telemetry.enabled=false`. The sidecar has no connection-string opt-out — the `grpc://` URI carries no `telemetry` option. See [Telemetry & Privacy](telemetry.md) for details.

---

Expand Down
Loading
Loading