[C++] Add C++ SDK (RAII wrapper over the C FFI) by zlata-stefanovic-db · Pull Request #415 · databricks/zerobus-sdk

zlata-stefanovic-db · 2026-06-24T12:58:09Z

Summary

Adds a C++17 SDK under cpp/, a thin RAII wrapper over the Zerobus C FFI
(rust/ffi). It exposes the same gRPC streaming / OAuth / recovery / ingestion
engine as the other SDKs, with a C++-idiomatic surface:

Proto and JSON ingestion — single, batched, and fire-and-forget (*_nowait).
Arrow Flight ingestion (Beta).
Dynamic protobuf from Unity Catalog metadata (ProtoSchema) — build a
descriptor and encode records with no .proto file or protoc.
Custom auth via a HeadersProvider interface.
Failures surface as zerobus::ZerobusException (is_retryable()); resources
are RAII and move-only (handles freed exactly once).

Distribution

CMake + GitHub Releases only — no package manager (no Conan/vcpkg), mirroring the
C FFI:

Build from source via add_subdirectory / FetchContent, or
find_package(zerobus) against an install tree or a prebuilt per-platform
archive attached to a cpp/v* GitHub Release. cmake --install produces that
archive: headers, libzerobus_cpp, the bundled libzerobus_ffi, and the
find_package package config.

Testing

Hermetic, dependency-free unit suite (tiny in-repo harness — no framework,
package manager, or network), run via make test. Covers marshalling, RAII /
move, CResult → exception mapping, the headers trampoline, config
conversion, and the proto-schema round trip.
ZEROBUS_SANITIZE CMake option + a sanitize CI job that runs the suite under
AddressSanitizer (with ASAN_OPTIONS=detect_leaks=0, since the core's global
tokio runtime is intentionally never freed) to catch the memory bugs a thin
FFI wrapper is prone to.

Rationale for not adding a simulated-server/integration test: the C++ layer is a
thin forwarder; the server behavior is covered end-to-end by the Rust mock tests,
and the same C FFI binding path is exercised end-to-end by the Go integration
tests. ASan targets the only genuinely C++-specific risk (memory/lifetime in the
glue) without adding dependencies.

Docs and examples

README, per-API doc comments on the public headers, file- and implementation-
level comments in src/, runnable examples/, and cpp/CLAUDE.md.

Test plan

make test — hermetic unit suite.
make test SANITIZE=address — same suite under AddressSanitizer (new
sanitize CI job).
Verified a separate consumer project can find_package(zerobus) and link
zerobus::zerobus from a cmake --install tree.

Follow-ups (not in this PR)

release-zerobus-sdk-cpp.yml (per-platform build → GitHub Release), to live in
the release pipeline.
The 0.1.0 CHANGELOG.md entry, written when the first release is cut.

Add a new C++17 SDK that wraps the Zerobus C FFI layer (rust/ffi/). This provides an idiomatic, modern C++ interface with RAII ownership, exception-based error handling, and support for Proto, JSON, and Arrow Flight ingestion. Key features: - RAII wrappers (Sdk, Stream, ArrowStream) — handles free themselves - Exception-based errors (ZerobusException with is_retryable()) - Proto and JSON ingestion (single, batch, fire-and-forget variants) - Arrow Flight streaming (Beta) - Dynamic protobuf via ProtoSchema::from_uc_json() - Custom authentication via HeadersProvider interface - CMake build system with automatic FFI building or prebuilt linking Architecture: - Pure wrapper — zero business logic duplication - Delegates all gRPC, OAuth, recovery to Rust core via C FFI - Links against libzerobus_ffi.a (built from rust/ffi/) - ~2,000 LOC wrapper + ~1,000 LOC tests/examples Updates: - Add cpp/ directory with full SDK implementation - Add CI workflow (.github/workflows/ci-cpp.yml) - Update CLAUDE.md to document C++ in architecture table - Update README.md to list C++ SDK - Update push.yml to trigger C++ CI on cpp/** changes Status: v0.1.0 initial release

Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>

Replace the GoogleTest dependency with a tiny in-repo header-only harness (tests/test_harness.hpp) so the test build needs no external framework, package manager, or network â�� keeping CI green on locked-down runners. Tests keep the same TEST/EXPECT_*/ASSERT_*/ EXPECT_THROW/FAIL API; the suite runs as one CTest entry. Also document that callers should prefer an explicit Stream/ArrowStream close() over the destructor, since close flushes synchronously and can block up to flush_timeout_ms. Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>

There is no prior release to diff against, so an itemized New Features / Bug Fixes list doesn't apply to v0.1.0. Reset NEXT_CHANGELOG to the empty section template; the initial release notes will be written into CHANGELOG.md when the first release is cut. Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>

Add install(EXPORT) plus a generated zerobus-config.cmake so downstream projects can consume the SDK via find_package(zerobus) and link zerobus::zerobus. Bundle the Rust C FFI archive and recreate its imported target in the config so the linked dependency resolves; set EXPORT_NAME so the installed target matches the in-build zerobus::zerobus alias. Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>

Add a Conan recipe (conanfile.py) that builds the SDK and its Rust C FFI dependency from source and exposes find_package(zerobus) / zerobus::zerobus, plus a test_package that links and runs the packaged library as a smoke test. The recipe handles the monorepo via export_sources(), copying both cpp/ and rust/ so BuildRustFfi.cmake still finds ../rust. Document the find_package and Conan (preview) consumption paths in the README. This is a first cut; the public distribution path is still being settled. Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>

- conanfile.py: derive the package version from version.hpp via set_version() so 'conan create cpp/' works (the recipe previously specified no version). - test_package: construct an SdkBuilder so the smoke test actually links the FFI archive, not just a header-only inline; correct the misleading comment. - examples/headers_provider.cpp: document the DATABRICKS_TOKEN env var it reads. - CLAUDE.md: add conanfile.py, test_package/, and zerobus-config.cmake.in to the structure tree. - sdk.hpp: reword Sdk::create() doc ('legacy' -> 'simpler'). Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>

Add module-style file headers to the implementation units (sdk.cpp, stream.cpp, arrow_stream.cpp, proto_schema.cpp, headers_callback.cpp) and the test files, matching the documentation style used in the Rust core, C FFI, and other SDKs. Each header describes what the unit implements, the FFI-forwarding/ResultGuard pattern, and ownership specifics; per-function API docs remain on the header declarations to avoid duplication and drift. Also adds two implementation comments (the empty-payload sentinel and the Arrow schema pointer helper). Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>

Bring the .cpp files up to the comment density of the Python and Rust SDKs: definition-site intent comments on the Sdk/SdkBuilder, Stream, ArrowStream, and ProtoSchema methods, plus inline notes on the FFI ownership and lifetime contracts (handle stealing on move, the empty-payload sentinel, copy-out-then- free for borrowed buffers, FFI-allocated buffers freed via the matching free function, blocking vs _nowait). Comments explain the implementation; the public API contract stays on the header declarations to avoid duplication and drift. Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>

Remove the Conan recipe and test_package: distribution is CMake-only (source via add_subdirectory/FetchContent/find_package) plus prebuilt per-platform archives attached to GitHub Releases, mirroring the C FFI. The CMake install/export (find_package(zerobus) + bundled FFI archive) is the foundation for both and is unchanged. Update README and CLAUDE.md accordingly. Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>

Add a ZEROBUS_SANITIZE CMake option (address/thread/undefined, off by default) that applies -fsanitize to the SDK, tests, and examples, plus a SANITIZE pass-through in the Makefile (make test SANITIZE=address). Add a sanitize job to ci-cpp.yml that runs the existing suite under AddressSanitizer to catch the memory bugs a thin FFI wrapper is prone to (use-after-free, double-free) with no added dependency. It sets ASAN_OPTIONS=detect_leaks=0 because the core's global tokio runtime is intentionally never freed. Document in CLAUDE.md. Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>

zlata-stefanovic-db · 2026-06-24T13:45:46Z

Splitting this PR into smaller, reviewable pieces:

1/6 meta: split/415/meta
2/6 docs: split/415/docs
3/6 issue templates: split/415/issue-templates
4/6 core (draft): split/415/core
5/6 tests (draft, on core): split/415/tests
6/6 examples (draft, on tests): split/415/examples

PRs 1-3 are independent; 4-6 are a stack (core -> tests -> examples). Keeping
this PR open as the umbrella until the pieces land.

…415) ## Summary License, notices, ignore rules, and changelog scaffolding for the C++ SDK (`cpp/`): `LICENSE`, `NOTICE`, `.gitignore`, `CHANGELOG.md`, `NEXT_CHANGELOG.md`. No code or build impact. Part of the #415 split (1/6). Independent — can merge in any order. Split from #415. Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>

## Summary Documentation for the C++ SDK: `cpp/README.md` (usage, quickstart, API overview) and `cpp/CLAUDE.md` (contributor guide), plus the C++ rows/sections added to the root `CLAUDE.md` and `README.md`. Docs only. Part of the #415 split (2/6). Independent — can merge in any order. Split from #415. Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>

## Summary Add **C++** to the SDK dropdown in the bug, feature, and task issue forms so issues can be filed against the C++ SDK. The value yields the `[C++]` title prefix via `issue-label.yml`, matching the commit convention. Part of the #415 split (3/6). Independent — can merge in any order. Split from #415. Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>

## Summary Core C++ SDK: the public headers (`include/`), implementation (`src/`), the Rust C FFI build glue (`cmake/`), the CMake build (library target, install / `find_package` export, sanitizer option), the `Makefile`, `.clang-format`, and the C++ CI (`ci-cpp.yml` + `push.yml` path filter). Builds the library. Part of the #415 split (4/6). ### Merge order Off `main`. **Tests (5/6) and examples (6/6) are stacked on this PR** and merge after it. The `add_subdirectory(tests)`/`(examples)` wiring is intentionally not here yet — it arrives with those PRs. Draft until the stack is reviewed. Split from #415. Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>

## Summary The C++ SDK test suite (`tests/`) — the tiny hermetic in-repo harness plus unit tests for config conversion, errors, the headers trampoline, proto schema, and the SDK — and the `add_subdirectory(tests)` wiring. Part of the #415 split (5/6). ### Merge order **Stacked on #4 (core).** Merge after core; this branch is based on it. Draft until the stack is reviewed. Split from #415. Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>

## Summary Runnable C++ SDK examples (`examples/`) — JSON single/batch, proto-from-UC-schema, custom headers provider, Arrow Flight — and the `add_subdirectory(examples)` wiring. Part of the #415 split (6/6). ### Merge order **Stacked on #5 (tests).** Merge last. Draft until the stack is reviewed. Split from #415. Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>

zlata-stefanovic-db · 2026-06-24T13:57:14Z

Splitting this PR into smaller, reviewable pieces:

1/6 meta: split/415/meta
2/6 docs: split/415/docs
3/6 issue templates: split/415/issue-templates
4/6 core (draft): split/415/core
5/6 tests (draft, on core): split/415/tests
6/6 examples (draft, on tests): split/415/examples

PRs 1-3 are independent; 4-6 are a stack (core -> tests -> examples). Keeping
this PR open as the umbrella until the pieces land.

## Summary Core C++ SDK: the public headers (`include/`), implementation (`src/`), the Rust C FFI build glue (`cmake/`), the CMake build (library target, install / `find_package` export, sanitizer option), the `Makefile`, `.clang-format`, and the C++ CI (`ci-cpp.yml` + `push.yml` path filter). Builds the library. Part of the #415 split (4/6). ### Merge order Off `main`. **Tests (5/6) and examples (6/6) are stacked on this PR** and merge after it. The `add_subdirectory(tests)`/`(examples)` wiring is intentionally not here yet — it arrives with those PRs. Draft until the stack is reviewed. Split from #415. Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>

zlata-stefanovic-db added 11 commits June 23, 2026 14:35

[C++] Harden FFI pointer and callback safety

54c15f2

Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>

zlata-stefanovic-db requested review from elenagaljak-db and teodordelibasic-db and removed request for teodordelibasic-db June 24, 2026 13:00

zlata-stefanovic-db self-assigned this Jun 24, 2026

zlata-stefanovic-db requested review from danilonajkov-db and teodordelibasic-db June 24, 2026 13:01

zlata-stefanovic-db mentioned this pull request Jun 24, 2026

[C++] Add release workflow and cut the 0.1.0 release #416

Open

5 tasks

zlata-stefanovic-db linked an issue Jun 24, 2026 that may be closed by this pull request

[C++] Add release workflow and cut the 0.1.0 release #416

Open

5 tasks

zlata-stefanovic-db closed this Jun 24, 2026

zlata-stefanovic-db mentioned this pull request Jun 25, 2026

[C++] Arrow Flight stream (Beta), split from #415 #428

Draft

zlata-stefanovic-db removed a link to an issue Jun 25, 2026

[C++] Add release workflow and cut the 0.1.0 release #416

Open

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[C++] Add C++ SDK (RAII wrapper over the C FFI)#415

[C++] Add C++ SDK (RAII wrapper over the C FFI)#415
zlata-stefanovic-db wants to merge 11 commits into
mainfrom
cpp-sdk

zlata-stefanovic-db commented Jun 24, 2026

Uh oh!

zlata-stefanovic-db commented Jun 24, 2026

Uh oh!

zlata-stefanovic-db commented Jun 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

zlata-stefanovic-db commented Jun 24, 2026

Summary

Distribution

Testing

Docs and examples

Test plan

Follow-ups (not in this PR)

Uh oh!

zlata-stefanovic-db commented Jun 24, 2026

Uh oh!

zlata-stefanovic-db commented Jun 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant