Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
ce6f4d9
feat(docs): add implementation plan for Go and Rust FFI bindings
martincousido Jun 17, 2026
908fd0c
feat(native-lib): initialize Go module for FFI bindings
martincousido Jun 17, 2026
4679f0e
test(native-lib): add failing test for Go Run() function
martincousido Jun 17, 2026
da1386b
feat(native-lib): implement Go FFI bindings for run_script
martincousido Jun 17, 2026
9cb26d4
test(native-lib): add test for Go Run with inputs
martincousido Jun 17, 2026
5fa585a
test(native-lib): add error handling test for Go bindings
martincousido Jun 17, 2026
55d2b38
docs(native-lib): add simple Go demo program
martincousido Jun 17, 2026
8b0f5cc
docs(native-lib): add README for Go bindings
martincousido Jun 17, 2026
c65276b
feat(native-lib): initialize Rust crate for FFI bindings
martincousido Jun 17, 2026
6910894
test(native-lib): add failing test for Rust run() function
martincousido Jun 17, 2026
1cb6b6e
feat(native-lib): implement Rust FFI bindings for run_script
martincousido Jun 17, 2026
fc7b6c5
test(native-lib): add test for Rust run with inputs
martincousido Jun 17, 2026
5663a66
test(native-lib): add error handling test for Rust bindings
martincousido Jun 17, 2026
12ce5d4
docs(native-lib): add simple Rust demo program
martincousido Jun 17, 2026
1617e66
docs(native-lib): add README for Rust bindings
martincousido Jun 17, 2026
3fd29cb
build(native-lib): add Gradle task for Go tests
martincousido Jun 17, 2026
daafca7
build(native-lib): add Gradle task for Rust tests
martincousido Jun 17, 2026
49826e0
docs(native-lib): add Go and Rust sections to main README
martincousido Jun 17, 2026
7b6f1c1
chore: ignore Go and Rust build artifacts
martincousido Jun 17, 2026
f3a33dd
docs(native-lib): add Go streaming demo stub
martincousido Jun 17, 2026
72f7ba5
docs(native-lib): add Rust streaming demo stub
martincousido Jun 17, 2026
1dc581e
feat(native-lib): add streaming support for Go and Rust bindings
martincousido Jun 17, 2026
72f73c0
claude-unleashed: 9534eb5e-8df9-4116-a2ce-a1c2d24ddaa2
Jun 17, 2026
af26a23
fix(native-lib): add explicit task inputs/outputs for Gradle dependen…
martincousido Jun 18, 2026
bcaeb2d
Test changes configuration in gradle
martincousido Jun 19, 2026
b25b601
fix(native-lib): comprehensive Go/Rust bindings audit fixes and impro…
martincousido Jun 24, 2026
81ec4ed
fix(native-lib): resolve checkptr violation and add race detector tests
martincousido Jun 26, 2026
39210e6
Add C bindings and architectural documentation from feat/native-lib-l…
martincousido Jun 26, 2026
5bd14fb
refactor(native-lib): modularize Rust bindings into ffi/result/stream…
martincousido Jun 26, 2026
99142b8
Add comprehensive branch comparison report
martincousido Jun 26, 2026
2091ed4
Update native-lib README to document all five language bindings
martincousido Jun 26, 2026
1b7a80a
Add merge summary documenting the native bindings integration
martincousido Jun 26, 2026
0528fc8
Add comprehensive build validation report
martincousido Jun 26, 2026
9595812
fix(native-lib): remove deprecated SetFileDescriptorLimit GraalVM flag
martincousido Jun 26, 2026
3de031d
docs: update build validation with GraalVM 24.x compatibility fix
martincousido Jun 26, 2026
eb7a61b
Add comprehensive demos for all language bindings
martincousido Jun 27, 2026
002c714
Add API quick reference comparing all five language bindings
martincousido Jun 27, 2026
0eaf998
W-20884161: Add Node.js API for DataWeave native library (#115)
svacas Jun 19, 2026
54b7792
Add comprehensive production hardening plan for native bindings
martincousido Jun 30, 2026
d952ff7
feat: production-harden all native bindings for external integration
martincousido Jun 30, 2026
710d572
docs: add comprehensive production hardening status report
martincousido Jun 30, 2026
eb28b8d
feat: add complete release artifact packaging for all bindings
martincousido Jun 30, 2026
088a10e
docs: add final delivery report
martincousido Jun 30, 2026
85c5142
chore: remove internal planning and audit reports
martincousido Jun 30, 2026
dc06124
docs: add PR summary for production hardening
martincousido Jun 30, 2026
bce3ad1
docs: add comprehensive guide for building and running Go/Rust bindings
martincousido Jun 30, 2026
cdecca8
docs: add comprehensive demo walkthrough for recording
martincousido Jun 30, 2026
1192b30
docs: add comprehensive integration plan for api-platform-api
martincousido Jun 30, 2026
3c9d88d
docs: add detailed execution guide for api-platform-api integration
martincousido Jun 30, 2026
050644c
docs: add build troubleshooting guide and verification script
martincousido Jun 30, 2026
d8f7681
fix: improve build verification and Go test configuration
martincousido Jun 30, 2026
01fb1b0
fix(c-binding): use absolute paths for library resolution in tests
martincousido Jun 30, 2026
96b4704
fix: address all critical, high, and medium severity findings from na…
martincousido Jun 30, 2026
81cbd21
fix: address all critical, high, and medium severity findings from na…
martincousido Jul 1, 2026
969c7f8
Merge master into feat/new-native-bindings
martincousido Jul 1, 2026
6088ae9
Changing github workflows
martincousido Jul 2, 2026
4774410
fix(c-binding): add Windows platform support for native C library
martincousido Jul 3, 2026
a08db5f
fix(c-binding): remove Windows-incompatible unistd.h from test file
martincousido Jul 3, 2026
0df8824
fix(c-binding): configure CMake to avoid Debug/Release subdirectories…
martincousido Jul 3, 2026
84c7243
fix(c-binding): properly configure output directories for Visual Stud…
martincousido Jul 3, 2026
df10e6c
fix(c-binding): enable symbol export for Windows DLL to generate impo…
martincousido Jul 3, 2026
5cd4830
fix(c-binding): use CMake generator expression for test command to su…
martincousido Jul 3, 2026
ad19a98
fix(c-binding): pass -C Release flag to CTest on Windows
martincousido Jul 3, 2026
9991658
fix(ci): use native cmd shell for Rust tests on Windows
martincousido Jul 3, 2026
eebbf30
fix(go-binding): create libdwlib symlink for standard linking on Unix
martincousido Jul 3, 2026
2105196
docs: add implementation plan for CI native bindings fix
martincousido Jul 3, 2026
3187c94
fix(ci): resolve Windows Rust linker PATH conflict in main build step
martincousido Jul 3, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 108 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
## Description

<!-- Provide a clear and concise description of what this PR does -->

## Motivation

<!-- Why is this change needed? What problem does it solve? Link any related issues. -->

Fixes #(issue)

## Type of Change

<!-- Mark the relevant option with an "x" -->

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring
- [ ] Test addition/update
- [ ] Build/CI change

## Changes Made

<!-- List the main changes in this PR -->

-
-
-

## Testing

<!-- Describe the tests you ran and how to reproduce them -->

### Test Coverage

- [ ] Added new tests for new functionality
- [ ] Updated existing tests for changed functionality
- [ ] All tests pass locally (`./gradlew test`)
- [ ] Integration tests pass (if applicable)

### Manual Testing

<!-- Describe any manual testing performed -->

**Steps to test:**
1.
2.
3.

**Test environment:**
- OS:
- Language version:
- DataWeave CLI version:

## Documentation

- [ ] Updated README (if needed)
- [ ] Updated API documentation (if needed)
- [ ] Updated CHANGELOG.md (for user-facing changes)
- [ ] Added/updated code comments (for complex logic)
- [ ] Updated language binding docs (if applicable)

## Checklist

<!-- Review these items before submitting -->

- [ ] My code follows the project's coding standards
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings or errors
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing tests pass locally with my changes
- [ ] Any dependent changes have been merged and published

## Breaking Changes

<!-- If this PR includes breaking changes, list them here with migration guidance -->

- [ ] This PR includes breaking changes

**Breaking changes:**
-

**Migration guide:**
-

## Screenshots (if applicable)

<!-- Add screenshots to help explain your changes, especially for UI-related PRs -->

## Additional Context

<!-- Add any other context about the PR here -->

## Reviewer Notes

<!-- Anything specific you want reviewers to focus on? -->

---

**For Maintainers:**
- [ ] Version bumped (if needed)
- [ ] Release notes prepared (if needed)
- [ ] CI/CD checks pass
- [ ] Ready to merge
1 change: 0 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,6 @@ jobs:
with:
name: dwlib-${{env.NATIVE_VERSION}}-${{runner.os}}
path: |
native-lib/python/src/dataweave/native/dwlib.dylib
native-lib/python/src/dataweave/native/dwlib.so
native-lib/python/src/dataweave/native/dwlib.dll
native-lib/python/src/dataweave/native/dwlib.h
155 changes: 129 additions & 26 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,23 +40,75 @@ jobs:
distribution: 'graalvm-community'
github-token: ${{ secrets.GITHUB_TOKEN }}

# Runs a single command using the runners shell
- name: Run Build
run: |
./gradlew --stacktrace --no-problems-report -PskipNodeTests=true build
# Setup Node.js for native-lib Node binding tests and package
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '18'

# Setup Go for Go binding tests
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: '1.21'

# Setup Rust for Rust binding tests
- name: Setup Rust
uses: dtolnay/rust-toolchain@stable

# Setup CMake for C binding tests
- name: Setup CMake
uses: lukka/get-cmake@latest

# All native binding tests (Python, Node.js, Go, Rust, C) are executed
# during the build step via the build -> test task dependency chain.
# Each test task in native-lib/build.gradle uses platform-appropriate
# shell commands (cmd on Windows, bash on Unix) to ensure correct PATH
# resolution for native toolchains.

# Run Build (Windows)
# Note: Uses 'shell: cmd' on Windows to ensure Visual Studio's link.exe
# is found before Git Bash's /usr/bin/link.exe. This prevents Rust linker
# PATH conflicts that cause compilation failures.
- name: Run Build (Windows)
if: runner.os == 'Windows'
run: .\gradlew.bat --stacktrace --no-problems-report build
shell: cmd

- name: Run Build (Unix)
if: runner.os != 'Windows'
run: ./gradlew --stacktrace --no-problems-report build
shell: bash
#Run regression tests
- name: Run regression test 2.9.8
run: |
./gradlew --stacktrace -PweaveTestSuiteVersion=2.9.8 -DweaveSuiteVersion=2.9.8 native-cli-integration-tests:test

# Run regression tests
- name: Run regression test 2.9.8 (Windows)
if: runner.os == 'Windows'
run: .\gradlew.bat --stacktrace -PweaveTestSuiteVersion=2.9.8 -DweaveSuiteVersion=2.9.8 native-cli-integration-tests:test
shell: cmd

- name: Run regression test 2.9.8 (Unix)
if: runner.os != 'Windows'
run: ./gradlew --stacktrace -PweaveTestSuiteVersion=2.9.8 -DweaveSuiteVersion=2.9.8 native-cli-integration-tests:test
shell: bash
- name: Run regression test 2.10
run: |
./gradlew --stacktrace -PweaveTestSuiteVersion=2.10.0 -DweaveSuiteVersion=2.10.0 native-cli-integration-tests:test

- name: Run regression test 2.10 (Windows)
if: runner.os == 'Windows'
run: .\gradlew.bat --stacktrace -PweaveTestSuiteVersion=2.10.0 -DweaveSuiteVersion=2.10.0 native-cli-integration-tests:test
shell: cmd

- name: Run regression test 2.10 (Unix)
if: runner.os != 'Windows'
run: ./gradlew --stacktrace -PweaveTestSuiteVersion=2.10.0 -DweaveSuiteVersion=2.10.0 native-cli-integration-tests:test
shell: bash

# Generate distro
- name: Create Distro
- name: Create Distro (Windows)
if: runner.os == 'Windows'
run: .\gradlew.bat --stacktrace --no-problems-report native-cli:distro
shell: cmd

- name: Create Distro (Unix)
if: runner.os != 'Windows'
run: ./gradlew --stacktrace --no-problems-report native-cli:distro
shell: bash

Expand All @@ -66,24 +118,25 @@ jobs:
shell: bash

# Generate native-lib python wheel
- name: Create Native Lib Python Wheel
- name: Create Native Lib Python Wheel (Windows)
if: runner.os == 'Windows'
run: .\gradlew.bat --stacktrace --no-problems-report native-lib:buildPythonWheel
shell: cmd

- name: Create Native Lib Python Wheel (Unix)
if: runner.os != 'Windows'
run: ./gradlew --stacktrace --no-problems-report native-lib:buildPythonWheel
shell: bash

# Setup Node.js for native-lib Node package
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '18'

# Stage the native lib and build Node package (npm install, node-gyp, tsc, npm pack)
- name: Create Native Lib Node Package
run: ./gradlew --stacktrace --no-problems-report native-lib:buildNodePackage
shell: bash
- name: Create Native Lib Node Package (Windows)
if: runner.os == 'Windows'
run: .\gradlew.bat --stacktrace --no-problems-report native-lib:buildNodePackage
shell: cmd

# Run Node.js tests
- name: Run Node.js Tests
run: ./gradlew --stacktrace --no-problems-report native-lib:nodeTest
- name: Create Native Lib Node Package (Unix)
if: runner.os != 'Windows'
run: ./gradlew --stacktrace --no-problems-report native-lib:buildNodePackage
shell: bash

# Upload the artifact file
Expand All @@ -107,13 +160,63 @@ jobs:
name: dw-node-package-${{env.NATIVE_VERSION}}-${{runner.os}}
path: native-lib/node/dataweave-native-0.0.1.tgz

# Package and upload Go module
- name: Package Go module (Windows)
if: runner.os == 'Windows'
run: .\gradlew.bat --stacktrace --no-problems-report native-lib:packageGo
shell: cmd

- name: Package Go module (Unix)
if: runner.os != 'Windows'
run: ./gradlew --stacktrace --no-problems-report native-lib:packageGo
shell: bash

- name: Upload Go module
uses: actions/upload-artifact@v4
with:
name: dw-go-module-${{env.NATIVE_VERSION}}-${{runner.os}}
path: native-lib/build/packages/dw-go-*.tar.gz

# Package and upload Rust crate
- name: Package Rust crate (Windows)
if: runner.os == 'Windows'
run: .\gradlew.bat --stacktrace --no-problems-report native-lib:packageRust
shell: cmd

- name: Package Rust crate (Unix)
if: runner.os != 'Windows'
run: ./gradlew --stacktrace --no-problems-report native-lib:packageRust
shell: bash

- name: Upload Rust crate
uses: actions/upload-artifact@v4
with:
name: dw-rust-crate-${{env.NATIVE_VERSION}}-${{runner.os}}
path: native-lib/build/packages/*.crate

# Package and upload C library
- name: Package C library (Windows)
if: runner.os == 'Windows'
run: .\gradlew.bat --stacktrace --no-problems-report native-lib:packageC
shell: cmd

- name: Package C library (Unix)
if: runner.os != 'Windows'
run: ./gradlew --stacktrace --no-problems-report native-lib:packageC
shell: bash

- name: Upload C library
uses: actions/upload-artifact@v4
with:
name: dw-c-library-${{env.NATIVE_VERSION}}-${{runner.os}}
path: native-lib/build/packages/libdataweave-*.tar.gz

# Upload the native shared library + header together per OS
- name: Upload native shared library
uses: actions/upload-artifact@v4
with:
name: dwlib-${{env.NATIVE_VERSION}}-${{runner.os}}
path: |
native-lib/python/src/dataweave/native/dwlib.dylib
native-lib/python/src/dataweave/native/dwlib.so
native-lib/python/src/dataweave/native/dwlib.dll
native-lib/python/src/dataweave/native/dwlib.h
65 changes: 64 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,35 @@ jobs:
run: ./gradlew --stacktrace --no-problems-report native-lib:nodeTest -PnativeVersion=${{env.NATIVE_VERSION}}
shell: bash

# Setup Go for Go module packaging
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: '1.21'

# Package Go module
- name: Package Go module
run: ./gradlew --stacktrace --no-problems-report native-lib:packageGo -PnativeVersion=${{env.NATIVE_VERSION}}
shell: bash

# Setup Rust for Rust crate packaging
- name: Setup Rust
uses: dtolnay/rust-toolchain@stable

# Package Rust crate
- name: Package Rust crate
run: ./gradlew --stacktrace --no-problems-report native-lib:packageRust -PnativeVersion=${{env.NATIVE_VERSION}}
shell: bash

# Setup CMake for C library packaging
- name: Setup CMake
uses: lukka/get-cmake@latest

# Package C library
- name: Package C library
run: ./gradlew --stacktrace --no-problems-report native-lib:packageC -PnativeVersion=${{env.NATIVE_VERSION}}
shell: bash

# Upload the artifact file
- name: Upload binaries to release
uses: svenstaro/upload-release-action@v2
Expand All @@ -92,10 +121,11 @@ jobs:
uses: svenstaro/upload-release-action@v2
with:
repo_token: ${{ secrets.GITHUB_TOKEN }}
file: native-lib/python/dist/dataweave_native-0.0.1-py3-none-any.whl
file: native-lib/python/dist/dataweave_native-0.0.1-py3-*.whl
asset_name: dw-python-wheel-${{env.NATIVE_VERSION}}-${{runner.os}}.whl
tag: ${{ github.ref }}
overwrite: true
file_glob: true

# Upload the Node.js package
- name: Upload Node package to release
Expand All @@ -107,6 +137,39 @@ jobs:
tag: ${{ github.ref }}
overwrite: true

# Upload the Go module
- name: Upload Go module to release
uses: svenstaro/upload-release-action@v2
with:
repo_token: ${{ secrets.GITHUB_TOKEN }}
file: native-lib/build/packages/dw-go-*.tar.gz
asset_name: dw-go-module-${{env.NATIVE_VERSION}}.tar.gz
tag: ${{ github.ref }}
overwrite: true
file_glob: true

# Upload the Rust crate
- name: Upload Rust crate to release
uses: svenstaro/upload-release-action@v2
with:
repo_token: ${{ secrets.GITHUB_TOKEN }}
file: native-lib/build/packages/*.crate
asset_name: dw-rust-crate-${{env.NATIVE_VERSION}}-${{runner.os}}.crate
tag: ${{ github.ref }}
overwrite: true
file_glob: true

# Upload the C library
- name: Upload C library to release
uses: svenstaro/upload-release-action@v2
with:
repo_token: ${{ secrets.GITHUB_TOKEN }}
file: native-lib/build/packages/libdataweave-*.tar.gz
asset_name: dw-c-library-${{env.NATIVE_VERSION}}-${{runner.os}}.tar.gz
tag: ${{ github.ref }}
overwrite: true
file_glob: true

# Upload the native shared library
- name: Upload native shared library to release (Linux)
if: runner.os == 'Linux'
Expand Down
Loading
Loading