diff --git a/docs/source/developers/java/building.rst b/docs/source/developers/java/building.rst index b45afa70a9d0..fd125e178db5 100644 --- a/docs/source/developers/java/building.rst +++ b/docs/source/developers/java/building.rst @@ -47,11 +47,19 @@ repository: $ cd arrow $ git submodule update --init --recursive -Basic Installation ------------------- +These are the options available to compile Arrow Java modules with: +- Maven build tool +- Docker compose +- Archery + +Building Java Modules +--------------------- To build the default modules, go to the project root and execute: +Maven +~~~~~ + .. code-block:: $ cd arrow/java @@ -59,13 +67,136 @@ To build the default modules, go to the project root and execute: $ java --version $ mvn clean install -Building JNI Libraries on Linux -------------------------------- +Docker compose +~~~~~~~~~~~~~~ + +.. code-block:: + + $ cd arrow/java + $ export JAVA_HOME= + $ java --version + $ docker-compose run debian-java + +Archery +~~~~~~~ + +.. code-block:: + + $ cd arrow/java + $ export JAVA_HOME= + $ java --version + $ archery docker run debian-java + +Building JNI Libraries (.dylib / .so) +------------------------------------- First, we need to build the `C++ shared libraries`_ that the JNI bindings will use. We can build these manually or we can use `Archery`_ to build them using a Docker container (This will require installing Docker, Docker Compose, and Archery). +Note: If you are building on Apple Silicon, be sure to use a JDK version that was compiled +for that architecture. See, for example, the `Azul JDK `_. + +Maven +~~~~~ + +- To build only the C Data Interface library: + + .. code-block:: + + $ cd arrow/java + $ export JAVA_HOME= + $ java --version + $ mvn clean generate-resources -Pgenerate-cdata-dylib_so -N + $ ls -latr ../java-dist/lib + |__ libarrow_cdata_jni.dylib + +- To build other JNI libraries: + + Not options available on Maven. + +CMake +~~~~~ + +- To build only the JNI C Data Interface library: + + .. code-block:: + + $ cd arrow + $ mkdir -p java-dist java-jni + $ cmake \ + -S java \ + -B java-jni \ + -DARROW_JAVA_JNI_ENABLE_C=ON \ + -DARROW_JAVA_JNI_ENABLE_DEFAULT=OFF \ + -DBUILD_TESTING=OFF \ + -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_INSTALL_PREFIX=java-dist/lib + $ cmake --build java-jni --target install --config Release + $ ls -latr java-dist/lib + |__ libarrow_cdata_jni.dylib + +- To build JNI ORC & Gandiva libraries: + + .. code-block:: + + $ cd arrow + $ brew bundle --file=cpp/Brewfile + Homebrew Bundle complete! 25 Brewfile dependencies now installed. + $ brew uninstall aws-sdk-cpp + (We can't use aws-sdk-cpp installed by Homebrew because it has + an issue: https://github.com/aws/aws-sdk-cpp/issues/1809 ) + $ export JAVA_HOME= + $ mkdir -p java-dist cpp-jni + $ cmake \ + -S cpp \ + -B cpp-jni \ + -DARROW_CSV=ON \ + -DARROW_DATASET=ON \ + -DARROW_DEPENDENCY_SOURCE=BUNDLED \ + -DARROW_DEPENDENCY_USE_SHARED=OFF \ + -DARROW_FILESYSTEM=ON \ + -DARROW_GANDIVA=ON \ + -DARROW_GANDIVA_JAVA=ON \ + -DARROW_GANDIVA_STATIC_LIBSTDCPP=ON \ + -DARROW_JNI=ON \ + -DARROW_ORC=ON \ + -DARROW_PARQUET=ON \ + -DARROW_PLASMA=ON \ + -DARROW_PLASMA_JAVA_CLIENT=ON \ + -DARROW_S3=ON \ + -DARROW_USE_CCACHE=ON \ + -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_INSTALL_LIBDIR=lib \ + -DCMAKE_INSTALL_PREFIX=java-dist \ + -DCMAKE_UNITY_BUILD=ON + $ cmake --build cpp-jni --target install --config Release + $ ls -latr java-dist/lib + |__ libarrow_orc_jni.dylib + |__ libgandiva_jni.dylib + +- To build only the Dataset library: + + .. code-block:: + + $ cd arrow + $ mkdir -p java-dist java-jni + $ cmake \ + -S java \ + -B java-jni \ + -DARROW_JAVA_JNI_ENABLE_DATASET=ON \ + -DARROW_JAVA_JNI_ENABLE_DEFAULT=OFF \ + -DBUILD_TESTING=OFF \ + -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_INSTALL_PREFIX=java-dist/lib \ + -DCMAKE_PREFIX_PATH=$PWD/java-dist + $ cmake --build java-jni --target install --config Release + $ ls -latr java-dist/lib + |__ libarrow_dataset_jni.dylib + +Archery +~~~~~~~ + .. code-block:: $ cd arrow @@ -77,85 +208,22 @@ We can build these manually or we can use `Archery`_ to build them using a Docke |__ libgandiva_jni.so |__ libplasma_java.so -Building JNI Libraries on MacOS -------------------------------- -Note: If you are building on Apple Silicon, be sure to use a JDK version that was compiled for that architecture. See, for example, the `Azul JDK `_. +Building Java JNI Modules +------------------------- -First, you need to build Apache Arrow C++: +- To compile the JNI bindings, use the ``arrow-c-data`` Maven profile: -.. code-block:: - - $ cd arrow - $ brew bundle --file=cpp/Brewfile - Homebrew Bundle complete! 25 Brewfile dependencies now installed. - $ brew uninstall aws-sdk-cpp - (We can't use aws-sdk-cpp installed by Homebrew because it has - an issue: https://github.com/aws/aws-sdk-cpp/issues/1809 ) - $ export JAVA_HOME= - $ mkdir -p java-dist cpp-jni - $ cmake \ - -S cpp \ - -B cpp-jni \ - -DARROW_CSV=ON \ - -DARROW_DATASET=ON \ - -DARROW_DEPENDENCY_USE_SHARED=OFF \ - -DARROW_FILESYSTEM=ON \ - -DARROW_GANDIVA=ON \ - -DARROW_GANDIVA_JAVA=ON \ - -DARROW_GANDIVA_STATIC_LIBSTDCPP=ON \ - -DARROW_JNI=ON \ - -DARROW_ORC=ON \ - -DARROW_PARQUET=ON \ - -DARROW_PLASMA=ON \ - -DARROW_PLASMA_JAVA_CLIENT=ON \ - -DARROW_S3=ON \ - -DAWSSDK_SOURCE=BUNDLED \ - -DCMAKE_BUILD_TYPE=Release \ - -DCMAKE_INSTALL_LIBDIR=lib \ - -DCMAKE_INSTALL_PREFIX=java-dist \ - -DCMAKE_UNITY_BUILD=ON \ - -Dre2_SOURCE=BUNDLED - $ cmake --build cpp-jni --target install - $ ls -latr ../java-dist/lib - |__ libarrow_orc_jni.dylib - |__ libgandiva_jni.dylib - |__ libplasma_java.dylib - -Then, you can build JNI libraries: - -.. code-block:: - - $ mkdir -p java-jni - $ cmake \ - -S java \ - -B java-jni \ - -DCMAKE_BUILD_TYPE=Release \ - -DCMAKE_INSTALL_PREFIX=java-dist/lib \ - -DCMAKE_PREFIX_PATH=java-dist - $ cmake --build java-jni --target install - $ ls -latr ../java-dist/lib - |__ libarrow_cdata_jni.dylib - |__ libarrow_dataset_jni.dylib - -To build other JNI libraries: + .. code-block:: + $ cd arrow/java + $ mvn -Darrow.c.jni.dist.dir=/java-dist/lib -Parrow-c-data clean install -Building Arrow JNI Modules --------------------------- +- To compile the JNI bindings for ORC / Gandiva / Dataset, use the ``arrow-jni`` Maven profile: -To compile the JNI bindings, use the ``arrow-c-data`` Maven profile: + .. code-block:: -.. code-block:: - - $ cd arrow/java - $ mvn -Darrow.c.jni.dist.dir=/java-dist/lib -Parrow-c-data clean install - -To compile the JNI bindings for ORC / Gandiva / Dataset, use the ``arrow-jni`` Maven profile: - -.. code-block:: - - $ cd arrow/java - $ mvn -Darrow.cpp.build.dir=/java-dist/lib -Parrow-jni clean install + $ cd arrow/java + $ mvn -Darrow.cpp.build.dir=/java-dist/lib -Parrow-jni clean install IDE Configuration ================= diff --git a/java/pom.xml b/java/pom.xml index 0d123977fc63..7bdfa7c65634 100644 --- a/java/pom.xml +++ b/java/pom.xml @@ -905,6 +905,70 @@ + + generate-cdata-dylib_so + + java-dist/lib + + + + + org.codehaus.mojo + exec-maven-plugin + 3.1.0 + + + cdatadir + generate-resources + + exec + + + mkdir + -p java-dist java-jni + ../ + + + + cdatadefine + generate-resources + + exec + + + cmake + + -S java + -B java-jni + -DARROW_JAVA_JNI_ENABLE_C=ON + -DARROW_JAVA_JNI_ENABLE_DEFAULT=OFF + -DBUILD_TESTING=OFF + -DCMAKE_BUILD_TYPE=Release + -DCMAKE_INSTALL_PREFIX=${arrow.c.jni.dist.dir} + + ../ + + + + cdatabuild + generate-resources + + exec + + + cmake + + --build java-jni --target install --config Release + + ../ + + + + + + + +