[Java] Consolidate JNI compilation

[asfimport]

Umbrella ticket for consolidating Java JNI compilation initiative

Seems we have spread the JNI code across the cpp and java directories. As for other bindings (Python) we already discussed it would be great to consolidate and move all cpp code related to PYthon into PyArrow, we should do something equivalent for Java too and move all C++ code specific to Java into the Java project.

At the moment there are two JNI related directories:

• https://github.com/apache/arrow/tree/master/java/c

• https://github.com/apache/arrow/tree/master/cpp/src/jni

  Let's also research what's the best method to build those. The java/c directory seems to be already integrated with the Java build process, let's check if that approach is something we can reuse for the dataset directory too

Reporter: Alessandro Molina / @amol-
Assignee: Larry White / @lwhite1

Subtasks:

• [Java] Consolidate ORC code
• [Java] Consolidate Dataset code
• [Java] C Data Interface library (.so / .dylib) able to compile with mvn command

Related issues:

• [Java] Dataset JNI bridge should use the C data interface (is related to)

_{Note: This issue was originally created as ARROW-15174. Please see the migration documentation for further details.}

[asfimport]

Antoine Pitrou / @pitrou:
There's also some JNI code in https://github.com/apache/arrow/tree/master/cpp/src/plasma/lib/java , but we can probably ignore this one as Plasma is deprecated (and we'd eventually like to drop it from the repo?).

[asfimport]

Antoine Pitrou / @pitrou:
But there's also https://github.com/apache/arrow/tree/master/cpp/src/gandiva/jni which is obviously not deprecated.

[asfimport]

Antoine Pitrou / @pitrou:
The relevant CMake config options on the C++ side seem to be ARROW_JNI (for cpp/src/jni) and ARROW_GANDIVA_JAVA (for cpp/src/gandiva).

[asfimport]

Antoine Pitrou / @pitrou:
I took a look at the dataset and ORC JNI bridges. I don't think it makes sense to consolidate them without also moving them to the C data interface, as that would remove much of the custom code.

[asfimport]

Antoine Pitrou / @pitrou:
Note that moving the dataset JNI bridge to use the C data interface is being handled in ARROW-7272.

[asfimport]

Larry White / @lwhite1:
Note that the is a new JNI interface (to datafusion) that does not use the c-data-interface.  (and, incidentally, also takes a different approach to  packaging the native binaries in Maven than the other JNI interfaces).

See: https://lists.apache.org/thread/6h1mry3n3fb2omf3o24dzbf0n9rhs51h for the archive thread.

Announcement email below: 

Hi dev@arrow,

Recently I've created and published a Java binding1 to datafusion2,
as part of datafusion-contrib projects3. I've updated the README.md4
so people can pick it up via maven[5] or gradle.

Any feedback or contributions are welcome!

java/blob/main/README.md
[5]: https://repo.maven.apache.org/maven2/io/github/datafusion-
contrib/datafusion-java/