`swift-java` does not work on Android API 28-30, due to missing JavaVM JNI symbols

[madsodgaard]

Currently, swift-java can crash runtime for Android API 28-30, if you try to use the JavaVirtualMachine class. This means that any JExtract generated sources (JNI mode) only works on API 31+, since we rely on JavaVirtualMachine.shared().environment().

This is due to the fact that Android first added official support for the JavaVM JNI functions, such as JNI_GetCreatedJavaVMs, in API 31+.
We rely on these methods for getting the active JavaVM, through for example JavaVirtualMachine.shared().

We have a shim for these JNI functions in https://github.com/swiftlang/swift-java/blob/main/Sources/CSwiftJavaJNI/AndroidSupport.cpp - this works on API 31+, because those Android versions have the libnativehelper.so. Previous versions did not. We cannot load libart.so or libdvm.so instead, since previous Android versions restricted dynamically loading these "private" libraries.

It seems like the recommended solution to this issue on Android, is to tap into JNI_OnLoad and store the VM from there. This would mean that we would need to update the shared VM at runtime:

swift-java/Sources/SwiftJava/JavaKitVM/JavaVirtualMachine.swift

Line 223 in cc0ff8f

private static let sharedJVM: LockedState<JavaVirtualMachine?> = .init(initialState: nil)

We need to figure out which library to dynamically load, that contains the JNI_OnLoad and can set the shared VM. My concern is since we already statically link the SwiftJava package (the generated sources depend on it), it would not be OK to just also dynamically load it on the Java side, but I am not sure.

An alternative solution could be (if this works in practice):

1. Define a function in SwiftJava with @_silgen_name("_swift_java_setSharedVM"), which takes in a JavaVM and updates the shared one.
2. Call that function in JNI_OnLoad from the already dynamically loaded SwiftRuntimeFunctions target to set the shared VM.

[marcprux]

It's a bit of a horror show, but here's how we do it in Skip:

https://github.com/skiptools/swift-jni/blob/747b4999e9426bafb6fa676f5989b33694fba8ad/Sources/SwiftJNI/SwiftJNI.swift#L1319

TLDR: hunt around some well-known shared libraries and dlsym it.

[marcprux]

As discussed offline, the aforementioned technique might not work for API 28—31 (in between the time they added restrictions on direct loading of some libraries and when they made the JNI_GetCreatedJavaVMs symbol public). This is discussed at android/ndk#1320 (comment).

As to the other solutions:

1. Define a function in SwiftJava with @_silgen_name("_swift_java_setSharedVM"), which takes in a JavaVM and updates the shared one.

This is actually how we do it in Skip (the dynamic loading is just a fallback): we simply require that the app's onCreate call AndroidBridge.initBridge which has knock-on effect of caching the JavaVM. We do a lot of other setup there as well (like configuring libcurl to use system SSL certificates and setting up FileManager paths and stuff), so it isn't any additional burden for us, but it is annoying that it is needed.

2. Call that function in JNI_OnLoad from the already dynamically loaded SwiftRuntimeFunctions target to set the shared VM.

JNI_OnLoad ought to be the ideal solution, but unfortunately it isn't transitively called. So if your app's Java/Kotlin does System.loadLibrary("liba") and that library has a dynamic dependency on SwiftJava.so, SwiftJava.JNI_OnLoad won't automatically be called. I guess you could always require that the app's Java startup code invoke System.loadLibrary("SwiftJava").

This one of the areas I was planning to raise once swift-java-jni is available as a top-level package, since it will be nice to unify the handling of the bootstrapping process across all platforms.

I'd be interested in hearing from others on their experiences, like @colemancda and @andriydruk and @purpln.

[marcprux]

I just did some experimentation, and confirmed that dlsym(RTLD_DEFAULT, "JNI_GetCreatedJavaVMs") works in API 28 (and presumably below, before they started restricting "private" libraries) as well as API 31+ (after they officially added JNI_GetCreatedJavaVMs in libnativehelper.so), but it does not work in either API 29 or API 30.

[andriydruk]

@marcprux we always build all Swift modules into a single dynamic library
That’s why we have only one JNI_OnLoad function, located in the main library that we load directly

[marcprux]

@marcprux we always build all Swift modules into a single dynamic library That’s why we have only one JNI_OnLoad function, located in the main library that we load directly

I imagine we should come up with a recommendation for this. Skip does package multiple separate .so files (for various reasons).

I suppose there's also nothing stopping the startup Main.kt from explicitly invoking System.loadLibrary("SwiftJava") along with whatever other System.localLibrary("MySwiftLibrary") will be needed by the app, in which case JNI_OnLoad could be used to setup the shared JVM.

[ktoso]

Yeah I think going multiple dynamically linked libs is going to be the recommendation here. If some things are statically linked and others not that becomes rather chaotic to manage...

[marcprux]

Yeah I think going multiple dynamically linked libs is going to be the recommendation here.

👍

If some things are statically linked and others not that becomes rather chaotic to manage...

Yeah. We've experienced issues where a diamond dependency with dynamic/static combinations:

graph TD
    A[A] --> B[B<br/>dynamic]
    A --> C[C<br/>dynamic]
    B --> D[D<br/>static]
    C --> D
    
    style B fill:#ffd700
    style C fill:#ffd700
    style D fill:#90ee90

Loading

…results in D being included twice statically by B and C, resulting in duplicated global variables and other attendant craziness.

With Skip, it gets complicated because we'd prefer to not impose dynamic linking on the iOS side while at the same time want to compel everything to be dynamic on the Android side, so we need to take some extra steps to post-process the packages during the Android build, which gets hairy. But short of there being a SwiftPM solution (like products: [.library(name: "MyApp", type: .dynamicTransitive(.when(platforms: [.android])), targets: ["MyApp"])]), I'm not sure what the best solution is.

[ktoso]

This is an interesting thread for SwiftPM by the way @dschaefer2 -- it would be interesting to get SwiftPM into "link everything dynamically" mode perhaps when we're targeting android 🤔