CMake in version 1.38.31+ getting unresolved externals for static linking with libraries

[briangavin]

CMake in version 1.38.31+ getting unresolved externals for static linking. I use CMake to build my emscripten project. In older version 1.38.30 it worked correctly but when I upgraded to 1.38.31 I get unresolved externals for code inside of static libraries I link against. If I replace the files in cmake/Modules with the older version(1.38.30) it works. Looks like a regression bug?

[sbc100]

Looks like the only change to those files in that range was #8362. i.e:

set_property(GLOBAL PROPERTY TARGET_SUPPORTS_SHARED_LIBS FALSE)

The addition of this flag which tells cmake we don't support shared libraries. Which technically we don't (at least not in the normal .so sense).

Can you elaborate more on your error or use case? Or find a repro case? Were you somehow previously relying on shared libraries being reported as supported?

[briangavin]

In my application I try to link libjpeg which is compiled as a static libary and linked with my application.

Here is a simplified version of my application.
libjpeg Cmake file
CMakeLists_LIBJPEG.txt

Pseudo version of Main App that links against libjpeg
CMakeLists_MainAPP.txt

When using CMake and building in version 1.38.31 it warns I should use STATIC instead of SHARED but even with this is changed it still gets an unresolved external. Using STATIC instead of SHARED generates an unresolved external in 1.38.30.

[sbc100]

What are the unresovled symbols? Where should they be defined?

[sbc100]

Can you include the failing command line and the error it produces?

[briangavin]

Here are the command line errors. I am using CMake to generate a ninja build.
error: undefined symbol: compress2
error: undefined symbol: jpeg_CreateCompress
error: undefined symbol: jpeg_destroy_compress
error: undefined symbol: jpeg_finish_compress
error: undefined symbol: jpeg_set_defaults
error: undefined symbol: jpeg_set_quality
error: undefined symbol: jpeg_start_compress
error: undefined symbol: jpeg_std_error
error: undefined symbol: jpeg_stdio_dest
error: undefined symbol: jpeg_write_scanlines

I also get these warnings which is new with 1.38.31+
CMake Warning (dev) at D:/tfs/Main/SRC/Common/3DEngine/Trunk/ThirdParty/jpeg-8d/CMakeLists.txt:80 (add_library):
ADD_LIBRARY called with SHARED option but the target platform does not
support dynamic linking. Building a STATIC library instead. This may lead
to problems.
This warning is for project developers. Use -Wno-dev to suppress it.

[sbc100]

Looks like the binary your building is not being linked against libjpeg or zlib. When building for emscripten you should run cmake -DBUILD_SHARED_LIBS=OFF and ensure that your cmake files honor this setting by building static, and not dynamic libraries.

BTW, I still don't see the failing link command, does in include libjpeg or libz anywhere?

[hpohl]

Just ran into the same thing.

In pre 1.38.31 you could build SHARED libraries that pull in all STATIC library dependencies and there would only be a warning about changing the file name (which disappears if change the suffix):

emcc:WARNING: When Emscripten compiles to a typical native suffix for shared libraries (.so, .dylib, .dll) then it emits an LLVM bitcode file. You should then compile that to an emscripten SIDE_MODULE (using that flag) with suffix .wasm (for wasm) or .js (for asm.js). (You may also want to adapt your build system to emit the more standard suffix for a file with LLVM bitcode, '.bc', which would avoid this warning.)

I guess the library you have been trying to link was a shared one that depends on libjpeg or so.

@sbc100 Is there an alternative to SHARED in 1.38.31? I understand that you cannot build truly shared libraries yet, but keeping the linking functionality around would be useful.

[sbc100]

OK, I guess there was some use case for the fake shared library support. We can consider reverting back to the old behaviour.

Emscripten does not support shared libraries as such. It supports SIDE_MODULE and MAIN_MODULE, but they are quite different to normal shared libraries. The relevant difference here is that you can't pass a SIDE_MODULE on the command line like you can with a shared library.

Since we don't support shared libraries is seemed better not to pretend that we did.

Can I ask you about your use case:

1. Is the library you are building actually used dynamically in your app? (i.e. do you like it as a SIDE_MODULE later?)
2. Is there any reason you cannot switch using static libraries? (Normally done via -DBUILD_SHARED_LIBS=OFF)
3. Is the library in question open source so I can take a look, it would be good for me to see exactly how your use case works.

[hpohl]

1. Not right now, but possibly in the future.
2. If I do, the produced .bc will not include libraries I linked against in CMake. Otherwise there is no reason right now, no.
3. I'm afraid it's not open source, but what we are doing is having protobuf as a third party library (via add_subdirectory) and link against it (libprotobuf).

To fix it temporarily, I added an extra step that extracts all the object files from the main library and all its dependencies and puts them together manually.

[sbc100]

Do you mean that your generted library file doesn't include its library dependencies? That is what i would expect and this is normal for static linking, no? With static linking you need to do -lfoo and -lfoo_depenenency when you your final link. With dynamic linking you only need to do -lfoo.

I guess our previous "fake" dynamic linking was allowing is to get the benefit to only specifying -lfoo because foo contains a copy of all its dependencies too. I'd rather have developers use the normal static linking appraoches until we have real dynamic linking.. but I'm prepared to be swayed on this point if enough people say the previous hacks were useful to them.

[hpohl]

That is correct. I only used it for "bundling". And I guess @briangavin did too.

[briangavin]

I tried adding -DBUILD_SHARED_LIBS=OFF and it did not help attached is a simple test case of the problem.

EMTest.zip

[JDaance]

I have the same problem, what is the proper solution to this? My project is a bunch of .a files normally, but I guess a workaround is used with the .so stuff. Can i specify to emscripten to include all .a files?

[kripken]

@sbc100 may have a better idea, but using -s LINKABLE would force all .a contents to be included, I believe.