undefined reference to __divdi3

[bulwahn]

clang fails to properly link the kernel build with tinyconfig:

LD      vmlinux
ld: kernel/events/core.o: in function `perf_adjust_period':
core.c:(.text+0x5bb9): undefined reference to `__divdi3'
ld: mm/filemap.o: in function `filemap_fault':
filemap.c:(.text+0x1d25): undefined reference to `__divdi3'
ld: filemap.c:(.text+0x1f25): undefined reference to `__divdi3'
ld: mm/filemap.o: in function `filemap_map_pages':
filemap.c:(.text+0x213f): undefined reference to `__divdi3'
ld: mm/page-writeback.o: in function `balance_dirty_pages_ratelimited':
page-writeback.c:(.text+0xaf3): undefined reference to `__divdi3'
make: *** [Makefile:1037: vmlinux] Error 1

I tested it with clang-7 and the recent version of clang-8 (clang version 8.0.0-svn343562-1exp1+020181002091440.2051~1.gbp47b1b6 (trunk)).

To get to this point of compiling and linking the kernel (tag: next-20181002 from linux-next), I applied three hacky patches:

hack around to make it compile with clang
compiler.h: give up __compiletime_assert_fallback()
Make clang hate percpu.h less in 32-bit mode 

I provide the three patches in this repository here:

https://github.com/bulwahn/linux-kernel-analysis/tree/clang-patches/clang/next/next-20181002/patches

[tpimh]

What linker do you use? Do you use ld.bfd (GNU binutils) or ld.lld? I remember having same undefined reference with ld.lld, but this problem was fixed.

[nickdesaulniers]

I think __divdi3 should come from libgcc? perf_adjust_period is very clearly performing signed division and using the quotient. I don't understand why those functions would be pulled from glibc.

Curious too that these occur after LD vmlinux. Did you have HOSTCC= set?

Also, it's probably possible to bisect configs down to the one that's causing this failure, if not seen with defconfig.

[bulwahn]

I simply build in a docker container with a debian-sid image and clang-7/8 installed. which ld returns GNU ld (GNU Binutils for Debian) 2.31.1. I am unaware of any special linking options (I do not modify them) and I call it with "make CC=clang-7 HOSTCC=clang-7".

I agree that bisecting the config is a good next step; I will need to see when I will have the time to do that properly.

[bulwahn]

I bisected between allnoconfig (which was good) and tinyconfig (which was bad) and this resulted in config-bisect.pl to report:

Hmm, can't make any more changes without making good == bad?
Difference between good (+) and bad (-)
 CC_OPTIMIZE_FOR_SIZE n -> y
 CC_OPTIMIZE_FOR_PERFORMANCE y -> n

[pirama-arumuga-nainar]

IIUC CC_OPTIMIZE_FOR_PERFORMANCE y -> n builds with -O0 or similar? If so that may cause Clang to use a library call instead of lowering to instructions.

But I'm pretty sure both clang_rt and libgcc provide this function. Do you happen to get the link command that failed? If you add -v, you should be able to see if libgcc or libclang_rt is getting used.

[nickdesaulniers]

Thank you for taking the time to bisect, it is always painful.

Right, so those flags are mutually exclusive and change whether we compile with -O2 (performance) vs -Os (-Oz for Clang)(size).

kernel/configs/tiny.config sets CONFIG_CC_OPTIMIZE_FOR_SIZE=y. I was not able to go from make defconfig, then using make menuconfig to set CONFIG_CC_OPTIMIZE_FOR_SIZE=y then reproduce. So that flag alone is not the problem.

[nathanchance]

Somewhat interesting...

make -j8 CC=clang HOSTCC=clang defconfig tiny.config
make -j8 CC=clang HOSTCC=clang

links just fine for me (I have the same hacky patches that you do for x86). Is that the right command to reproduce this failure? I'm on Arch Linux with the following versions:

$ clang --version
clang version 7.0.0 (tags/RELEASE_700/final)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
$ gcc --version
gcc (GCC) 8.2.1 20180831
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[bulwahn]

With a bit of bisecting and educating guessing, I could boil it down to:

allnoconfig works!
allnoconfig + OPTIMIZE_FOR_SIZE fails!
allnoconfig + OPTIMIZE_FOR_SIZE + "64-bit kernel" works!
allnoconfig + "64-bit kernel" works!

defconfig + OPTIMIZE_FOR_SIZE works!
defconfig + OPTIMIZE_FOR_SIZE - "64-bit kernel" fails! (with the reported issue and even more other linking problems)
defconfig - "64-bit kernel" is strange:

• does not fail with linking problem above
• but fails with:

  LD      vmlinux.o
  MODPOST vmlinux.o
ld: drivers/gpu/drm/i915/i915_params.o: in function `i915_params_dump':
i915_params.c:(.text+0x56): undefined reference to `__compiletime_assert_191'
ld: i915_params.c:(.text+0xed): undefined reference to `__compiletime_assert_191'
ld: i915_params.c:(.text+0x188): undefined reference to `__compiletime_assert_191'
ld: i915_params.c:(.text+0x223): undefined reference to `__compiletime_assert_191'
ld: i915_params.c:(.text+0x2be): undefined reference to `__compiletime_assert_191'
ld: drivers/gpu/drm/i915/i915_params.o:i915_params.c:(.text+0x359): more undefined references to `__compiletime_assert_191' follow
make: *** [Makefile:1037: vmlinux] Error 1

Conclusion:

• OPTIMIZE_FOR_SIZE and no "64-bit kernel" seems to be the issue.
• there is another issue with defconfig and no "64-bit kernel"

[bulwahn]

@nathanchance I can confirm that these two commands link just fine on my machine as well:

make -j8 CC=clang-8 HOSTCC=clang-8 defconfig tiny.config
make -j8 CC=clang-8 HOSTCC=clang-8

[bulwahn]

@nathanchance Just to be clear, these commands show the linking issue:

make -j8 CC=clang-8 HOSTCC=clang-8 tinyconfig
make -j8 CC=clang-8 HOSTCC=clang-8

I guess this fits well into the other observations above.

[nathanchance]

Ah thanks. I didn't try just tinyconfig, today I learned that's a thing :P

Looks like tinyconfig = allnoconfig + tiny.config, which certainly aligns with what you said. I can reproduce the failure here as well, I'll start digging in to see what is up.

[bulwahn]

Well, it is the "64-bit kernel" option between defconfig and allnoconfig that makes the crucial difference.

[nathanchance]

Moving from -Oz to -Os fixes this for me:

diff --git a/Makefile b/Makefile
index 4f08e99c267c..22916c188419 100644
--- a/Makefile
+++ b/Makefile
@@ -663,7 +663,7 @@ KBUILD_CFLAGS       += $(call cc-disable-warning, format-overflow)
 KBUILD_CFLAGS  += $(call cc-disable-warning, int-in-bool-context)
 
 ifdef CONFIG_CC_OPTIMIZE_FOR_SIZE
-KBUILD_CFLAGS  += $(call cc-option,-Oz,-Os)
+KBUILD_CFLAGS  += -Os
 KBUILD_CFLAGS  += $(call cc-disable-warning,maybe-uninitialized,)
 else
 ifdef CONFIG_PROFILE_ALL_BRANCHES

Time to find out why.