[BUG] [compiler-rt] CFI unwinder crash on armeabi-v7a

[kasper93]

Description

A CFI unwinder crashes. The minimal way to reproduce is to set fast_unwind_on_malloc=0 option. But it will crash regardless of this option whenever "slow" unwinder is used. But needs bigger project to reproduce. Basically any use of slow unwinder in more complex scenario crashes.

ASAN output

==9588==AddressSanitizer: libc interceptors initialized                                                                                                                                                                                                                           
|| `[0xef940000, 0xffffffff]` || HighMem    ||                                                                                                                                                                                                                                    
|| `[0xed868000, 0xef93ffff]` || HighShadow ||                                                                                                                                                                                                                                    
|| `[0xe9868000, 0xed867fff]` || ShadowGap  ||                                                                                                                                                                                                                                    
|| `[0xcf940000, 0xe9867fff]` || LowShadow  ||                                                                                                                                                                                                                                    
|| `[0x00000000, 0xcf93ffff]` || LowMem     ||                                                                                                                                                                                                                                    
MemToShadow(shadow): 0xe9868000 0xecc4cfff 0xed44d000 0xed867fff                                                                                                                                                                                                                  
redzone=16                                                                                                                                                                                                                                                                        
max_redzone=2048                                                                                                                                                                                                                                                                  
quarantine_size_mb=16M                                                                                                                                                                                                                                                            
thread_local_quarantine_size_kb=64K                                                                                                                                                                                                                                               
malloc_context_size=30                                                                                                                                                                                                                                                            
SHADOW_SCALE: 3                                                                                                                                                                                                                                                                   
SHADOW_GRANULARITY: 8                                                                                                                                                                                                                                                             
SHADOW_OFFSET: 0xcf940000                                                                                                                                                                                                                                                         
==9588==Installed the sigaction for signal 11                                                                                                                                                                                                                                     
==9588==Installed the sigaction for signal 7                                                                                                                                                                                                                                      
==9588==Installed the sigaction for signal 8                                                                                                                                                                                                                                      
==9588==T0: stack [0xff3ca000,0xffbca000) size 0x800000; local=0xffbc7a58                                                                                                                                                                                                         
==9588==AddressSanitizer Init done                                                                                                                                                                                                                                                
AddressSanitizer:DEADLYSIGNAL                                                                                                                                                                                                                                                     
=================================================================                                                                                                                                                                                                                 
==9588==ERROR: AddressSanitizer: SEGV on unknown address 0x0000001f (pc 0xefa4c236 bp 0xffbc68c0 sp 0xffbc6168 T0)                                                                                                                                                                
==9588==The signal is caused by a READ memory access.                                                                                                                                                                                                                             
==9588==Hint: address points to the zero page.                                                                                                                                                                                                                                    
    <empty stack>                                                                                                                                                                                                                                                                 
                                                                                                                                                                                                                                                                                  
AddressSanitizer can not provide additional info.                                                                                                                                                                                                                                 
SUMMARY: AddressSanitizer: SEGV                                                                                                                                                                                                                                                   
==9588==ABORTING                                                                                                                                                                                                                                                                  
Aborted                                                                                                                                                                                                                                                                           

backtrace of the crash from gdb (frames >= 8 are not really important)

#0  0xf7bd4fee in unw_step () from target:/system/lib/libc++.so                                                                                                                                                                                                                   
#1  0xf7bd4df6 in __gnu_unwind_frame () from target:/system/lib/libc++.so                                                                                                                                                                                                         
#2  0xf7b970b2 in __gxx_personality_v0 () from target:/system/lib/libc++.so                                                                                                                                                                                                       
#3  0xf7921904 in __gnu_Unwind_Backtrace (trace=0xf78924f0 <Unwind_Trace()>, trace_argument=0xfffec938, entry_vrs=<optimized out>)                                                                                                                                                
    at /usr/local/google/buildbot/src/android/gcc/toolchain/build/../gcc/gcc-4.9/libgcc/unwind-arm-common.inc:589                                                                                                                                                                 
#4  0xf7922238 in ___Unwind_Backtrace ()                                                                                                                                                                                                                                          
    at /usr/local/google/buildbot/src/android/gcc/toolchain/build/../gcc/gcc-4.9/libgcc/config/arm/libunwind.S:360                                                                                                                                                                
#5  0xf7892658 in SlowUnwindStack ()                                                                                                                                                                                                                                              
    at /usr/local/google/buildbot/src/android/llvm-toolchain/toolchain/compiler-rt/lib/sanitizer_common/sanitizer_unwind_linux_libcdep.cc:127                                                                                                                                     
#6  0xf7910f78 in GetStackTrace ()
    at /usr/local/google/buildbot/src/android/llvm-toolchain/toolchain/compiler-rt/lib/asan/asan_stack.h:47                              
#7  __interceptor_malloc ()                                         
    at /usr/local/google/buildbot/src/android/llvm-toolchain/toolchain/compiler-rt/lib/asan/asan_malloc_linux.cc:146                     
#8  0xf7ae8ab0 in operator new(unsigned int) () from target:/apex/com.android.runtime/lib/bionic/libc.so                                 
#9  0xf7aab1c4 in newlocale () from target:/apex/com.android.runtime/lib/bionic/libc.so                                                  
#10 0xf7bbb586 in std::__1::locale::__imp::__imp(unsigned int) () from target:/system/lib/libc++.so                                      
#11 0xf7bbf68a in std::__1::locale::__global() () from target:/system/lib/libc++.so                                                      
#12 0xf7bbf6e0 in std::__1::locale::locale() () from target:/system/lib/libc++.so                                                        
#13 0xf7ba2e50 in std::__1::basic_streambuf<char, std::__1::char_traits<char> >::basic_streambuf() () from target:/system/lib/libc++.so  
#14 0xf7baa584 in std::__1::ios_base::Init::Init() () from target:/system/lib/libc++.so                                                  
#15 0xf7bab218 in _GLOBAL__sub_I_iostream.cpp () from target:/system/lib/libc++.so                                                       
#16 0xf7f544d2 in __dl__ZL10call_arrayIPFviPPcS1_EEvPKcPT_jbS5_ () from target:/system/bin/linker                                        
#17 0xf7f546d0 in __dl__ZN6soinfo17call_constructorsEv () from target:/system/bin/linker                                                 
#18 0xf7f545ce in __dl__ZN6soinfo17call_constructorsEv () from target:/system/bin/linker                                                 
#19 0xf7f43d68 in __dl__Z9do_dlopenPKciPK17android_dlextinfoPKv () from target:/system/bin/linker                                        
#20 0xf7f400de in __dl___loader_dlopen () from target:/system/bin/linker                                                                 
#21 0xf7826010 in dlopen () from target:/apex/com.android.runtime/lib/bionic/libdl.so                                                    
#22 0xf7a6f34e in netdClientInitImpl() () from target:/apex/com.android.runtime/lib/bionic/libc.so                                       
#23 0xf7af3804 in pthread_once () from target:/apex/com.android.runtime/lib/bionic/libc.so                                               
#24 0xf7a6f2fe in netdClientInit () from target:/apex/com.android.runtime/lib/bionic/libc.so                                             
#25 0xf7f544d2 in __dl__ZL10call_arrayIPFviPPcS1_EEvPKcPT_jbS5_ () from target:/system/bin/linker                                        
#26 0xf7f546d0 in __dl__ZN6soinfo17call_constructorsEv () from target:/system/bin/linker                                                 
#27 0xf7f545ce in __dl__ZN6soinfo17call_constructorsEv () from target:/system/bin/linker                                                 
#28 0xf7f545ce in __dl__ZN6soinfo17call_constructorsEv () from target:/system/bin/linker                                                 
#29 0xf7f5135a in __dl__ZL29__linker_init_post_relocationR19KernelArgumentBlockR6soinfo () from target:/system/bin/linker                
#30 0xf7f50760 in __dl___linker_init () from target:/system/bin/linker                                                                   
#31 0xf7f56450 in __dl__start () from target:/system/bin/linker

Steps to reproduce

1. Create project with the following files. Sorry, I didn't bother to create repository for those four files.

jni/Android.mk

LOCAL_PATH := $(call my-dir)

include $(CLEAR_VARS)

LOCAL_SRC_FILES := main.cpp
LOCAL_MODULE    := unwinder-test
cmd-strip       :=

include $(BUILD_EXECUTABLE)

jni/Application.mk

APP_PLATFORM := android-28
APP_ABI := armeabi-v7a
APP_STL := none
APP_OPTIM := release
COMMON_FLAGS := -fsanitize=address -g -fno-omit-frame-pointer -funwind-tables
APP_CFLAGS := ${COMMON_FLAGS}
APP_LDFLAGS := ${COMMON_FLAGS} -fuse-ld=lld

jni/main.cpp

int main()
{
    return 0;
}

run.sh

#!/bin/sh -e

adb push libs/armeabi-v7a /data/local/tmp/
adb exec-out "cd /data/local/tmp/armeabi-v7a; LD_LIBRARY_PATH=. ASAN_OPTIONS=fast_unwind_on_malloc=0:verbosity=1 ./unwinder-test;"

2. Run

ndk-build && ./run.sh

Environment Details

• NDK Version: r20, but all versions crashes as long as clang is used
• Build system: ndk-build
• Host OS: Doesn't matter
• ABI: armeabi-v7a
• NDK API level: 28
• Device API level: 28

[DanAlbert]

This looks like our typical "you mixed and matched unwinders and shouldn't do that", but the incorrect library is actually the asan runtime, coupled with the problem discussed in #988.

@pirama-arumuga-nainar I think we're missing a -Wl,--exclude-libs,libunwind.a -Wl,--exclude-libs,libgcc.a when building the sanitizers. Looks like we might also be missing it when building the platform's libc++.

[xiaoyur347]

@DanAlbert I have mentioned the unwind problem in cfe-dev before. http://lists.llvm.org/pipermail/cfe-dev/2018-February/056992.html
The problem is mix libgcc_s and libcompiler-rt(libunwind_llvm), and I workaround this problem to avoid use the __gxx_personality_v0 defined in the shared library but the libunwind_llvm one.

[DanAlbert]

@pirama-arumuga-nainar took a look and my first guess doesn't appear to be the problem.

There's a lot weird with that stack trace. For one, it hasn't even made it to the app's code. It also looks a lot like the static asan runtime somehow got used, which shouldn't be happening. I think that's the actual bug here (we shouldn't even be shipping those afaik).

Back to me to look in to this.

[DanAlbert]

(I don't think I'll have time for this in r22, but if someone else wants to pick it up feel free to retriage)

[DanAlbert]

Seems this is fixed in the latest toolchain. Will be in r23.