fix pointer aliasing in malloc dispatch tables in InitNativeAllocatorDispatch

[rdevshp]

The aliased malloc dispatch table can cause issues with malloc debug.

[thestinger]

Please explain what this is addressing.

[rdevshp]

Inside MallocInitImpl, it first calls InitNativeAllocatorDispatch, which sets both globals->current_dispatch_table and globals->default_dispatch_table to the address of globals->malloc_dispatch_table.

Then it calls InstallHooks when MallocDebug is enabled, which then calls 1. LoadSharedLibrary and then 2. FinishInstallHooks.

LoadSharedLibrary loads the globals->malloc_dispatch_table with the malloc debug functions. Due to the aliasing, globals->current_dispatch_table and globals->default_dispatch_table are also set to the malloc debug functions.

Then in FinishInstallHooks, debug_initialize is called in the following code

  const MallocDispatch* prev_dispatch = GetDefaultDispatchTable();
  if (prev_dispatch == nullptr) {
    prev_dispatch = NativeAllocatorDispatch();
  }

  if (!init_func(prev_dispatch, &gZygoteChild, options)) {
    error_log("%s: failed to enable malloc %s", getprogname(), prefix);
    ClearGlobalFunctions();
    return false;
  }

This means that debug_initialize is not going to be able to properly save prev_dispatch due to the aliasing as it is already overridden by LoadSharedLibrary.

[thestinger]

I don't understand why you're using atomic_store.

[rdevshp]

I have removed atomic_store.

[rdevshp]

Basically globals->malloc_dispatch_table is the dispatch table to store allocator hooks, not for storing the underlying allocator implementations.