Unhandled exceptions cause assert / crash Unix

[janvorli]

On Windows, both software and hardware unhandled exception causes assert in StackFrameIterator::NextInternal:
ASSERT(m_pInstance->FindCodeManagerByAddress(m_ControlPC));
The m_ControlPC points to a code inside the wmain function, which doesn't belong to any of the registered modules.
On Unix, it ends up in an infinite loop in EH.DispatchEx calling RhpSfiNext and getting stuck at Exceptions_BringUpTest__Main for unhandled software exception and it fails with SIGSEGV for any hardware unhandled exception.

[jkotas]

This bug is caused by this TODO in *CodeManager::UnwindStackFrame:

    // @TODO: CORERT: PInvoke transitions
    *ppPreviousTransitionFrame = NULL;

[christianscheuer]

I believe a variant of this bug also occurs on Unix when a P/Invoked function calls back from the native world to a managed function, in which you have a try/catch block spanning the entire method that should handle all exceptions, yet it still gets stuck at the infinite loop in EH.DispatchEx as mentioned above.

Pseudocode:

SomePInvokeFunction(AddrOfIntrinsics.AddrOf(new Action(Callback)));

void Callback()
{
  try {
    throw new Exception("test");
  } catch (Exception e)
  {
    //We don't go here, but to the infinite loop
  }
}

[jkotas]

The remaining issue is Segmentation Fault at:

frame #0: 0x000000010005eb73 src`UnixNativeCodeManager::GetFramePointer(MethodInfo*, REGDISPLAY*) + 19
frame #1: 0x000000010001e39b src`StackFrameIterator::CalculateCurrentMethodState() + 187
frame #2: 0x000000010001ed21 src`RhpSfiNext + 113
frame #3: 0x0000000100065350 src`System_Private_CoreLib_System_Runtime_EH__DispatchEx + 296
frame #4: 0x0000000100062ee2 src`RhThrowEx + 82
frame #5: 0x000000010005f1b7 src`RhpThrowEx + 148

We need unwinder that keeps track of where everything is stored. Statically linking customized version of LLVM's libuwind seems to be the best solution for it.

[christianscheuer]

@jkotas Would there be any workaround for letting exceptions fail more gracefully / not crash the process, until this is resolved?

[jkotas]

The quick fix to stop EH from crashing would be to add UIntNative FP to REGDISPLAY for Unix or MacOS; get/set it in UnwindCursorToRegDisplay/RegDisplayToUnwindCursor to UNW_X86_64_RBP; and return it from GetFP() instead *pRbp. @christianscheuer Would you like to give it a shot?

[christianscheuer]

@jkotas thanks! That would be my pleasure :)