threading/forking

[lunixbochs]

I want to be able to emulate threading and forking inside a single process under Unicorn. I think this will require at minimum a shared memory API.

Perhaps the easiest way would be a function allowing you to source a Unicorn's MMU from another Unicorn instance? New mappings against either instance should affect both instances. It's up to me (the "kernel") to keep things thread-safe. (I'm okay with Unicorn marking the memory mapping functions thread-unsafe and making me lock myself, as I'll already be dealing with that internally)

For fork(), I think I can implement copy-on-write by mapping everything read-only, trapping invalid memory access, and copying/remapping pages when you try to write to a CoW region. For this to be implemented via the Unicorn API, I'd need a way to do a zero-copy MMU clone pointing at the same memory but with everything marked read-only.

[aquynh]

a huge change in the structure of current code is needed to support multiple threads. to save time, i would start by looking at some projects trying to add this feature in Qemu, then see how to adapt them.

[lunixbochs]

Sure, here's a related project using threaded QEMU:

https://github.com/qemu/qemu/blob/master/linux-user/syscall.c#L4559
https://github.com/qemu/qemu/blob/master/linux-user/syscall.c#L4510

I'm not really looking to have Unicorn manage threads whatsoever. I'm fine pushing emu_start() calls into threads myself and doing my own locking when changing the memory map.

[Guy1524]

What do you guys think about a uc_engine_clone function, which preserves all state associated with the existing engine, and is just used as a separate CPU thread.

[lunixbochs]

You manage the CPU threads yourself by which thread you call uc_emu_start() from. The main issues are sharing memory as well as memory synchronization/barriers/locks between unicorn instances.

You should already be able to clone a thread by mapping all of your memory with uc_mem_map_ptr, then on clone, copy the registers and mappings into the new unicorn instance (I think you can use context_save to copy the registers in one shot)

To emulate threading with shared memory, my approach was going to be to just use a single host thread and manually "schedule" the guest threads at intervals like this:

1. uc_emu_stop
2. save registers for running thread
3. restore registers for target thread
4. uc_emu_start(target thread pc)

[Guy1524]

Yeah, implementing a custom scheduler would work, but that is slow ):

I think I'll go the uc_mem_map_ptr route, but I wish the API made this more convenient. Memory mappings change over the execution of the program, so I'll have to make a large wrapper around the Unicorn APIs so that each engine gets updated.

[lunixbochs]

You don't need to constantly update every thread's memory view every time a new mapping is made. Just add a memory fault handler that maps in regions on demand, like so:

https://github.com/lunixbochs/precorn/blob/master/src/precorn.c#L15

When a thread faults, check if the mapping exists in your host page list and if so map it into that thread.

[Guy1524]

In my use-case, the mappings aren't 1:1 host to guest. The host is in user-mode and the guest is in kernel mode with addresses in that space. Plus, not all of mappings have the same protections.

[lunixbochs]

That’s not really relevant for the example. I was just showing you how to do demand paging with unicorn. It’s up to you how the actual backing pages are managed.

[Guy1524]

Ah, so I should I should protect the pages myself using OS functionality instead of the builtin unicorn protections, OK, that works.

[lunixbochs]

No, it's fine to use the unicorn protections. I hoped you'd infer more of this yourself.

1. When something maps memory, allocate it on the host, and save the allocation information (guest addr, host addr, size, protection) somewhere.
2. When a thread segfaults, scan this list (if it's sorted by start addr you can binary search). If you find a matching page, call map_ptr to map it in with the right protections.

Worth noting if you're emulating a complex OS, you'll need to handle weird stuff like partial unmapping. Usercorn does a lot of this already (it has a shadow MMU that knows the entire state of memory and has enough information to do stuff like demand paging).

[Guy1524]

Okay, so I store all the information about the address space myself and correct it when a thread isn't up to date. Unmapping would be another pain point. I'll just do the scheduler thing until perf is a problem. Thanks.