New API doesn't let you access memory while async function call is in progress

[tomaka]

👋
I have an existing code that is using wasmtime 0.27, and that is doing more or less something like this (pseudo-code):

let my_func = instance.get_export("foo").unwrap().into_func().unwrap();
let my_memory = instance.get_export("memory").unwrap().into_memory().unwrap();

let future = my_func.call_async(&[]);
futures::pin_mut!(future);

loop {
    // Execute a bit of Wasm.
    (&mut future).now_or_never();

    // Read the memory of the Wasm VM.
    let data = my_memory.read(...);
    if some_cond(&data) {
        break;
    }
}

The actual code is a bit more complicated than that (I'm also sharing a struct between the host functions and the caller), but I'm basically using call_async in order to start execution, manually polling the future returned by call_async, and while the execution is paused I can read/write the memory of the Wasm VM.

I've been trying to upgrade to wasmtime 0.28, but the new API is giving me a lot of trouble.
Calling call_async now requires passing an impl AsMutContext that is being held for the entire duration of the call, meaning that we lose the ability to read/write memory from outside the host functions while the asynchronous call is in progress.

I was wondering if you could give me suggestions on how to upgrade 🙏

What I've tried

My little snippet maybe implies that the Wasm memory is used as a vector of communication between the host functions and the "outside", and that this could be refactored. But in practice the fact that I can read/write the memory of the Wasm function while execution is paused is something deeply ingrained in the API of my code (as this seems like a sensible thing to do), and tens of thousands of lines of code depend on the ability to do that.

I've been trying to think of a way to pass to call_async some sort of glorified Arc<Mutex<Store>> that would implement AsMutContext, but I believe that the way the trait is defined makes it impossible to do in a safe way.

There is a way to implement this safely:
The Caller<_> that is accessible by the host functions can be used to read and write memory, so in principle one could imagine a system where an Rc<RefCell<Vec<Operation>>> is shared between the "outside" and the host functions, where Operation is some sort of enum { ReadMemory(_), WriteMemory(_) } and host functions process these operations after each await.
However this seems like a lot of complication.

One potential change in wasmtime that could make sense to me would be to implement AsContext and AsContextMut on the future returned by call_async. In practice, however, this seems almost impossible to implement.

Thanks in advance!

[alexcrichton]

Thanks for the report, and this is a good question! I think you cover the problem pretty well here and have a pretty good grasp on what the limitations are. I agree that there's no great way (even with workarounds) to do this today. I personally would think that the best fix would be to implement AsContext and AsContextMut for the returned futures. That would indeed be significantly difficult for us on an implementation side but that's not necessarily a deterrent per-se.

I think that the only workaround possible for now is the Rc<...>-based solution where the Store contains a channel through which communication can be done with the original call-site. The future closes over the StoreContext<'_, T> which means you can't get access to T while the future is running, so that sort of side-channel is the only method.

[alexcrichton]

I've posted some initial work for this at #2999, but I think that manually implementing futures for Instance::new_async and friends is not really feasible to do. That means that only call_async gives access to the underlying context for now, so I'm not sure if it's the best to merge.