add WASIp2 support via wasi-libc#1931
Conversation
This adds support for WASIp2 networking using the `poll(2)`-based selector and
leaving most of the details to `wasi-libc`.
This includes a new `Waker` implementation since WASIp2 has neither
mulithreading, nor `pipe(2)`, nor `eventfd(2)`. Eventually, `wasi-libc`'s
support for WASIp3 will have both multithreading and `pipe(2)`, so that port
will be simpler.
I've triaged each previously-disabled-on-WASI integration test into one of three
categories:
- Enabled for p2; (still) disabled for p1
- Disabled for both p1 and p2 (e.g. due to use of `thread::spawn`)
- We should be able to enable some of these for p3 once it has multithreading
- Temporarily disabled for p2 due to bugs in `wasi-libc` and/or `wasmtime-wasi`
- These bugs have all been fixed, but have not yet made their way to a stable release
Note that I've added a `TestWASI` job to CI to run the tests using Wasmtime.
I've taken care to ensure that the existing `wasm32-wasip1` support has not
regressed. If desired, I could follow this up with a PR which removes p1
support entirely given its limited utility, which would reduce the amount of
`#[cfg]` clutter and maintenance burden.
5e5aff3 to
5fc5408
Compare
5fc5408 to
1ea9c38
Compare
|
Why not WASIp3?, it should be out next week afaik |
Thomasdezeeuw
left a comment
There was a problem hiding this comment.
Took a quick look, this looks a lot better to me, thanks for this.
Thomasdezeeuw
left a comment
There was a problem hiding this comment.
This is getting quite close. Two open questions left. For the CI maybe we switch to cargo check in this pr and punt the actual testing to #1932?
| notified = waker_events != 0; | ||
| num_fd_events = if notified { num_events - 1 } else { num_events } | ||
| } else { | ||
| waker_events = 0; |
There was a problem hiding this comment.
This might cause an issue when people check the waking event's readiness, as it will report false for everything.
Maybe we should set it to POLLIN if self.notify_waker.woken()? If we go this route we can also simply the code by only setting waker_events differently. notified and num_fd_events can use the old/pipe waker code.
There was a problem hiding this comment.
AFAICT, the only place we use waker_events is here, but I believe that the body of that if statement is unreachable on WASI due to SelectorState::wake being disabled.
Of course, that raises other questions:
- Should
SelectorState::wakebe enabled on WASI? I added the config gate because the compiler said it was unused, but I didn't dig into whether it should be used. - If the answer to the above is "no", then when would
Waker::wokenactually return true on WASI? The only place I seeWaker::wakecalled is in themodify_fdsfunction used byreregister, and in that case it appears to be a no-op sinceWaker::ack_and_resetis called whenWaker::wakereturnsOk(())(which it always will for thesingle_threaded.rsimplementation), so AFAICTWaker::wokenwill never return true inSelectorState::select. In other words, there's a lot of dead code related toWakerinpoll.rsfor WASI right now, and perhaps the least confusing approach would be tocfgit all away (assuming we leaveSelectorState::wakedisabled).
There was a problem hiding this comment.
The most common use case for waking is simply interrupting a call to Poll::poll, where the generated is usually ignored. However, an observable event is generated. So, it could be used for more than just waking where the event is actually used for something.
So to answer your question
Should SelectorState::wake be enabled on WASI?
I don't know. It would be an observable API difference between wasi and Unix/Windows. But I don't know if it will actually matter.
Let's leave it for now and add it #1932 as a potential item.
| notified = waker_events != 0; | ||
| num_fd_events = if notified { num_events - 1 } else { num_events } | ||
| } else { | ||
| waker_events = 0; |
There was a problem hiding this comment.
The most common use case for waking is simply interrupting a call to Poll::poll, where the generated is usually ignored. However, an observable event is generated. So, it could be used for more than just waking where the event is actually used for something.
So to answer your question
Should SelectorState::wake be enabled on WASI?
I don't know. It would be an observable API difference between wasi and Unix/Windows. But I don't know if it will actually matter.
Let's leave it for now and add it #1932 as a potential item.
|
Thanks @dicej! |
This adds networking support for the `wasm32-wasip2` target platform, which includes more extensive support for sockets than `wasm32-wasip1`. The bulk of the changes are in tokio-rs/mio#1931. This patch mainly tweaks a few `cfg` directives to indicate `wasm32-wasip2`'s additional capabilities. Note that this is a draft PR until until tokio-rs/mio#1931 and rust-lang/socket2#639 have been include in stable releases of their respective projects. Also note that I've added a `test-wasi` target to CI and triaged each test which was previously disabled for WASI into one of three categories: - Disabled on both WASIp1 and p2 due to not-yet-supported features such as multithreading - Disabled on p1 but enabled on p2 - Disabled on p1 and _temporarily_ disabled on p2 due to `wasi-libc` bugfixes which have been merged but not yet included in a Rust release. I'll open an issue to re-enable them when the fixes land in Rust. In the future, we could consider adding support for `tokio::net::lookup_host`. WASIp2 natively supports asynchronous DNS lookups and is single threaded, whereas Tokio currently assumes DNS lookups are blocking and require multithreading to emulate async lookups. A WASIp2-specific implementation could do the lookup directly without multithreading. WASIp2 also supports single-threaded, asynchronous file I/O, timers, etc. We could either support those directly or wait for WASIp3's multithreading support, in which case most of `tokio::fs` (as well as `tokio::net::lookup_host`, etc.) _should_ work unchanged via `wasi-libc` and worker threads. Currently, building for WASIp2 requires both `nightly` Rust and `RUSTFLAGS="--cfg tokio_unstable"`. The `nightly` requirement could be removed once either the `feature(wasip2)` is stabilized or `rustix` is updated not to use it. It would be nice to remove the `--cfg tokio_unstable` requirement as well, assuming we have a solid maintainance plan.
This adds networking support for the `wasm32-wasip2` target platform, which includes more extensive support for sockets than `wasm32-wasip1`. The bulk of the changes are in tokio-rs/mio#1931. This patch mainly tweaks a few `cfg` directives to indicate `wasm32-wasip2`'s additional capabilities. Note that this is a draft PR until until tokio-rs/mio#1931 and rust-lang/socket2#639 have been include in stable releases of their respective projects. Also note that I've added a `test-wasi` target to CI and triaged each test which was previously disabled for WASI into one of three categories: - Disabled on both WASIp1 and p2 due to not-yet-supported features such as multithreading - Disabled on p1 but enabled on p2 - Disabled on p1 and _temporarily_ disabled on p2 due to `wasi-libc` bugfixes which have been merged but not yet included in a Rust release. I'll open an issue to re-enable them when the fixes land in Rust. In the future, we could consider adding support for `tokio::net::lookup_host`. WASIp2 natively supports asynchronous DNS lookups and is single threaded, whereas Tokio currently assumes DNS lookups are blocking and require multithreading to emulate async lookups. A WASIp2-specific implementation could do the lookup directly without multithreading. WASIp2 also supports single-threaded, asynchronous file I/O, timers, etc. We could either support those directly or wait for WASIp3's multithreading support, in which case most of `tokio::fs` (as well as `tokio::net::lookup_host`, etc.) _should_ work unchanged via `wasi-libc` and worker threads. Currently, building for WASIp2 requires both `nightly` Rust and `RUSTFLAGS="--cfg tokio_unstable"`. The `nightly` requirement could be removed once either the `feature(wasip2)` is stabilized or `rustix` is updated not to use it. It would be nice to remove the `--cfg tokio_unstable` requirement as well, assuming we have a solid maintainance plan.
This adds networking support for the `wasm32-wasip2` target platform, which includes more extensive support for sockets than `wasm32-wasip1`. The bulk of the changes are in tokio-rs/mio#1931. This patch mainly tweaks a few `cfg` directives to indicate `wasm32-wasip2`'s additional capabilities. Note that this is a draft PR until until tokio-rs/mio#1931 and rust-lang/socket2#639 have been include in stable releases of their respective projects. Also note that I've added a `test-wasi` target to CI and triaged each test which was previously disabled for WASI into one of three categories: - Disabled on both WASIp1 and p2 due to not-yet-supported features such as multithreading - Disabled on p1 but enabled on p2 - Disabled on p1 and _temporarily_ disabled on p2 due to `wasi-libc` bugfixes which have been merged but not yet included in a Rust release. I'll open an issue to re-enable them when the fixes land in Rust. In the future, we could consider adding support for `tokio::net::lookup_host`. WASIp2 natively supports asynchronous DNS lookups and is single threaded, whereas Tokio currently assumes DNS lookups are blocking and require multithreading to emulate async lookups. A WASIp2-specific implementation could do the lookup directly without multithreading. WASIp2 also supports single-threaded, asynchronous file I/O, timers, etc. We could either support those directly or wait for WASIp3's multithreading support, in which case most of `tokio::fs` (as well as `tokio::net::lookup_host`, etc.) _should_ work unchanged via `wasi-libc` and worker threads. Currently, building for WASIp2 requires both `nightly` Rust and `RUSTFLAGS="--cfg tokio_unstable"`. The `nightly` requirement could be removed once either the `feature(wasip2)` is stabilized or `rustix` is updated not to use it. It would be nice to remove the `--cfg tokio_unstable` requirement as well, assuming we have a solid maintainance plan.
This adds networking support for the `wasm32-wasip2` target platform, which includes more extensive support for sockets than `wasm32-wasip1`. The bulk of the changes are in tokio-rs/mio#1931. This patch mainly tweaks a few `cfg` directives to indicate `wasm32-wasip2`'s additional capabilities. Note that this is a draft PR until until tokio-rs/mio#1931 and rust-lang/socket2#639 have been include in stable releases of their respective projects. Also note that I've added a `test-wasi` target to CI and triaged each test which was previously disabled for WASI into one of three categories: - Disabled on both WASIp1 and p2 due to not-yet-supported features such as multithreading - Disabled on p1 but enabled on p2 - Disabled on p1 and _temporarily_ disabled on p2 due to `wasi-libc` bugfixes which have been merged but not yet included in a Rust release. I'll open an issue to re-enable them when the fixes land in Rust. In the future, we could consider adding support for `tokio::net::lookup_host`. WASIp2 natively supports asynchronous DNS lookups and is single threaded, whereas Tokio currently assumes DNS lookups are blocking and require multithreading to emulate async lookups. A WASIp2-specific implementation could do the lookup directly without multithreading. WASIp2 also supports single-threaded, asynchronous file I/O, timers, etc. We could either support those directly or wait for WASIp3's multithreading support, in which case most of `tokio::fs` (as well as `tokio::net::lookup_host`, etc.) _should_ work unchanged via `wasi-libc` and worker threads. Currently, building for WASIp2 requires both `nightly` Rust and `RUSTFLAGS="--cfg tokio_unstable"`. The `nightly` requirement could be removed once either the `feature(wasip2)` is stabilized or `rustix` is updated not to use it. It would be nice to remove the `--cfg tokio_unstable` requirement as well, assuming we have a solid maintainance plan.
This adds networking support for the `wasm32-wasip2` target platform, which includes more extensive support for sockets than `wasm32-wasip1`. The bulk of the changes are in tokio-rs/mio#1931. This patch mainly tweaks a few `cfg` directives to indicate `wasm32-wasip2`'s additional capabilities. Note that this is a draft PR until until tokio-rs/mio#1931 and rust-lang/socket2#639 have been include in stable releases of their respective projects. Also note that I've added a `test-wasi` target to CI and triaged each test which was previously disabled for WASI into one of three categories: - Disabled on both WASIp1 and p2 due to not-yet-supported features such as multithreading - Disabled on p1 but enabled on p2 - Disabled on p1 and _temporarily_ disabled on p2 due to `wasi-libc` bugfixes which have been merged but not yet included in a Rust release. I'll open an issue to re-enable them when the fixes land in Rust. In the future, we could consider adding support for `tokio::net::lookup_host`. WASIp2 natively supports asynchronous DNS lookups and is single threaded, whereas Tokio currently assumes DNS lookups are blocking and require multithreading to emulate async lookups. A WASIp2-specific implementation could do the lookup directly without multithreading. WASIp2 also supports single-threaded, asynchronous file I/O, timers, etc. We could either support those directly or wait for WASIp3's multithreading support, in which case most of `tokio::fs` (as well as `tokio::net::lookup_host`, etc.) _should_ work unchanged via `wasi-libc` and worker threads. Currently, building for WASIp2 requires both `nightly` Rust and `RUSTFLAGS="--cfg tokio_unstable"`. The `nightly` requirement could be removed once either the `feature(wasip2)` is stabilized or `rustix` is updated not to use it. It would be nice to remove the `--cfg tokio_unstable` requirement as well, assuming we have a solid maintainance plan.
This adds networking support for the `wasm32-wasip2` target platform, which includes more extensive support for sockets than `wasm32-wasip1`. The bulk of the changes are in tokio-rs/mio#1931. This patch mainly tweaks a few `cfg` directives to indicate `wasm32-wasip2`'s additional capabilities. Note that this is a draft PR until until tokio-rs/mio#1931 and rust-lang/socket2#639 have been include in stable releases of their respective projects. Also note that I've added a `test-wasi` target to CI and triaged each test which was previously disabled for WASI into one of three categories: - Disabled on both WASIp1 and p2 due to not-yet-supported features such as multithreading - Disabled on p1 but enabled on p2 - Disabled on p1 and _temporarily_ disabled on p2 due to `wasi-libc` bugfixes which have been merged but not yet included in a Rust release. I'll open an issue to re-enable them when the fixes land in Rust. In the future, we could consider adding support for `tokio::net::lookup_host`. WASIp2 natively supports asynchronous DNS lookups and is single threaded, whereas Tokio currently assumes DNS lookups are blocking and require multithreading to emulate async lookups. A WASIp2-specific implementation could do the lookup directly without multithreading. WASIp2 also supports single-threaded, asynchronous file I/O, timers, etc. We could either support those directly or wait for WASIp3's multithreading support, in which case most of `tokio::fs` (as well as `tokio::net::lookup_host`, etc.) _should_ work unchanged via `wasi-libc` and worker threads. Currently, building for WASIp2 requires both `nightly` Rust and `RUSTFLAGS="--cfg tokio_unstable"`. The `nightly` requirement could be removed once either the `feature(wasip2)` is stabilized or `rustix` is updated not to use it. It would be nice to remove the `--cfg tokio_unstable` requirement as well, assuming we have a solid maintainance plan.
This adds networking support for the `wasm32-wasip2` target platform, which includes more extensive support for sockets than `wasm32-wasip1`. The bulk of the changes are in tokio-rs/mio#1931. This patch mainly tweaks a few `cfg` directives to indicate `wasm32-wasip2`'s additional capabilities. Note that this is a draft PR until until tokio-rs/mio#1931 and rust-lang/socket2#639 have been include in stable releases of their respective projects. Also note that I've added a `test-wasi` target to CI and triaged each test which was previously disabled for WASI into one of three categories: - Disabled on both WASIp1 and p2 due to not-yet-supported features such as multithreading - Disabled on p1 but enabled on p2 - Disabled on p1 and _temporarily_ disabled on p2 due to `wasi-libc` bugfixes which have been merged but not yet included in a Rust release. I'll open an issue to re-enable them when the fixes land in Rust. In the future, we could consider adding support for `tokio::net::lookup_host`. WASIp2 natively supports asynchronous DNS lookups and is single threaded, whereas Tokio currently assumes DNS lookups are blocking and require multithreading to emulate async lookups. A WASIp2-specific implementation could do the lookup directly without multithreading. WASIp2 also supports single-threaded, asynchronous file I/O, timers, etc. We could either support those directly or wait for WASIp3's multithreading support, in which case most of `tokio::fs` (as well as `tokio::net::lookup_host`, etc.) _should_ work unchanged via `wasi-libc` and worker threads. Currently, building for WASIp2 requires both `nightly` Rust and `RUSTFLAGS="--cfg tokio_unstable"`. The `nightly` requirement could be removed once either the `feature(wasip2)` is stabilized or `rustix` is updated not to use it. It would be nice to remove the `--cfg tokio_unstable` requirement as well, assuming we have a solid maintainance plan.
This adds networking support for the `wasm32-wasip2` target platform, which includes more extensive support for sockets than `wasm32-wasip1`. The bulk of the changes are in tokio-rs/mio#1931. This patch mainly tweaks a few `cfg` directives to indicate `wasm32-wasip2`'s additional capabilities. Note that this is a draft PR until until tokio-rs/mio#1931 and rust-lang/socket2#639 have been include in stable releases of their respective projects. Also note that I've added a `test-wasi` target to CI and triaged each test which was previously disabled for WASI into one of three categories: - Disabled on both WASIp1 and p2 due to not-yet-supported features such as multithreading - Disabled on p1 but enabled on p2 - Disabled on p1 and _temporarily_ disabled on p2 due to `wasi-libc` bugfixes which have been merged but not yet included in a Rust release. I'll open an issue to re-enable them when the fixes land in Rust. In the future, we could consider adding support for `tokio::net::lookup_host`. WASIp2 natively supports asynchronous DNS lookups and is single threaded, whereas Tokio currently assumes DNS lookups are blocking and require multithreading to emulate async lookups. A WASIp2-specific implementation could do the lookup directly without multithreading. WASIp2 also supports single-threaded, asynchronous file I/O, timers, etc. We could either support those directly or wait for WASIp3's multithreading support, in which case most of `tokio::fs` (as well as `tokio::net::lookup_host`, etc.) _should_ work unchanged via `wasi-libc` and worker threads. Currently, building for WASIp2 requires both `nightly` Rust and `RUSTFLAGS="--cfg tokio_unstable"`. The `nightly` requirement could be removed once either the `feature(wasip2)` is stabilized or `rustix` is updated not to use it. It would be nice to remove the `--cfg tokio_unstable` requirement as well, assuming we have a solid maintainance plan.
Motivation This adds networking support for the `wasm32-wasip2` target platform, which includes more extensive support for sockets than `wasm32-wasip1`. Solution The bulk of the changes are in tokio-rs/mio#1931. This patch mainly tweaks a few `cfg` directives to indicate `wasm32-wasip2`'s additional capabilities. Note that this is a draft PR until until tokio-rs/mio#1931 and rust-lang/socket2#639 have been include in stable releases of their respective projects. Also note that I've added a `test-wasi` target to CI and triaged each test which was previously disabled for WASI into one of three categories: - Disabled on both WASIp1 and p2 due to not-yet-supported features such as multithreading - Disabled on p1 but enabled on p2 - Disabled on p1 and _temporarily_ disabled on p2 due to `wasi-libc` bugfixes which have been merged but not yet included in a Rust release. I'll open an issue to re-enable them when the fixes land in Rust. Future Work In the future, we could consider adding support for `tokio::net::lookup_host`. WASIp2 natively supports asynchronous DNS lookups and is single threaded, whereas Tokio currently assumes DNS lookups are blocking and require multithreading to emulate async lookups. A WASIp2-specific implementation could do the lookup directly without multithreading. WASIp2 also supports single-threaded, asynchronous file I/O, timers, etc. We could either support those directly or wait for WASIp3's multithreading support, in which case most of `tokio::fs` (as well as `tokio::net::lookup_host`, etc.) _should_ work unchanged via `wasi-libc` and worker threads. Currently, building for WASIp2 requires RUSTFLAGS="--cfg tokio_unstable"`. Once we have a solid maintenance plan, we can remove that requirement.
Motivation This adds networking support for the `wasm32-wasip2` target platform, which includes more extensive support for sockets than `wasm32-wasip1`. Solution The bulk of the changes are in tokio-rs/mio#1931. This patch mainly tweaks a few `cfg` directives to indicate `wasm32-wasip2`'s additional capabilities. Note that this is a draft PR until until tokio-rs/mio#1931 and rust-lang/socket2#639 have been include in stable releases of their respective projects. Also note that I've added a `test-wasi` target to CI and triaged each test which was previously disabled for WASI into one of three categories: - Disabled on both WASIp1 and p2 due to not-yet-supported features such as multithreading - Disabled on p1 but enabled on p2 - Disabled on p1 and _temporarily_ disabled on p2 due to `wasi-libc` bugfixes which have been merged but not yet included in a Rust release. I'll open an issue to re-enable them when the fixes land in Rust. Future Work In the future, we could consider adding support for `tokio::net::lookup_host`. WASIp2 natively supports asynchronous DNS lookups and is single threaded, whereas Tokio currently assumes DNS lookups are blocking and require multithreading to emulate async lookups. A WASIp2-specific implementation could do the lookup directly without multithreading. WASIp2 also supports single-threaded, asynchronous file I/O, timers, etc. We could either support those directly or wait for WASIp3's multithreading support, in which case most of `tokio::fs` (as well as `tokio::net::lookup_host`, etc.) _should_ work unchanged via `wasi-libc` and worker threads. Currently, building for WASIp2 requires RUSTFLAGS="--cfg tokio_unstable"`. Once we have a solid maintenance plan, we can remove that requirement.
Motivation This adds networking support for the `wasm32-wasip2` target platform, which includes more extensive support for sockets than `wasm32-wasip1`. Solution The bulk of the changes are in tokio-rs/mio#1931. This patch mainly tweaks a few `cfg` directives to indicate `wasm32-wasip2`'s additional capabilities. Note that this is a draft PR until until tokio-rs/mio#1931 and rust-lang/socket2#639 have been include in stable releases of their respective projects. Also note that I've added a `test-wasi` target to CI and triaged each test which was previously disabled for WASI into one of three categories: - Disabled on both WASIp1 and p2 due to not-yet-supported features such as multithreading - Disabled on p1 but enabled on p2 - Disabled on p1 and _temporarily_ disabled on p2 due to `wasi-libc` bugfixes which have been merged but not yet included in a Rust release. I'll open an issue to re-enable them when the fixes land in Rust. Future Work In the future, we could consider adding support for `tokio::net::lookup_host`. WASIp2 natively supports asynchronous DNS lookups and is single threaded, whereas Tokio currently assumes DNS lookups are blocking and require multithreading to emulate async lookups. A WASIp2-specific implementation could do the lookup directly without multithreading. WASIp2 also supports single-threaded, asynchronous file I/O, timers, etc. We could either support those directly or wait for WASIp3's multithreading support, in which case most of `tokio::fs` (as well as `tokio::net::lookup_host`, etc.) _should_ work unchanged via `wasi-libc` and worker threads. Currently, building for WASIp2 requires RUSTFLAGS="--cfg tokio_unstable"`. Once we have a solid maintenance plan, we can remove that requirement.
Motivation This adds networking support for the `wasm32-wasip2` target platform, which includes more extensive support for sockets than `wasm32-wasip1`. Solution The bulk of the changes are in tokio-rs/mio#1931. This patch mainly tweaks a few `cfg` directives to indicate `wasm32-wasip2`'s additional capabilities. Note that this is a draft PR until until tokio-rs/mio#1931 and rust-lang/socket2#639 have been include in stable releases of their respective projects. Also note that I've added a `test-wasi` target to CI and triaged each test which was previously disabled for WASI into one of three categories: - Disabled on both WASIp1 and p2 due to not-yet-supported features such as multithreading - Disabled on p1 but enabled on p2 - Disabled on p1 and _temporarily_ disabled on p2 due to `wasi-libc` bugfixes which have been merged but not yet included in a Rust release. I'll open an issue to re-enable them when the fixes land in Rust. Future Work In the future, we could consider adding support for `tokio::net::lookup_host`. WASIp2 natively supports asynchronous DNS lookups and is single threaded, whereas Tokio currently assumes DNS lookups are blocking and require multithreading to emulate async lookups. A WASIp2-specific implementation could do the lookup directly without multithreading. WASIp2 also supports single-threaded, asynchronous file I/O, timers, etc. We could either support those directly or wait for WASIp3's multithreading support, in which case most of `tokio::fs` (as well as `tokio::net::lookup_host`, etc.) _should_ work unchanged via `wasi-libc` and worker threads. Currently, building for WASIp2 requires RUSTFLAGS="--cfg tokio_unstable"`. Once we have a solid maintenance plan, we can remove that requirement.
Motivation This adds networking support for the `wasm32-wasip2` target platform, which includes more extensive support for sockets than `wasm32-wasip1`. Solution The bulk of the changes are in tokio-rs/mio#1931. This patch mainly tweaks a few `cfg` directives to indicate `wasm32-wasip2`'s additional capabilities. Note that this is a draft PR until until tokio-rs/mio#1931 and rust-lang/socket2#639 have been include in stable releases of their respective projects. Also note that I've added a `wasm32-wasip2` target to CI and triaged each test which was previously disabled for WASI into one of three categories: - Disabled on both WASIp1 and p2 due to not-yet-supported features such as multithreading - Disabled on p1 but enabled on p2 - Disabled on p1 and _temporarily_ disabled on p2 due to `wasi-libc` bugfixes which have been merged but not yet included in a Rust release. I'll open an issue to re-enable them when the fixes land in Rust. Future Work In the future, we could consider adding support for `tokio::net::lookup_host`. WASIp2 natively supports asynchronous DNS lookups and is single threaded, whereas Tokio currently assumes DNS lookups are blocking and require multithreading to emulate async lookups. A WASIp2-specific implementation could do the lookup directly without multithreading. WASIp2 also supports single-threaded, asynchronous file I/O, timers, etc. We could either support those directly or wait for WASIp3's multithreading support, in which case most of `tokio::fs` (as well as `tokio::net::lookup_host`, etc.) _should_ work unchanged via `wasi-libc` and worker threads. Currently, building for WASIp2 requires RUSTFLAGS="--cfg tokio_unstable"`. Once we have a solid maintenance plan, we can remove that requirement.
Motivation This adds networking support for the `wasm32-wasip2` target platform, which includes more extensive support for sockets than `wasm32-wasip1`. Solution The bulk of the changes are in tokio-rs/mio#1931. This patch mainly tweaks a few `cfg` directives to indicate `wasm32-wasip2`'s additional capabilities. Note that this is a draft PR until until tokio-rs/mio#1931 and rust-lang/socket2#639 have been include in stable releases of their respective projects. Also note that I've added a `wasm32-wasip2` target to CI and triaged each test which was previously disabled for WASI into one of three categories: - Disabled on both WASIp1 and p2 due to not-yet-supported features such as multithreading - Disabled on p1 but enabled on p2 - Disabled on p1 and _temporarily_ disabled on p2 due to `wasi-libc` bugfixes which have been merged but not yet included in a Rust release. I'll open an issue to re-enable them when the fixes land in Rust. Future Work In the future, we could consider adding support for `tokio::net::lookup_host`. WASIp2 natively supports asynchronous DNS lookups and is single threaded, whereas Tokio currently assumes DNS lookups are blocking and require multithreading to emulate async lookups. A WASIp2-specific implementation could do the lookup directly without multithreading. WASIp2 also supports single-threaded, asynchronous file I/O, timers, etc. We could either support those directly or wait for WASIp3's multithreading support, in which case most of `tokio::fs` (as well as `tokio::net::lookup_host`, etc.) _should_ work unchanged via `wasi-libc` and worker threads. Currently, building for WASIp2 requires RUSTFLAGS="--cfg tokio_unstable"`. Once we have a solid maintenance plan, we can remove that requirement.
Motivation This adds networking support for the `wasm32-wasip2` target platform, which includes more extensive support for sockets than `wasm32-wasip1`. Solution The bulk of the changes are in tokio-rs/mio#1931. This patch mainly tweaks a few `cfg` directives to indicate `wasm32-wasip2`'s additional capabilities. Note that this is a draft PR until until tokio-rs/mio#1931 and rust-lang/socket2#639 have been include in stable releases of their respective projects. Also note that I've added a `wasm32-wasip2` target to CI and triaged each test which was previously disabled for WASI into one of three categories: - Disabled on both WASIp1 and p2 due to not-yet-supported features such as multithreading - Disabled on p1 but enabled on p2 - Disabled on p1 and _temporarily_ disabled on p2 due to `wasi-libc` bugfixes which have been merged but not yet included in a Rust release. I'll open an issue to re-enable them when the fixes land in Rust. Future Work In the future, we could consider adding support for `tokio::net::lookup_host`. WASIp2 natively supports asynchronous DNS lookups and is single threaded, whereas Tokio currently assumes DNS lookups are blocking and require multithreading to emulate async lookups. A WASIp2-specific implementation could do the lookup directly without multithreading. WASIp2 also supports single-threaded, asynchronous file I/O, timers, etc. We could either support those directly or wait for WASIp3's multithreading support, in which case most of `tokio::fs` (as well as `tokio::net::lookup_host`, etc.) _should_ work unchanged via `wasi-libc` and worker threads. Currently, building for WASIp2 requires RUSTFLAGS="--cfg tokio_unstable"`. Once we have a solid maintenance plan, we can remove that requirement.
This adds support for WASIp2 networking using the
poll(2)-based selector and leaving most of the details towasi-libc.This includes a new
Wakerimplementation since WASIp2 has neither mulithreading, norpipe(2), noreventfd(2). Eventually,wasi-libc's support for WASIp3 will have both multithreading andpipe(2), so that port will be simpler.I've triaged each previously-disabled-on-WASI integration test into one of three categories:
thread::spawn)wasi-libcand/orwasmtime-wasiNote that I've added a
TestWASIjob to CI to run the tests using Wasmtime.I've taken care to ensure that the existing
wasm32-wasip1support has not regressed. If desired, I could follow this up with a PR which removes p1 support entirely given its limited utility, which would reduce the amount of#[cfg]clutter and maintenance burden.