Extend standalone support by jerbob92 · Pull Request #18285 · emscripten-core/emscripten

jerbob92 · 2022-12-01T12:21:14Z

Implement _emscripten_throw_longjmp that aborts the program
Add nice messages to __cxa_throw and __cxa_allocate_exception
Implement getentropy using WASI
Implement WASI FS for syscalls lstat64, stat64, newfstatat, getdents64, fstat64 and openat

This is far from complete if you look at the WASI FS spec and the syscalls, but at least it gets basic operations like directory listing, file/directory opening and file/directory stat working. It was enough for me to get pdfium working in standalone mode.

I didn't really know what I should do with AT_FDCWD or what the runtime should do with it when passing AT_FDCWD. AT_FDCWD is also negative while I think fd in WASI is unsigned?

Most of the syscall code was taken from or based on wasi-libc.

sbc100

Thanks for working on this!

I left a few nits. Perhaps we can maybe split this up.

sbc100 · 2022-12-01T17:38:51Z

Can we add some some new tests based on these?

jerbob92 · 2022-12-05T09:55:13Z

Can we add some some new tests based on these?

I have never developed in C before, I don't really know where to start, can you give me some hints where you want these tests added and what they should test?

See #18285

sbc100 · 2022-12-06T16:35:40Z


+__attribute__((__weak__))
+void _emscripten_throw_longjmp() {
+  REPORT_UNSUPPORTED(do an invalid longjmp);


Just call longjmp would be more accurate I think.

I think I could only trigger this by doing an invalid jump, but I think you would know better.

No.. all longjmp's require callout out of JS IIUC. WebAssembly, as it stands, has no support for longjmp so we call out the JS VM to throw for us.

Thanks! Not that it matters but do you have any C to demonstrate this? Just curious.
In my tests I was only able to trigger this by doing an invalid jmp. I think the jumps were being handled by the invoke_* methods?

Any call to longjmp from C code will called _emscripten_throw_longjmp IIUC.

So here I would just do REPORT_UNSUPPORTED(longjmp)

See #18285

sbc100 · 2022-12-11T15:39:52Z

+ *   - https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_14
+ *   - https://github.com/WebAssembly/wasi-libc/blob/wasi-sdk-16/libc-bottom-half/sources/preopens.c#L215
+ */
+#define __WASI_FD_ROOT 3


I believe the way it works is that there can be any number of pre-opens, starting at 3: https://github.com/WebAssembly/wasi-libc/blob/30094b6ed05f19cee102115215863d185f2db4f0/libc-bottom-half/sources/preopens.c#L212-L215. And non of these pre-opens are necessarily the root, they can be mounted in various places.

Also, can you move this macros (if you keep it), into standalone.c. api.h is copied from wasi-libc, and I think its better to avoid have local modifications to it.

sbc100

I think we really needs some tests for this.

The most basic test for the standalone syscalls is in test_standalone_syscalls in test_other.py.

For other tests look for @also_with_standalone_wasm in test_core.py

codefromthecrypt · 2022-12-14T01:37:03Z

on behalf of wasm people I know I would like to thank @jerbob92 and @sbc100 for progressing this, as it not only unlocks things like pdfium, but also generic wasm utilities that use emscripten (like wabt). Ex being able to run wat2wasm using any wasi runtime instead of whatever was released and packaged.

If any one not currently involved has skills to contribute, please do as this is very likely well into "hobby time" for @jerbob92 and it is at the core a pretty substantial infrastructure change and a lot of work to test it. other wasm people will thank you, but I'll thank in advance!

sbc100 · 2022-12-14T02:02:05Z

on behalf of wasm people I know I would like to thank @jerbob92 and @sbc100 for progressing this, as it not only unlocks things like pdfium, but also generic wasm utilities that use emscripten (like wabt). Ex being able to run wat2wasm using any wasi runtime instead of whatever was released and packaged.

If you just wast a wasi version of the wabt tools I think the simplest path would be to use wasi-sdk to build it. Of course that doesn't mean we shouldn't land this change too.

codefromthecrypt · 2022-12-14T02:23:26Z

@sbc100 ps your last comment didn't format right. might want to polish it

codefromthecrypt · 2022-12-14T02:29:07Z

If you just wast a wasi version of the wabt tools I think the simplest path would be to use wasi-sdk to build it. Of course that doesn't mean we shouldn't land this change too.

good idea, on the wabt thing and indeed a separate topic if they are open to decoupling from emscripten.

sbc100 · 2022-12-14T02:35:40Z

If you just wast a wasi version of the wabt tools I think the simplest path would be to use wasi-sdk to build it. Of course that doesn't mean we shouldn't land this change too.

good idea, on the wabt thing and indeed a separate topic if they are open to decoupling from emscripten.

IIUC wabt is not coupled to emscripten, we just happen to build it with emscripten for the web demo.

sbc100 · 2022-12-14T02:36:37Z

(I guess maybe what you are saying is that it would be nice if the web version of wabt, which is run by emscripten, happened to also be WASI compliant.. in that case I agree that would be very cool).

codefromthecrypt · 2022-12-14T02:46:36Z

yeah sorry it was that we were looking for a wabt compiled to wasm and found the one that was compiled with emscripten for the web. I raised this issue for out of browser wasi binary WebAssembly/wabt#2101

jerbob92 · 2022-12-14T08:15:49Z

Thanks @codefromthecrypt! So in hindsight, I don't think my C skills are good enough to get this PR into a merge-able state, implementing AT_FDCWD, using the preload dirs and not the FD 3, implementing tests.

codefromthecrypt · 2022-12-14T23:32:16Z

@jerbob92 no worries, I think you got things very far. I'll keep recruiting to whatever end on this.

jerbob92 · 2023-04-14T10:26:14Z

@sbc100 I have done some more progress on this PR now:

Implemented WASI pre-opens
Implement mkdirat

I do have some questions though:

How to get started on adding tests for this?
The standalone.c file is getting very big right now, how do we want to split this up? Perhaps something like standalone_wasi_preopens.c, standalone_wasi_fs.c, standalone_wasi_random.c go keep things clean?
Is there an easy way to check which syscalls still need to be implemented in standalone to make it full WASI compliant?
Code is mostly copied from wasi-libc right now with some changes, is this something that we want?
wasi-libc has a whole system to maintain a CWD, is this also something that we want to implement?

sbc100 · 2023-04-14T22:58:00Z

@sbc100 I have done some more progress on this PR now:

Implemented WASI pre-opens

Implement mkdirat

I do have some questions though:

How to get started on adding tests for this?

The standalone.c file is getting very big right now, how do we want to split this up? Perhaps something like standalone_wasi_preopens.c, standalone_wasi_fs.c, standalone_wasi_random.c go keep things clean?

I don't feel strongly about this. Perhaps use the same split that wasi-libc uses? Also I don't think we need standalone in each filename since they will all be in the standalone directory.

Is there an easy way to check which syscalls still need to be implemented in standalone to make it full WASI compliant?

The ideas is that we will be running the full wasi testsuite. I started adding some of it in #12704 and I have some
more plans.

Code is mostly copied from wasi-libc right now with some changes, is this something that we want?

I don't think its a problem, but please document the origin and try to document any changes from upstream.

wasi-libc has a whole system to maintain a CWD, is this also something that we want to implement?

Sure. We could even consider adding wasi-libc as a submodule and including certain files directly?

jerbob92 · 2023-04-18T08:42:34Z

I don't feel strongly about this. Perhaps use the same split that wasi-libc uses? Also I don't think we need standalone in each filename since they will all be in the standalone directory.

Sounds good!

The ideas is that we will be running the full wasi testsuite. I started adding some of it in #12704 and I have some
more plans.

Nice, that will make it a lot easier!

I don't think its a problem, but please document the origin and try to document any changes from upstream.
Sure. We could even consider adding wasi-libc as a submodule and including certain files directly?

If that's a possibility that would be great, it would make it a lot easier because right now I'm just cherry-picking wasi-libc code to get the syscalls/WASI calls working that I require, especially if we want to pass the whole wasi-testsuite then we have to copy a lot from was-libc. Only thing I noticed while copying code is that their WASI function signature is a bit different sometimes, which prevents us from directly using the C code (AFAIK), so we might want to look into making that possible. For example:
wasi-libc __wasi_path_filestat_get = __wasi_errno_t __wasi_path_filestat_get(__wasi_fd_t fd, __wasi_lookupflags_t flags, const char *path, __wasi_filestat_t *retptr0)
Emscripten __wasi_path_filestat_get = __wasi_errno_t __wasi_path_filestat_get(__wasi_fd_t fd, __wasi_lookupflags_t flags, const char *path, size_t path_len, __wasi_filestat_t *buf)

So Emscripten requires the path length and wasi-libc doesn't. But perhaps we can work around this by completely using wasi-libc for the wasi part.

sbc100 · 2023-04-19T14:46:40Z

+  return 0;
+}
+
+int rmdirat(int dirfd, intptr_t path) {


Can you make this static and make it clear this is an internal helper?

karelbilek · 2023-04-25T07:54:58Z

A stupid nit - if it was based on wasi libc, shouldn't there be copyright attribution somewhere? (Or is that not necessary?)

sbc100 · 2023-04-25T18:34:19Z

A stupid nit - if it was based on wasi libc, shouldn't there be copyright attribution somewhere? (Or is that not necessary?)

No stupid at all, we should certainly consider how to handle this. I'm not sure why the right answer is. Or how we should deal with keeping the codebases in sync. One easy option wold be to add wasi-libc as a submodule instead of duplicating the code here.

kripken · 2024-12-02T20:37:10Z

@sbc100 What do you think about moving forward with this PR? It should at least help in some cases. E.g. one of the syscalls missing in #23008 is implemented here (and perhaps more, I'm not sure by the names).

sbc100 · 2024-12-02T20:50:44Z

Sure, I'd be OK with moving forward with this. I think it still needs a bit of work though.

turbolent · 2025-03-15T19:47:40Z

Would it be possible to create a TODO list for the work that still need to be done to get this PR merged? I'd be happy to help, but not quite sure where to start. Thanks!

karelbilek · 2025-04-23T20:08:15Z

I want to say I am using this branch in building ghostscript (both the ancient GPL version and the current AfferoGPL version) and Xpdf comand line tools (latest version) and it all works as expected in wazero (go WASI runtime).

…

On Sat 15. 3. 2025 at 20:48, Bastian Müller ***@***.***> wrote: Would it be possible to create a TODO list for the work that still need to be done to get this PR merged? I'd be happy to help, but not quite sure where to start. Thanks! — Reply to this email directly, view it on GitHub <#18285 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAZT4NUVKP4NLMZPWFB5ST2UR7XFAVCNFSM6AAAAAASQXFBISVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMRWHE3DIMBXGU> . You are receiving this because you were mentioned.Message ID: ***@***.***> [image: turbolent]*turbolent* left a comment (emscripten-core/emscripten#18285) <#18285 (comment)> Would it be possible to create a TODO list for the work that still need to be done to get this PR merged? I'd be happy to help, but not quite sure where to start. Thanks! — Reply to this email directly, view it on GitHub <#18285 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAZT4NUVKP4NLMZPWFB5ST2UR7XFAVCNFSM6AAAAAASQXFBISVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMRWHE3DIMBXGU> . You are receiving this because you were mentioned.Message ID: ***@***.***>

kripken · 2025-04-24T15:42:09Z

Concretely I think what is needed here is

Add testing to cover the new functionality
Get CI fully passing

Anything else @sbc100 ?

jiixyj · 2025-04-30T05:41:02Z

+    case O_RDONLY:
+    case O_RDWR:
+    case O_WRONLY:
+      if ((flags & O_RDONLY) != 0) {


Thanks for your work on this so far!

Musl defines O_RDONLY as 0, so this line will not work I think. This will only work in implementations where the file access modes are single bits, one for each file access mode (like wasi-libc, where this is the case).

The robust, POSIX compliant way to deal with file access modes is to:

first mask the file access modes part off with flags & O_ACCMODE

then compare the resulting value directly to the five defined access modes (O_RDONLY, O_WRONLY, O_RDWR, O_SEARCH, O_EXEC)

With this, it should be possible to open a file as O_RDONLY.

Oh, and if you test with Wasmer: Right now, it doesn't support opening a file with O_RDONLY (wasmerio/wasmer#4892). This took me quite a while to figure out.

This code comes from wasi-libc, which also uses musl, so then it would be broken there as well?

I don't know exactly how much they use from musl, but they defined the file access mode bits themselves at least:

https://github.com/WebAssembly/wasi-libc/blob/e9524a0980b9bb6bb92e87a41ed1055bdda5bb86/libc-bottom-half/headers/public/__header_fcntl.h#L18-L22

Hmm, yeah they defined it both at the top and bottom half (with different values). I think we will known soon enough if this works or not if we add tests.

I'm mostly just copy pasting stuff from wasi-libc here, so I wouldn't know if it's correct, it works for me but I'm not sure if I hit this code path.

jiixyj · 2025-05-01T06:02:08Z

+
+    // If we can't find a preopen for it, fail as if we can't find the path.
+    if (dirfd == -1) {
+      errno = ENOENT;


I think this should be return -ENOENT. And similar elsewhere. In wasi-libc, this code is used in function that use the "return -1 + errno" error convention, while the __syscall_* functions use the "return negative error" convention of the Linux syscall API.

jiixyj · 2025-05-01T12:17:57Z

@jerbob92 : Are you or is anyone else working on the test integration at the moment? Based on this PR I managed to get the ./test/fcntl/test_fcntl_open.c test working and hooked up using the @also_with_standalone_wasm test decorator.

I used this test because it is pretty simple. It just needed open, stat, mkdir and symlink.

Should I open a new PR?

jerbob92 · 2025-05-01T12:22:22Z

@jiixyj Not that I know of! New MR is fine, you can also update this one if you have access.

jiixyj · 2025-05-02T12:09:50Z

Based on code from this PR and wasm-libc, I've opened PRs #24246 and #24247 if anyone wants to have a look.

I have mostly concentrated on the filesystem stuff, so not all syscalls from this PR are included. But it shouldn't be too hard to add them in follow up PRs.

realoriginal · 2026-06-11T01:59:07Z

What is the status of this? Anyway I could help get this merged?

kripken · 2026-06-16T19:35:56Z

I think the overall status is that this would be good to land, and the next step is testing (test the new functionality, and get all the existing tests passing).

jerbob92 changed the title ~~Implement _emscripten_throw_longjmp that aborts the program, add nice…~~ Extend standalone support Dec 1, 2022

sbc100 reviewed Dec 1, 2022

View reviewed changes

Comment thread system/lib/standalone/standalone.c Outdated

Comment thread system/lib/standalone/standalone.c Outdated

Comment thread system/lib/standalone/standalone.c Outdated

Comment thread system/lib/standalone/standalone.c Outdated

Comment thread system/lib/standalone/standalone.c

sbc100 added a commit that referenced this pull request Dec 5, 2022

Fix signature mismatch in standalone.c. NFC

7fe18a1

See #18285

sbc100 mentioned this pull request Dec 5, 2022

Fix signature mismatch in standalone.c. NFC #18323

Merged

sbc100 added a commit that referenced this pull request Dec 5, 2022

Fix signature mismatch in standalone.c. NFC

67261e8

See #18285

sbc100 reviewed Dec 6, 2022

View reviewed changes

Comment thread system/lib/standalone/standalone.c

sbc100 reviewed Dec 6, 2022

View reviewed changes

Comment thread system/lib/standalone/standalone.c Outdated

sbc100 added a commit that referenced this pull request Dec 6, 2022

Fix signature mismatch in standalone.c. NFC (#18323)

9210d96

See #18285

jerbob92 requested a review from sbc100 December 11, 2022 12:11

sbc100 reviewed Dec 11, 2022

View reviewed changes

jerbob92 force-pushed the implement-more-syscalls-for-standalone branch from 55abfc0 to 9fbd75d Compare April 14, 2023 10:20

sbc100 reviewed Apr 19, 2023

View reviewed changes

Add missing functions in WASI

e4f5189

kripken mentioned this pull request Aug 22, 2024

Error While creating hello world with cout in c++ #22401

Open

turbolent mentioned this pull request Oct 22, 2024

Remove WASM2C option #19380

Merged

Merge in 3.1.72

5fcbf96

kripken mentioned this pull request Dec 2, 2024

STANDALONE_WASM still imports some simple filesystem operations from "env" #23008

Open

Merge tag '4.0.3' into implement-more-syscalls-for-standalone

6272353

jerbob92 added 3 commits April 23, 2025 15:58

Merge tag '4.0.7' into implement-more-syscalls-for-standalone

21737f7

Add time related libc

08f2aaf

Update time implementations for patch

8fd95dd

karelbilek mentioned this pull request Apr 27, 2025

Cannot use zig cc with --target=wasm32-wasi and -D_WASI_EMULATED_SIGNAL ziglang/zig#23709

Closed

jiixyj reviewed Apr 30, 2025

View reviewed changes

jiixyj reviewed May 1, 2025

View reviewed changes

This was referenced May 2, 2025

Implement open/stat/mkdir/symlink for standalone WASI mode #24246

Open

Add CWD handling to standalone WASI mode, and some related syscalls #24247

Draft

jerbob92 mentioned this pull request Jul 25, 2025

DO NOT MERGE: WebAssembly standalone support bblanchon/pdfium-binaries#103

Open

ajalt mentioned this pull request Aug 7, 2025

Add bindings to support Wasm WASI. ajrcarey/pdfium-render#159

Closed

jerbob92 added 4 commits November 25, 2025 11:40

Merge in 4.0.20

cf47e28

Remove idea

2d359d3

Merge in 5.0.5

d6b2f2c

Merge in 6.0.0

a2b152d

Uh oh!

Conversation

jerbob92 commented Dec 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sbc100 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sbc100 commented Dec 1, 2022

Uh oh!

jerbob92 commented Dec 5, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sbc100 left a comment

Choose a reason for hiding this comment

Uh oh!

codefromthecrypt commented Dec 14, 2022

Uh oh!

sbc100 commented Dec 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codefromthecrypt commented Dec 14, 2022

Uh oh!

codefromthecrypt commented Dec 14, 2022

Uh oh!

sbc100 commented Dec 14, 2022

Uh oh!

sbc100 commented Dec 14, 2022

Uh oh!

codefromthecrypt commented Dec 14, 2022

Uh oh!

jerbob92 commented Dec 14, 2022

Uh oh!

codefromthecrypt commented Dec 14, 2022

Uh oh!

jerbob92 commented Apr 14, 2023

Uh oh!

sbc100 commented Apr 14, 2023

Uh oh!

jerbob92 commented Apr 18, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

karelbilek commented Apr 25, 2023

Uh oh!

sbc100 commented Apr 25, 2023

Uh oh!

kripken commented Dec 2, 2024

Uh oh!

sbc100 commented Dec 2, 2024

Uh oh!

turbolent commented Mar 15, 2025

Uh oh!

karelbilek commented Apr 23, 2025 via email

Uh oh!

kripken commented Apr 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jerbob92 commented Dec 1, 2022 •

edited

Loading

sbc100 commented Dec 14, 2022 •

edited

Loading

kripken commented Apr 24, 2025 •

edited

Loading