library_webgl.js not proxied to main thread

[turran]

Version of emscripten/emsdk:
emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.42 (6ede0b8)
clang version 17.0.0 (https://github.com/llvm/llvm-project f3b64887de61020c09404bfee97b2fadd30df10a)
Target: wasm32-unknown-emscripten
Thread model: posix

Functions in library_egl.js do have the attribute __proxy: 'sync' on all methods. I had to add the OFFSCREEN_FRAMEBUFFER=1 to make the GLctx variable have a value. But after fixing the initial issue, I realized that the methods in library_webgl.js needed the same logic, to have the suffix __proxy: 'sync' in order to work and have a valid GLctx too. After manually changing all symbols, the application I'm building found the GLctx variables correctly.

Is there a reason not to follow the same approach as other GL/EGL symbols?

Still, my application needs to access other GL/EGL symbols like glClear, which are proxied directly to the C function and no js definition is found, so I can not do the __proxy: 'sync' trick. Can you advise how to proceed?

The application I'm building, in theory, proxies all calls to GL/EGL to a single thread, the same as what OFFSCREEN_FRAMEBUFFER=1 is doing so I guess I wouldn't need it, but still, I had to force it. Maybe is a problem on my side on how the multiple threads detect if they are on the GL thread or not, but in the meantime I was wondering in the above questions. Thanks for your help.

[kripken]

I think system/lib/gl/webgl1.c should handle proxying, but I don't remember the details. cc @juj

[juj]

The EGL library never got an upgrade to work properly with multithreading. Some years ago I did start a port of EGL to be multithreaded by rewriting it into C, but that PR branch was closed without merging, since I never had time to make it complete (and there were no users at the time who would have leveraged it). I recall I tentatively added all EGL symbols to be __proxy: 'sync' so they would work for minimal builds at least.

The OFFSCREEN_CANVAS and OFFSCREEN_FRAMEBUFFER methods require using the WebGL HTML5 context creation API to work, those features have also never been developed to work with EGL.

But after fixing the initial issue, I realized that the methods in library_webgl.js needed the same logic, to have the suffix __proxy: 'sync' in order to work

The utilized proxying mode depends on which WebGL multithreading model is used. If OFFSCREEN_CANVAS is used, then WebGL functions are not proxied at all, but are processed in the worker against an OffscreenCanvas local in that worker. When OFFSCREEN_FRAMEBUFFER is used, these functions are proxied via the custom proxying mechanism that is implemented in webgl1.c. Neither of these methods use the JS library side __proxy: 'x' machinery, so that is why those directives are not present there. (this enables users to do a hybrid OFFSCREEN_CANVAS+ OFFSCREEN_FRAMEBUFFER build, where the proxying logic is decided at runtime depending on whether target browser supports OffscreenCanvas, or if falling back to OFFSCREEN_FRAMEBUFFER is needed)

The unfortunate omission here is EGL, which does not work well with multithreading. Maybe it will work with some manual changes, but it would

Btw, when proxying GL functions, most of the GL functions need to utilize async proxying, as they are best to be run as fire-and-forget style and they do not return in any value. Only value-returning functions such as glGetError() and glReadPixels() and similar will need to proxy synchronously.

[turran]

Thanks @juj for the explanation. I can overcome the situation by using webglContext + GLES2 instead of EGL + GLES2. Given that the library I'm trying to port already does all the calls to GL in the same thread (and not the main thread), I can set EMSCRIPTEN_WEBGL_CONTEXT_PROXY_DISALLOW to proxyContextToMainThread when creating the context. The problem I'm facing now is that the desired target canvas has to be set before the thread is created by using emscripten_pthread_attr_settransferredcanvases, but I can not control what thread will be the rendering thread.

I'm checking the code on library_pthread.js and library_html5.js but doesn't seem straightforward. Before jumping in the needed modifications I'm wondering if there is a blocker for not making the context's canvas being "transferred" dynamically at context creation instead of statically setting it at the thread creation?

[juj]

I'm wondering if there is a blocker for not making the context's canvas being "transferred" dynamically at context creation instead of statically setting it at the thread creation?

That is because transferring a canvas to a worker as offscreencanvas is an asynchronous process. So when the worker is synchronously creating a GL context, it cannot block to wait for such an asynchronous event to come in.

It would be possible to create a separate JS function emscripten_transfer_canvas_to_pthread_async that would transfer a canvas after the pthread has already been created, but that will need to done with an async callback function pointer mechanism into the target thread to make sure it will not be racy. GL context creation in the worker should then only be attempted after the callback fires.

It would also be possible to create an asynchronous version of the GL context creation mechanism, which would send a message from the worker thread to the main thread to ask for a canvas, and then the main thread would send a second message over to the worker thread to pass the requested canvas over, and only then the context creation is attempted.

Neither of those mechanisms exist in Emscripten today however. I think if there is a core mechanism that would be added, it would be desirable to create one that would enable both WebGL and WebGPU behavior in the same manner.

[turran]

That is because transferring a canvas to a worker as offscreencanvas is an asynchronous process. So when the worker is synchronously creating a GL context, it cannot block to wait for such an asynchronous event to come in.

I'm not used to emscripten sync/async mechanics, but for a situation where the context should be created in a worker, isn't enough to asynchronously request the transfer of the canvas but still make the whole context creation sync and block until the return of such async call? At the end is a worker; no main thread will be blocked. (Thinking on the classic thread cond/mutex mechanism here).

Could you please point to some sources for messages from/to main thread examples?

[juj]

isn't enough to asynchronously request the transfer of the canvas but still make the whole context creation sync and block until the return of such async call?

That would unfortunately not work, since one cannot receive web events when synchrononously blocking. So the main thread transfering the canvas to a worker will not be received by that worker until it yields back to the event loop. That is why the context creation needs to be asynchronous.

[turran]

Thanks @juj for your explanations. I have some prototype almost working, the only missing part is setting some variables in the pthread's context. Similar to what is already there:

    receiveObjectTransfer: function(data) {
#if OFFSCREENCANVAS_SUPPORT
      if (typeof GL != 'undefined') {
        Object.assign(GL.offscreenCanvases, data.offscreenCanvases);
        if (!Module['canvas'] && data.moduleCanvasId && GL.offscreenCanvases[data.moduleCanvasId]) {
          Module['canvas'] = GL.offscreenCanvases[data.moduleCanvasId].offscreenCanvas;
          Module['canvas'].id = data.moduleCanvasId;
        }
      }
#endif

The problem is that the serialization between js calls passing through the main loop only accepts basic types (int, float, double, string, ertc), and I can't find an example of JS <-> JS code with complex values. Only C <-> JS through the HEAP/makeSetValue helpers.

So once the new offscreenCanvas is created on the main thread, I can't serialize it back to the caller. No IN/OUT parameters in the function call either. Is there any way to accomplish what I'm trying or simply it is not possible to serialize complex types in JS?

After reading the code I understand your point about messages and do everything through the postMessage/onmessage in the worker but the logic seems more complex than the above.

[turran]

Just following up with the implementation I'm doing, I'm following the asynchronous calls and post messages, but I'm confused about the threading model. Once the main thread has the offscreenCanvas and I want to post a message to the thread that wants the transferred canvas I don't receive such a message on any onmessage().

There is an onmessage at worker.js and another in library_pthread.js. But I can't understand the purpose and which should be receiving my message while the pthread is already created and running.

On the other end, from the POV of the C application, I guess that the thread wanting the context should somehow yield in order to let the onmessage() to be called. I tried the emscripten_thread_sleep with no luck

Can someone advise about the threading model and message handling? What should be called from the thread to let the message processing on that particular thread work?

[sbc100]

If you want to receive a postMessage on a pthread then you will need to return to the event loop.

You can do this with emscripten_exit_with_live_runtime() (which will return right away via thowning an unwind exception), or by calling emscripten_runtime_keepalive_push before returning normally.

[turran]

Thanks, now I have the whole picture of how to implement this. BTW, trying the gl_in_pthread test or either block or does not transfer the canvas to the second thread, depending on the specific defines. Checking the CircleCI results I couldn't find the test specific results

[turran]

Just to explain the solution found in case someone reaches this issue. As per version 3.1.61 using -sPROXY_TO_PTHREAD plus -sOFFSCREEN_CANVAS works like a charm. The thread correctly has the OffscreenCanvas associated to it. You just need some glue like setting the canvas id to be transferred in the pthread creation . The documentation is properly up to date on how to do it.