Minimal runtime pthreads by juj · Pull Request #10123 · emscripten-core/emscripten

juj · 2019-12-29T14:07:59Z

This adds support for multithreading with MINIMAL_RUNTIME, in both regular build mode and MODULARIZEd build mode.

juj · 2020-02-12T14:48:28Z

Brought this PR up to latest and all tests pass. This would be great to see a review and get landed soon, it'll bump up MINIMAL_RUNTIME features with threading support!

juj · 2020-02-12T14:49:58Z

-    STACK_MAX = stackMax;
-  },
-#endif
-


This deletion is not strictly related to MINIMAL_RUNTIME multithreading support, but while doing this PR I realized this whole function is redundant, so simplified the code.

juj · 2020-02-12T14:50:31Z

  if shared.Settings.USE_PTHREADS:
    funcs.append('''
-function establishStackSpace(stackBase, stackMax) {
+function asmJsEstablishStackFrame(stackBase, stackMax) {


Renamed this to highlight the fact that this functionality is only needed for old fastcomp threading support.

juj · 2020-02-12T14:51:09Z

 #endif
+#if !MINIMAL_RUNTIME
        'DYNAMIC_BASE': DYNAMIC_BASE,
+#endif


In MINIMAL_RUNTIME the DYNAMIC_BASE variable is not a thing.

juj · 2020-02-12T14:52:54Z

    // Creates a new web Worker and places it in the unused worker pool to wait for its use.
    allocateUnusedWorker: function() {
+#if MINIMAL_RUNTIME
+      var pthreadMainJs = Module['worker'];


This use of Module['worker'] is probably temporary/internal, and will go away later as the src/worker.js refactor happens.

juj · 2020-02-12T14:54:39Z

 #if !ALLOW_BLOCKING_ON_MAIN_THREAD
    abort('Blocking on the main thread is not allowed by default. See https://emscripten.org/docs/porting/pthreads.html#blocking-on-the-main-browser-thread');
+#endif
+


In MINIMAL_RUNTIME we do not want to pay code size for emscripten_check_blocking_allowed type of checks, so this is not generated there.

juj · 2020-02-12T14:55:38Z


 if (typeof Module !== 'undefined' && typeof document !== 'undefined' && typeof window !== 'undefined' && typeof process === 'undefined') emscriptenMemoryProfiler.initialize();
+
+#endif


Moving this #ifdef was done to be more idiomatic with include guards (the file itself guards, rather than the includer of that file)

juj · 2020-02-12T15:03:24Z

  report_result__sig: 'viii',
  report_result: function(param1, param2, param3) {
-    if (ENVIRONMENT_IS_WORKER) {
+    if (typeof ENVIRONMENT_IS_WORKER !== 'undefined' && ENVIRONMENT_IS_WORKER) {


This test change and all the test changes below are to make pthreads tests pass in MINIMAL_RUNTIME. It enables a workflow where I locally edit src/settings.js directly to change var MINIMAL_RUNTIME = 1; by default to run the whole browser.test_pthread* test suite in MINIMAL_RUNTIME, even though we don't duplicate all browser tests twice for MINIMAL_RUNTIME and non-MINIMAL_RUNTIME cases (only select tests are duplicated)

kripken · 2020-02-14T00:22:25Z

    # loading external .asm.js file that assigns to Module['asm'] works even when Closure is used.
-    if shared.Settings.MINIMAL_RUNTIME and (shared.Settings.target_environment_may_be('node') or
-                                            shared.Settings.target_environment_may_be('shell')):
+    if shared.Settings.MINIMAL_RUNTIME and not shared.Settings.MODULARIZE_INSTANCE and (shared.Settings.target_environment_may_be('node') or shared.Settings.target_environment_may_be('shell')):


why does modularize_instance matter here? (and, why not modularize?)

MODULARIZE_INSTANCE creates the Module on the next line, emitting

if(typeof Module==="undefined"){var Module={};} var Module = (

would be redundant for code size.

kripken · 2020-02-14T00:30:31Z

-#if ASSERTIONS
-    assert(ENVIRONMENT_IS_WEB);
-    warnOnce('Blocking on the main thread is very dangerous, see https://emscripten.org/docs/porting/pthreads.html#blocking-on-the-main-browser-thread');
+#if ASSERTIONS || IN_TEST_HARNESS || !MINIMAL_RUNTIME || !ALLOW_BLOCKING_ON_MAIN_THREAD


I don't understand why !MINIMAL_RUNTIME is here? This code is already behind ASSERTIONS, so it won't be in an optimized build with the minimal runtime or not? I think it's useful in assertion builds to have maximal checks, regardless of runtime.

Oh, that is a || !MINIMAL_RUNTIME statement and not a && !MINIMAL_RUNTIME statement. The || !MINIMAL_RUNTIME statement is here so that regular runtime will get the non-assertion release builds also to contain the allow blocking checks.

kripken · 2020-02-14T00:32:46Z


+    if (USE_PTHREADS) { // TODO: Currently WebGL proxying makes pthreads library depend on WebGL.
+      libraries.push('library_webgl.js');
+    }


is this a fix unrelated to this PR?

(we don't deduplicate this list, so this is extra work, and potentially buggy if we have global state - we should deduplicate it before doing this, probably)

This fix is not unrelated - in MINIMAL_RUNTIME and STRICT we don't autolink libraries, and that would cause pthreads builds to fail at missing GL libraries. Added deduplication.

kripken · 2020-02-14T00:37:46Z

+#if MODULARIZE_INSTANCE
+      var imports = Module;
+#else
+      var imports = {};


how does this empty imports object work? is it filled later? (seems not?) do we not need imports somehow?

The imports are filled from makeAsmImportsAccessInPthread() function.

kripken · 2020-02-14T00:46:29Z

-	getDomElementParentInnerHTML(name, dst, sizeof(dst));
+	char name[7] = "body";
+	const char *dst = getDomElementContents(name);
 	memset(name, 0, sizeof(name)); // Try to uncover if there might be a race condition and above line was not synchronously processed, and we could take name string away.


are the changes to this file necessary? (comment above said they were, but I don't see how?)

The old test asserted the existence of an inner HTML text node with text contents "resize" in it (that was the [ ] resize canvas checkbox in default shell). In MINIMAL_RUNTIME shell such checkbox does not exist, so I replaced it with a string snippet <canvas that can be found in both shells.

kripken · 2020-02-14T00:48:55Z


-		emscripten_set_main_loop(WaitToJoin, 0, 0);
+		emscripten_set_timeout_loop(WaitToJoin, 100, 0);
 	} else {


same question for this file - why doesn't it work in minimal_runtime without these?

MINIMAL_RUNTIME does not support library_browser.js because the $Browser object does not DCE well if linked in. This test was not looking to exercise emscripten_set_main_loop, so changed to emscripten_set_timeout_loop for the same effect.

juj · 2020-02-17T12:51:49Z

Ping, any chance this could get a second review pass?

…own file.

kripken

lgtm, and good plan for the followup!

juj added multithreading code size labels Dec 29, 2019

juj force-pushed the minimal_runtime_pthreads branch 2 times, most recently from 12011bb to e832463 Compare January 13, 2020 19:51

juj changed the base branch from incoming to master January 14, 2020 18:56

juj force-pushed the minimal_runtime_pthreads branch from e832463 to 428287a Compare January 18, 2020 12:06

juj force-pushed the minimal_runtime_pthreads branch 5 times, most recently from 870d380 to ea02fab Compare February 11, 2020 14:18

juj commented Feb 12, 2020

View reviewed changes

kripken reviewed Feb 14, 2020

View reviewed changes

juj force-pushed the minimal_runtime_pthreads branch from 2d1e36e to 2290619 Compare February 17, 2020 12:51

juj added 8 commits February 17, 2020 14:56

Add pthreads support to MINIMAL_RUNTIME.

d486dc1

Multithreaded MINIMAL_RUNTIME builds working with MODULARIZE.

4a787fe

Fix up MINIMAL_RUNTIME pthreads

4b8db00

Pthreads fixes in MINIMAL_RUNTIME.

f8e47fe

MINIMAL_RUNTIME fixes to pthreads.

fa9759a

Add threading related closure externs

ae6f617

Remove 'establishStackSpace' variable.

8a215be

Rename establishStackSpaceInJsModule to establishStackSpace.

b532545

juj added 19 commits February 17, 2020 14:56

flake

8713949

pthread_cleanup_push needs ATEXIT registration only if EXIT_RUNTIME=1.

405d731

Offset Converter loading for MINIMAL_RUNTIME pthreads

e91ee95

Add note about MODULARIZE_INSTANCE with MINIMAL_RUNTIME=1

cbb28db

MODULARIZE_INSTANCE working with MINIMAL_RUNTIME.

8c7e3d3

Fix Closure error

1a0d4e2

Separate MINIMAL_RUNTIME + USE_PTHREADS + !MODULARIZE externs to its …

ddd2ff8

…own file.

Remove unused function makeAsmImportsAccessInPthread

ef424d9

Remove unused STREAMING_WASM_COMPILATION

e09448f

More closure externs

46e4e03

Update call_sync_on_main_thread test

374ab6d

Fix emscripten_check_blocking_allowed

a2dd850

Restore asm.js specific function for setting the stack frame

4b8e4c5

Add phread exports to MINIMAL_RUNTIME

26395c7

Remove removeEmptyElements(x) as redundant

eac77f1

Remove duplicate MINIMAL_RUNTIME pthread test

9188ac7

Add deduplication of JS library files

ad9e79c

makeAsmImportsAccessInPthread('wasm')

eea85f6

Address review

61d0ea0

juj force-pushed the minimal_runtime_pthreads branch from 2290619 to 61d0ea0 Compare February 17, 2020 12:56

kripken approved these changes Feb 18, 2020

View reviewed changes

juj merged commit 673915c into emscripten-core:master Feb 18, 2020

sbc100 mentioned this pull request Feb 28, 2023

Make emscripten_check_blocking_allowed() no-op in release builds #18871

Merged


		if (typeof Module !== 'undefined' && typeof document !== 'undefined' && typeof window !== 'undefined' && typeof process === 'undefined') emscriptenMemoryProfiler.initialize();

		#endif

Uh oh!

Conversation

juj commented Dec 29, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

juj commented Feb 12, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

juj commented Feb 17, 2020

Uh oh!

kripken left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

juj commented Dec 29, 2019 •

edited

Loading