Skip to content

Running a python model instead#100

Closed
Peter9192 wants to merge 5 commits intomainfrom
python-backend
Closed

Running a python model instead#100
Peter9192 wants to merge 5 commits intomainfrom
python-backend

Conversation

@Peter9192
Copy link
Copy Markdown
Member

@Peter9192 Peter9192 commented Jan 14, 2025

I've been playing around to try and run python code from the web application. As such, we could have an analogous implementation in Python and use the nice web application to manage and analyse our experiments.

There are multiply ways to achieve this:

  • Pyodide: running a python interpreter in the browser.
    Advantage: easy (?!), actual python code is being executed
    Disadvantage: long load time as it needs to initialize the full pyodide environment (though only inside the web worker)
  • Transcrypt: transpiling the python code to js/ts: https://www.transcrypt.org/
    Advantage: easy and fast, code can be optimized during build
    Disadvantage: for the purpose of "proof of concept", this is not much different than just writing js/ts directly
  • Compiling python code directly to WASM, e.g. something like py2wasm
    Advantage: skips the overhead of loading pyodide and interpreting the python code; could scale to larger models and then be more efficient than js/ts
    Disadvantage: more challenging, would make more sense with more performance oriented libraries like Rust or C++

I decided to try the first option. Some notes

  • Pyodide docs on running in webworker https://pyodide.org/en/stable/usage/webworker.html#
  • However, we are using comlink; comlink doesn't like pyodide objects
  • It would be much easier if we simply exposed the run method only, instead of the full BMI with all controls
  • To still use BMI and comlink, we need a wrapper that takes care of properly forwarding function calls to python object and results back to js
  • Some nice ideas in this blogpost: https://adamemery.dev/articles/pyodide-react

I tried several approaches that go me almost there. Finally I managed to forward model output through comlink, but only through serialization on the python side and deserialization on the ts side. Then, comlink will do something similar again. How (un)desireable is that?

This PR still needs some work to reconcile the best elements of the different approaches I've tried and end up with a fully working version.

@Peter9192
Copy link
Copy Markdown
Member Author

Alternatively could keep BMI in ts and only CLASS in Python. Pro: comlink wraps ts object instead of pyodide. Con: more communication across pyodide boundary; Python BMI not self-sufficient (but is that necessary?)

@Peter9192 Peter9192 mentioned this pull request Jan 15, 2025
5 tasks
@Peter9192
Copy link
Copy Markdown
Member Author

Perhaps run should be implemented by the client rather than on the BMI instance, that would simplify life. Also see #101

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant