Split server up into manager + executor paradigm#2597
Merged
joeyballentine merged 50 commits intomainfrom Mar 1, 2024
Merged
Conversation
joeyballentine
force-pushed
the
double-server
branch
from
4bf856d to
86924a1
Compare
Member
Author
|
New issue discovered: Video iteration can fail/freeze when FFMPEG outputs to stdout. I currently have no idea how to resolve this and i would consider it a blocker for merging. |
* Make `DependencyInfo` a data class to simplify code * Minor bug
RunDevelopment
approved these changes
Mar 1, 2024
Member
RunDevelopment
left a comment
RunDevelopment
left a comment
There was a problem hiding this comment.
Feel free to merge when you think this is ready. The general approach and architecture look good. If it works, then it's ready.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A draft PR for what we've been discussing on Discord. Not ready to actually be merged yet.
This separates the server into two parts: The manager server which handles dependency tracking and installation, and the executor server, which handles package/node loading.
This is necessary to allow us to update and uninstall dependencies on the backend, due to the to-be-modified dependencies needing to not be loaded in python. With this new paradigm, the executor server can be killed while the manager server can stay open and manage the dependencies properly, then start the executor server back up. This also would allow us to implement the "timeout killing" feature backend-side rather than frontend-side, decoupling that even more.
The manager server acts as a proxy for most requests, while other requests are handled by it.
This does not actually implement the aforementioned dependency management and executor-killing features. Those will be tackled in a separate future PR once this one is finalized. This PR is an attempt to mimic current behavior as closely as possible.
Before being considered ready, this PR needs to be heavily tested, including through brand-new installs, to ensure no regressions.
TODO:
requestsdependency change