Skip to content

Add mprocs support for breeze start-airflow command - #58702

Merged
potiuk merged 1 commit into
apache:mainfrom
potiuk:add-mprocs-support
Nov 26, 2025
Merged

Add mprocs support for breeze start-airflow command#58702
potiuk merged 1 commit into
apache:mainfrom
potiuk:add-mprocs-support

Conversation

@potiuk

@potiuk potiuk commented Nov 26, 2025

Copy link
Copy Markdown
Member

For years we used tmux as the driver for breeze's start-airflow command, however there are better alternatives.

One of the best ones available now which seems to perfectly match our needs is mprocs.

It's a modern alternative to tmux for managing multiple processes. It provides:

  • Intuitive TUI with better visual feedback
  • Easy process management (start, stop, restart)
  • Mouse and keyboard navigation
  • Process status indicators
  • Better cross-platform support

For now you need to use --use-mprocs flag to enable it, but once we test that it works well we can make it default.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@potiuk

potiuk commented Nov 26, 2025

Copy link
Copy Markdown
Member Author
Cinnamon-Nov-26-2025_03.51.46am.mov

@jason810496 jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like a super nice feature to try!

Comment thread contributing-docs/MPROCS_QUICK_REFERENCE.md Outdated
Comment thread scripts/mprocs/README.md Outdated
Comment thread scripts/mprocs/README.md Outdated
Comment thread scripts/mprocs/basic.yaml Outdated
Comment thread scripts/in_container/bin/generate_mprocs_config.py Outdated
Comment thread scripts/in_container/bin/generate_mprocs_config.py Outdated
Comment thread scripts/in_container/bin/generate_mprocs_config.py Outdated

@jscheffl jscheffl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool! Need to try (and I thought I am quite modern using tmux...)

Comment thread scripts/mprocs/basic.yaml Outdated

@amoghrajesh amoghrajesh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From an initial look, it looks good! Will take it for a spin!

Comment thread contributing-docs/MPROCS_QUICK_REFERENCE.md Outdated
Comment thread contributing-docs/MPROCS_QUICK_REFERENCE.md Outdated
Comment thread scripts/in_container/bin/generate_mprocs_config.py Outdated
Comment thread scripts/in_container/bin/generate_mprocs_config.py Outdated
Comment thread scripts/in_container/bin/generate_mprocs_config.py
Comment thread scripts/in_container/bin/generate_mprocs_config.py Outdated
Comment thread scripts/in_container/bin/run_mprocs
Comment thread scripts/in_container/bin/run_mprocs Outdated
Comment thread scripts/mprocs/basic.yaml Outdated
@amoghrajesh

Copy link
Copy Markdown
Contributor

Redis failures seem unrelated, rebasing.

@amoghrajesh amoghrajesh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried it out and absolutely love it! The copy mode is a game changer for me, makes navigating and copying much much easier!

@potiuk

potiuk commented Nov 26, 2025

Copy link
Copy Markdown
Member Author

Tried it out and absolutely love it! The copy mode is a game changer for me, makes navigating and copying much much easier!

Agree.. This is love from the first sight !

@potiuk
potiuk force-pushed the add-mprocs-support branch from bc847bf to 2c0b81a Compare November 26, 2025 09:52
For years we used tmux as the driver for breeze's start-airflow
command, however there are better alternatives.

One of the best ones available now which seems to perfectly match
our needs is mprocs.

It's a modern alternative to tmux for managing multiple processes. It provides:

- Intuitive TUI with better visual feedback
- Easy process management (start, stop, restart)
- Mouse and keyboard navigation
- Process status indicators
- Better cross-platform support

For now you need to use `--use-mprocs` flag to enable it, but once we
test that it works well we can make it default.
@potiuk

potiuk commented Nov 26, 2025

Copy link
Copy Markdown
Member Author

Made a few updates:

  • added pre-started "shell"
  • beautified output of config generation
Screenshot 2025-11-26 at 11 24 30
  • cleaned-up some tmux old remnant variables
  • changed the title to be Apache Airflow
  • fixed all the issue pointed out by reviewers (thanks!)

I noticed that in my iTerm2 on MacOS) - things are somewhat broken, COPY MODE does not turn on on mouse actions, and in docker you cannot switch to output panel (real bummer).

Standard terminal on MacOS works without problems - so a way to workaround it is to use the standard terminal.

I opened an issue for that pvolok/dekit#179

@potiuk

potiuk commented Nov 26, 2025

Copy link
Copy Markdown
Member Author

Also added an issue for iTerm2 https://gitlab.com/gnachman/iterm2/-/issues/12628

@choo121600 choo121600 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, I’m looking forward to it 🙌

@potiuk
potiuk merged commit b7b4385 into apache:main Nov 26, 2025
118 checks passed
@potiuk
potiuk deleted the add-mprocs-support branch November 26, 2025 11:07
@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-1-test. View the failure log Run details

Status Branch Result
v3-1-test Commit Link

You can attempt to backport this manually by running:

cherry_picker b7b4385 v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

potiuk added a commit to potiuk/airflow that referenced this pull request Nov 26, 2025
…ache#58702)

For years we used tmux as the driver for breeze's start-airflow
command, however there are better alternatives.

One of the best ones available now which seems to perfectly match
our needs is mprocs.

It's a modern alternative to tmux for managing multiple processes. It provides:

- Intuitive TUI with better visual feedback
- Easy process management (start, stop, restart)
- Mouse and keyboard navigation
- Process status indicators
- Better cross-platform support

For now you need to use `--use-mprocs` flag to enable it, but once we
test that it works well we can make it default.
(cherry picked from commit b7b4385)

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
@potiuk

potiuk commented Nov 26, 2025

Copy link
Copy Markdown
Member Author

Backport here #58720

potiuk added a commit that referenced this pull request Nov 26, 2025
…8702) (#58720)

For years we used tmux as the driver for breeze's start-airflow
command, however there are better alternatives.

One of the best ones available now which seems to perfectly match
our needs is mprocs.

It's a modern alternative to tmux for managing multiple processes. It provides:

- Intuitive TUI with better visual feedback
- Easy process management (start, stop, restart)
- Mouse and keyboard navigation
- Process status indicators
- Better cross-platform support

For now you need to use `--use-mprocs` flag to enable it, but once we
test that it works well we can make it default.
(cherry picked from commit b7b4385)
@Dev-iL

Dev-iL commented Nov 26, 2025

Copy link
Copy Markdown
Collaborator

Can we have this in airflow standalone too?

@jscheffl

jscheffl commented Nov 26, 2025

Copy link
Copy Markdown
Contributor

Can we have this in airflow standalone too?

It is not tooling but as long as somebody installs the (external) dependency - I'd think would be cool. In my view: Feel free to contribute. But should be (for the beginning at least) an optional flag.

@choo121600

Copy link
Copy Markdown
Member
image

Wow! It looks coool 🙌

@potiuk

potiuk commented Nov 28, 2025

Copy link
Copy Markdown
Member Author

Can we have this in airflow standalone too?

It is not tooling but as long as somebody installs the (external) dependency - I'd think would be cool. In my view: Feel free to contribute. But should be (for the beginning at least) an optional flag.

We might thing of something like that. PRs are most welcome @Dev-iL

@shahar1

shahar1 commented Nov 29, 2025

Copy link
Copy Markdown
Contributor

I've been away for quite a while, and already breeze gets a complete makeover.
Well done Jarek! #protm :)

ephraimbuddy pushed a commit that referenced this pull request Dec 3, 2025
…8702) (#58720)

For years we used tmux as the driver for breeze's start-airflow
command, however there are better alternatives.

One of the best ones available now which seems to perfectly match
our needs is mprocs.

It's a modern alternative to tmux for managing multiple processes. It provides:

- Intuitive TUI with better visual feedback
- Easy process management (start, stop, restart)
- Mouse and keyboard navigation
- Process status indicators
- Better cross-platform support

For now you need to use `--use-mprocs` flag to enable it, but once we
test that it works well we can make it default.
(cherry picked from commit b7b4385)
Copilot AI pushed a commit to jason810496/airflow that referenced this pull request Dec 5, 2025
For years we used tmux as the driver for breeze's start-airflow
command, however there are better alternatives.

One of the best ones available now which seems to perfectly match
our needs is mprocs.

It's a modern alternative to tmux for managing multiple processes. It provides:

- Intuitive TUI with better visual feedback
- Easy process management (start, stop, restart)
- Mouse and keyboard navigation
- Process status indicators
- Better cross-platform support

For now you need to use `--use-mprocs` flag to enable it, but once we
test that it works well we can make it default.
itayweb pushed a commit to itayweb/airflow that referenced this pull request Dec 6, 2025
For years we used tmux as the driver for breeze's start-airflow
command, however there are better alternatives.

One of the best ones available now which seems to perfectly match
our needs is mprocs.

It's a modern alternative to tmux for managing multiple processes. It provides:

- Intuitive TUI with better visual feedback
- Easy process management (start, stop, restart)
- Mouse and keyboard navigation
- Process status indicators
- Better cross-platform support

For now you need to use `--use-mprocs` flag to enable it, but once we
test that it works well we can make it default.
Subham-KRLX pushed a commit to Subham-KRLX/airflow that referenced this pull request Mar 4, 2026
For years we used tmux as the driver for breeze's start-airflow
command, however there are better alternatives.

One of the best ones available now which seems to perfectly match
our needs is mprocs.

It's a modern alternative to tmux for managing multiple processes. It provides:

- Intuitive TUI with better visual feedback
- Easy process management (start, stop, restart)
- Mouse and keyboard navigation
- Process status indicators
- Better cross-platform support

For now you need to use `--use-mprocs` flag to enable it, but once we
test that it works well we can make it default.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants