Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
53 changes: 43 additions & 10 deletions contributing-docs/mprocs/MPROCS_QUICK_REFERENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,11 @@
breeze start-airflow --use-mprocs
```

Breeze will start Airflow components using mprocs to manage multiple processes in a single terminal window.
It will generate dynamically the required configuration based on the selected executor and options and use it,
the generated configuration file is stored in a `files` folder inside the container, so that you can also
use it outside of Breeze if needed.

## Common Usage Patterns

| Command | Description |
Expand All @@ -38,16 +43,15 @@ breeze start-airflow --use-mprocs

## mprocs Keyboard Shortcuts

| Key | Action |
|---------|------------------------------|
| `↑↓` | Navigate between processes |
| `Space` | Show/hide process output |
| `r` | Restart selected process |
| `k` | Kill selected process |
| `s` | Start selected process |
| `a` | Toggle showing all processes |
| `q` | Quit mprocs |
| `?` | Show help |
| Key | Action |
|------|----------------------------|
| `↑↓` | Navigate between processes |
| `r` | Restart selected process |
| `x` | Stop selected process |
| `s` | Start selected process |
| `a` | Add new process |
| `q` | Quit mprocs |
| `?` | Show help |

## Components Managed

Expand Down Expand Up @@ -97,3 +101,32 @@ curl -L "https://github.com/pvolok/mprocs/releases/download/v${MPROCS_VERSION}/m
| tar -xz -C /usr/local/bin/ mprocs
chmod +x /usr/local/bin/mprocs
```

## Mac OS X and iTerm2

There are some known issues with mprocs and iTerm2 (which is often used as terminal by
developers using MacOS) with default settings, but they have solutions and workarounds:

* Mouse clicks are not captured correctly by default

**Solution:** you need to configure "Enable Mouse reporting":

![Enable mouse reporting](../images/iterm2-enable-mouse-reporting.png)

* Ctrl-a does not work correctly when in breeze Docker container (issue https://github.com/pvolok/mprocs/issues/179)

**Workaround**: You need to use `Cmd - <-` to change focus instead of Ctrl-a. Also you can switch
between process list and output with mouse click providing that you applied the solution above.

## Standalone execution

You can run mprocs outside of Breeze for custom setups. Create a `mprocs.yaml` configuration
file defining your processes, then start mprocs:

```bash
mprocs -f mprocs.yaml
```

An example [mprocs.yaml](mprocs.yaml) file for Airflow components can be found in this directory - it has
default configurations for all major Airflow components, but you can customize it as needed by copying it
elsewhere and modifying the process definitions, uncommenting the commented or adding new processes.
123 changes: 0 additions & 123 deletions contributing-docs/mprocs/README.md

This file was deleted.

8 changes: 7 additions & 1 deletion scripts/in_container/bin/generate_mprocs_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,13 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

# /// script
# requires-python = ">=3.10"
# dependencies = [
# "rich>=13.6.0",
# "PyYAML>=6.0",
# ]
# ///
"""Generate mprocs configuration dynamically based on environment variables."""

from __future__ import annotations
Expand Down
12 changes: 2 additions & 10 deletions scripts/in_container/bin/run_mprocs
Original file line number Diff line number Diff line change
Expand Up @@ -15,19 +15,11 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

# Use LocalExecutor if not set and if backend is not sqlite as it gives
# better performance
if [[ ${BACKEND} != "sqlite" ]]; then
export AIRFLOW__CORE__EXECUTOR=${AIRFLOW__CORE__EXECUTOR:-LocalExecutor}
fi

# Generate mprocs configuration dynamically
MPROCS_CONFIG_PATH="/tmp/mprocs-airflow.yaml"

python /opt/airflow/scripts/in_container/bin/generate_mprocs_config.py "${MPROCS_CONFIG_PATH}"
MPROCS_CONFIG_PATH="/files/mprocs.yaml"

if [[ $? -ne 0 ]]; then
if ! uv run "/opt/airflow/scripts/in_container/bin/generate_mprocs_config.py" "${MPROCS_CONFIG_PATH}"; then
echo "Failed to generate mprocs configuration"
exit 1
fi
Expand Down
Loading