From 512aa1dd9876daf072cf4223118646ab5d0df59d Mon Sep 17 00:00:00 2001 From: LIU ZHE YOU Date: Wed, 24 Jun 2026 17:43:05 +0900 Subject: [PATCH 1/4] Add macOS setup and JUL logging guidance to Java SDK docs ## Why Running the Java SDK on macOS hits two failures the docs did not cover (the task JVM's callback connection being rejected, and duplicated/mislabeled JUL logs), plus a coordinator-config reload gotcha. ## What - Add a "macOS setup" section: the `-Djava.net.preferIPv4Stack=true` workaround for the `Rejected connection not owned by child process` failure, and pinning `java_executable` to an absolute path. - Update the `java.util.logging` section to remove JUL's default handlers before `AirflowJulHandler.install()`, so records are not also captured as `task.stderr` at ERROR level (duplicate lines, INFO shown as errors). - Note in the coordinator configuration section that `[sdk]` config is read at startup and only takes effect after a restart, while a rebuilt bundle JAR is picked up on the next task launch. --- ##### Was generative AI tooling used to co-author this PR? - [x] Yes, with help of Claude Code Opus 4.8 following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) --- .../language-sdks/java.rst | 58 ++++++++++++++++++- 1 file changed, 57 insertions(+), 1 deletion(-) diff --git a/airflow-core/docs/authoring-and-scheduling/language-sdks/java.rst b/airflow-core/docs/authoring-and-scheduling/language-sdks/java.rst index 25dac2e2e1043..870f5930cd32d 100644 --- a/airflow-core/docs/authoring-and-scheduling/language-sdks/java.rst +++ b/airflow-core/docs/authoring-and-scheduling/language-sdks/java.rst @@ -341,15 +341,33 @@ Add the artifact: implementation("org.apache.airflow:airflow-sdk-jul:${version}") and call ``AirflowJulHandler.install()`` on startup to attach the handler to the -JUL root logger before any task runs: +JUL root logger before any task runs. Remove JUL's default handlers first, so +records flow only through ``AirflowJulHandler``: .. code-block:: java + import java.util.logging.Handler; + import java.util.logging.Logger; + public static void main(String[] args) { + Logger root = Logger.getLogger(""); + for (Handler h : root.getHandlers()) { + root.removeHandler(h); + } AirflowJulHandler.install(); Server.create(args).serve(new MyBundle()); } +.. note:: + + ``install()`` only adds the handler; it does not remove JUL's default + ``ConsoleHandler``. If you leave that handler in place, every record is also + written to stderr, which Airflow captures separately as ``task.stderr`` at + ERROR level, so each line appears twice in the UI and ``INFO`` records are + mislabeled as errors. The ``logging.properties`` approach below avoids this, + since the ``handlers`` property replaces the root handler set instead of + adding to it. + Alternatively, declare the handler in a ``logging.properties`` file and point JUL at it with the ``java.util.logging.config.file`` system property (set via ``jvm_args`` in the coordinator configuration): @@ -669,6 +687,44 @@ All ``kwargs`` in the ``coordinators`` config entry are passed to the - Seconds to wait for the JVM subprocess to connect after launch. Increase this if your JVM startup is slow (e.g. on constrained hardware or with a large classpath). +.. note:: + + The ``[sdk]`` configuration is read at startup, so changes to ``coordinators`` or + ``queue_to_coordinator`` (for example adding ``jvm_args``) only take effect after you restart the + scheduler (or ``airflow standalone``). A rebuilt bundle JAR, by contrast, is picked up on the next + task launch without a restart, because a fresh JVM is spawned per task instance. + +.. _java-sdk/macos: + +macOS setup +----------- + +Two extra coordinator settings are needed when running tasks on macOS: + +* **Force the JVM onto IPv4.** The coordinator verifies that the task JVM owns its callback socket. On + macOS a dual-stack JVM reports its local address in the IPv4-compatible form (``::127.0.0.1``), + which fails that check, so the task exits immediately with + ``Rejected connection not owned by child process``. Add ``-Djava.net.preferIPv4Stack=true`` to + ``jvm_args`` so the JVM binds plain IPv4. +* **Pin** ``java_executable`` **to an absolute path.** Homebrew does not symlink the JDK onto + ``$PATH``, so set ``java_executable`` to the JDK you intend to use rather than relying on ``java`` + resolving from ``$PATH``. + +.. code-block:: ini + + [sdk] + coordinators = { + "java-jdk17": { + "classpath": "airflow.sdk.coordinators.java.JavaCoordinator", + "kwargs": { + "jars_root": ["/opt/airflow/jars"], + "jvm_args": ["-Djava.net.preferIPv4Stack=true"], + "java_executable": "/opt/homebrew/opt/openjdk@17/bin/java" + } + } + } + queue_to_coordinator = {"java": "java-jdk17"} + .. _java-sdk/limitations: Limitations From 6666a3bb753cd64ec6e576e6270c38c7e9b7b107 Mon Sep 17 00:00:00 2001 From: LIU ZHE YOU Date: Thu, 25 Jun 2026 15:21:17 +0900 Subject: [PATCH 2/4] Remove the JVM onto IPv4 workaround --- .../language-sdks/java.rst | 14 +++----------- 1 file changed, 3 insertions(+), 11 deletions(-) diff --git a/airflow-core/docs/authoring-and-scheduling/language-sdks/java.rst b/airflow-core/docs/authoring-and-scheduling/language-sdks/java.rst index 870f5930cd32d..7b2e4ef322e67 100644 --- a/airflow-core/docs/authoring-and-scheduling/language-sdks/java.rst +++ b/airflow-core/docs/authoring-and-scheduling/language-sdks/java.rst @@ -699,16 +699,9 @@ All ``kwargs`` in the ``coordinators`` config entry are passed to the macOS setup ----------- -Two extra coordinator settings are needed when running tasks on macOS: - -* **Force the JVM onto IPv4.** The coordinator verifies that the task JVM owns its callback socket. On - macOS a dual-stack JVM reports its local address in the IPv4-compatible form (``::127.0.0.1``), - which fails that check, so the task exits immediately with - ``Rejected connection not owned by child process``. Add ``-Djava.net.preferIPv4Stack=true`` to - ``jvm_args`` so the JVM binds plain IPv4. -* **Pin** ``java_executable`` **to an absolute path.** Homebrew does not symlink the JDK onto - ``$PATH``, so set ``java_executable`` to the JDK you intend to use rather than relying on ``java`` - resolving from ``$PATH``. +On macOS, pin ``java_executable`` to an absolute path. Homebrew does not symlink the JDK onto +``$PATH``, so set ``java_executable`` to the JDK you intend to use rather than relying on ``java`` +resolving from ``$PATH``. .. code-block:: ini @@ -718,7 +711,6 @@ Two extra coordinator settings are needed when running tasks on macOS: "classpath": "airflow.sdk.coordinators.java.JavaCoordinator", "kwargs": { "jars_root": ["/opt/airflow/jars"], - "jvm_args": ["-Djava.net.preferIPv4Stack=true"], "java_executable": "/opt/homebrew/opt/openjdk@17/bin/java" } } From 6a2b0b061faec9dc1ea07abbc2a44f4b65954523 Mon Sep 17 00:00:00 2001 From: LIU ZHE YOU Date: Thu, 25 Jun 2026 22:26:22 +0900 Subject: [PATCH 3/4] Java SDK docs: generalize java_executable advice and use JUL setup() The macOS note assumed Homebrew; reframe it as a general recommendation to pin java_executable to an absolute path (most valuable where the admin does not control the system-wide java), with Homebrew as one example. For JUL, the example called install() then manually stripped the root handlers; that removal now lives in AirflowJulHandler.setup(), so the docs call setup() and drop the manual loop and the double-logging note. --- .../language-sdks/java.rst | 42 +++++++------------ 1 file changed, 15 insertions(+), 27 deletions(-) diff --git a/airflow-core/docs/authoring-and-scheduling/language-sdks/java.rst b/airflow-core/docs/authoring-and-scheduling/language-sdks/java.rst index 7b2e4ef322e67..33b4de5f48a16 100644 --- a/airflow-core/docs/authoring-and-scheduling/language-sdks/java.rst +++ b/airflow-core/docs/authoring-and-scheduling/language-sdks/java.rst @@ -340,34 +340,18 @@ Add the artifact: implementation("org.apache.airflow:airflow-sdk-jul:${version}") -and call ``AirflowJulHandler.install()`` on startup to attach the handler to the -JUL root logger before any task runs. Remove JUL's default handlers first, so -records flow only through ``AirflowJulHandler``: +and call ``AirflowJulHandler.setup()`` on startup, before any task runs. It clears the JUL root +logger's existing handlers (including the default ``ConsoleHandler``, whose stderr output Airflow +would otherwise capture as ``task.stderr`` at ERROR level, duplicating each record and mislabeling +its level) and installs ``AirflowJulHandler`` in their place: .. code-block:: java - import java.util.logging.Handler; - import java.util.logging.Logger; - public static void main(String[] args) { - Logger root = Logger.getLogger(""); - for (Handler h : root.getHandlers()) { - root.removeHandler(h); - } - AirflowJulHandler.install(); + AirflowJulHandler.setup(); Server.create(args).serve(new MyBundle()); } -.. note:: - - ``install()`` only adds the handler; it does not remove JUL's default - ``ConsoleHandler``. If you leave that handler in place, every record is also - written to stderr, which Airflow captures separately as ``task.stderr`` at - ERROR level, so each line appears twice in the UI and ``INFO`` records are - mislabeled as errors. The ``logging.properties`` approach below avoids this, - since the ``handlers`` property replaces the root handler set instead of - adding to it. - Alternatively, declare the handler in a ``logging.properties`` file and point JUL at it with the ``java.util.logging.config.file`` system property (set via ``jvm_args`` in the coordinator configuration): @@ -694,14 +678,18 @@ All ``kwargs`` in the ``coordinators`` config entry are passed to the scheduler (or ``airflow standalone``). A rebuilt bundle JAR, by contrast, is picked up on the next task launch without a restart, because a fresh JVM is spawned per task instance. -.. _java-sdk/macos: +.. _java-sdk/java-executable: -macOS setup ------------ +Pinning the Java executable +--------------------------- + +As a general recommendation, set ``java_executable`` to an absolute path rather than relying on +``java`` resolving from ``$PATH``. This pins tasks to a known JDK, which matters most in production or +corporate environments where the Airflow admin may not control the system-wide ``java`` (the same +reasoning behind pinning a Python version). -On macOS, pin ``java_executable`` to an absolute path. Homebrew does not symlink the JDK onto -``$PATH``, so set ``java_executable`` to the JDK you intend to use rather than relying on ``java`` -resolving from ``$PATH``. +For example, if you install the JDK with Homebrew on macOS it is not symlinked onto ``$PATH``, so +point ``java_executable`` at it explicitly: .. code-block:: ini From e33e266ec303d9ce6e38b57044fef0ea3ec54541 Mon Sep 17 00:00:00 2001 From: LIU ZHE YOU Date: Fri, 26 Jun 2026 12:06:33 +0900 Subject: [PATCH 4/4] Java SDK docs: fix docs spellcheck failure for 'symlinked' The docs spellcheck build (Build documentation --spellcheck-only) rejected 'symlinked', which is not in docs/spelling_wordlist.txt. Reword the Homebrew example to plain wording ("its java is not on $PATH") so the build passes without growing the wordlist. --- .../docs/authoring-and-scheduling/language-sdks/java.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/airflow-core/docs/authoring-and-scheduling/language-sdks/java.rst b/airflow-core/docs/authoring-and-scheduling/language-sdks/java.rst index 33b4de5f48a16..55a80f9967fce 100644 --- a/airflow-core/docs/authoring-and-scheduling/language-sdks/java.rst +++ b/airflow-core/docs/authoring-and-scheduling/language-sdks/java.rst @@ -688,7 +688,7 @@ As a general recommendation, set ``java_executable`` to an absolute path rather corporate environments where the Airflow admin may not control the system-wide ``java`` (the same reasoning behind pinning a Python version). -For example, if you install the JDK with Homebrew on macOS it is not symlinked onto ``$PATH``, so +For example, if you install the JDK with Homebrew on macOS, its ``java`` is not on ``$PATH``, so point ``java_executable`` at it explicitly: .. code-block:: ini