Add ElasticsearchRemoteLogIO.from_config and register elasticsearch scheme

[yuseok89]

closes: #70271
related: #70265
related: #67056

Migrates the Elasticsearch remote-logging backend to the provider dispatch mechanism: adds ElasticsearchRemoteLogIO.from_config() and registers an elasticsearch scheme, so [logging] remote_base_log_folder = elasticsearch:// routes through the provider.

The legacy host-based selection still works via the fallback in airflow_local_settings.py, so existing configs are unaffected.

End-to-end verification

breeze start-airflow --integration elasticsearch, remote_base_log_folder = elasticsearch://, ran a task with remote logging enabled.

Written to Elasticsearch:

[Breeze:3.10.20] root@e01884b83d1e:/opt/airflow$ curl -s "http://elasticsearch:9200/_cat/indices?v" | grep airflow-logs
yellow open   airflow-logs lejo5BB3RJa-3h0J0fzmBA   1   1         10            0     20.7kb         20.7kb       20.7kb
[Breeze:3.10.20] root@e01884b83d1e:/opt/airflow$ curl -s "http://elasticsearch:9200/airflow-logs/_search?size=1&pretty" | head -40
{
  "took" : 1104,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 10,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "airflow-logs",
        "_id" : "urj5o58BYIM3Bhrt3lZ7",
        "_score" : 1.0,
        "_source" : {
          "timestamp" : "2026-07-27T14:27:55.643862Z",
          "level" : "info",
          "event" : "::group::Pre Execute",
          "logger" : "task",
          "filename" : "task_runner.py",
          "lineno" : 2356,
          "log_id" : "example_xcom-push_sample_xcoms-manual__2026-07-27T14:27:53.127874+00:00--1-1",
          "offset" : 1
        }
      }
    ]
  }
}

Read back in the UI:

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)
  • Opus 4.8

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[potiuk]

I reviewed this alongside the four sibling ports (#70301 wasb, #70682 oss, #70549 stackdriver, #70295 opensearch), and this is the one that got the contentious question right.

All ten arguments match the legacy airflow_local_settings.py branch, and — the part that matters — it does not merge remote_task_handler_kwargs IO-kwargs. That is genuinely what legacy does for Elasticsearch, unlike the object-storage backends which apply | _io_kwargs. Recording the reasoning in the docstring rather than leaving it implicit is what makes the difference; #70295 hit the same fork and went the other way, and I have asked about it there.

One thing I checked closely: conf.get("elasticsearch", "host") or "http://localhost:9200" has no legacy counterpart, since legacy gated the whole branch on elif ELASTICSEARCH_HOST: and so could never reach construction with an empty host. Under scheme dispatch it can. The fallback turns out to be the class's own attrs default rather than a new value, and test_from_config_missing_host_keeps_class_default names precisely that, with a comment explaining that Elasticsearch("") would raise and silently disable remote logging. Good call to cover it.

The ProvidersManager test is worth having too — it verifies the provider.yaml registration actually resolves rather than just assuming the YAML is picked up, and the pytest.skip keeps it honest on cores without the dispatch mechanism.

Rebased onto main before merging.

---

Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting