Skip to content

git dag bundle to use the private key in submodule pull - #62938

Merged
potiuk merged 1 commit into
apache:mainfrom
Pandry:main
Aug 1, 2026
Merged

git dag bundle to use the private key in submodule pull#62938
potiuk merged 1 commit into
apache:mainfrom
Pandry:main

Conversation

@Pandry

@Pandry Pandry commented Mar 5, 2026

Copy link
Copy Markdown
Contributor

The git dag bundle did not use the private key to
pull submodules.
This lead to issues when working with private submodules. This commit uses the same key used to pull the dag bundle to pull the submodules

I literally copy-pasted the code from _fetch_bare_repo

@Ajay9704 Ajay9704 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the CI failure is due to a trailing whitespace change. Fixing the whitespace should resolve the static check failure.

@Pandry

Pandry commented Mar 13, 2026

Copy link
Copy Markdown
Contributor Author

Oh; I saw the diff but I was not 100% sure it was the newline!

@potiuk
potiuk marked this pull request as draft April 2, 2026 16:42
@potiuk

potiuk commented Apr 2, 2026

Copy link
Copy Markdown
Member

@Pandry Converting to draft — this PR doesn't yet meet our Pull Request quality criteria.

  • Build docs: Failing: CI image checks / Build documentation (--docs-only). Run breeze build-docs locally to reproduce. See Build docs docs.
  • Other failing CI checks: Failing: CI image checks / Build documentation (--spellcheck-only), Low dep tests:core / All-core:LowestDeps:14:3.10:API...Serialization. Run prek run --from-ref main locally to reproduce. See static checks docs.

Note: Your branch is 867 commits behind main. Please rebase and push again to get up-to-date CI results.

See the linked criteria for how to fix each item, then mark the PR "Ready for review". This is not a rejection — just an invitation to bring the PR up to standard. No rush.


Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

@Pandry

Pandry commented Apr 2, 2026

Copy link
Copy Markdown
Contributor Author

Hi, I just rebased on main, but the build docs link you provided is a 404 (also I did not touch the documentation)

@Pandry

Pandry commented Apr 7, 2026

Copy link
Copy Markdown
Contributor Author

Hi, I didn't hear back from my last comment
I've just rebased agin from main
All the failing tests are not about my commit in particular, so I assume it is fine to perhaps submit again for review?

@Pandry
Pandry marked this pull request as ready for review April 7, 2026 10:52
@kaxil
kaxil requested a review from Copilot April 10, 2026 19:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates GitDagBundle submodule initialization to reuse the same SSH private-key configuration used when fetching the main DAG bundle, fixing pulls for private git submodules.

Changes:

  • Wrap git submodule sync/update with a custom_environment(GIT_SSH_COMMAND=...) context when GitHook provides GIT_SSH_COMMAND.
  • Align submodule-fetch authentication behavior with _fetch_bare_repo.

Comment thread providers/git/src/airflow/providers/git/bundles/git.py
@potiuk
potiuk marked this pull request as draft April 22, 2026 15:11
@potiuk

potiuk commented Apr 22, 2026

Copy link
Copy Markdown
Member

@Pandry Thanks for your earlier replies — some threads are still marked unresolved and there has been no movement for 20 days. Converting to draft — this PR doesn't yet meet our Pull Request quality criteria.

  • Unresolved review comments (1 thread): please walk through each unresolved review thread. Even if a suggestion looks incorrect or irrelevant — and some of them will be, especially any comments left by automated reviewers like GitHub Copilot — it is still the author's responsibility to respond: apply the fix, reply in-thread with a brief explanation of why the suggestion does not apply, or resolve the thread if the feedback is no longer relevant. Leaving threads unaddressed for weeks blocks the PR from moving forward.

See the linked criteria for how to fix each item, then mark the PR "Ready for review". This is not a rejection — just an invitation to bring the PR up to standard. No rush.


Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

@potiuk

potiuk commented Apr 22, 2026

Copy link
Copy Markdown
Member

Quick follow-up to the triage comment above — one clarification on the "Unresolved review comments" item:

Once you believe a thread has been addressed — whether by pushing a fix, or by replying in-thread with an explanation of why the suggestion doesn't apply — please mark the thread as resolved yourself by clicking the "Resolve conversation" button at the bottom of each thread. Reviewers don't auto-close their own threads, so an addressed-but-unresolved thread reads as "still waiting on the author" and keeps the PR from moving forward. The author doing the resolve-click is the expected convention on this project.


Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

@Pandry
Pandry force-pushed the main branch 2 times, most recently from 1ca3260 to 6a853e1 Compare April 25, 2026 18:12
@Pandry

Pandry commented Apr 25, 2026

Copy link
Copy Markdown
Contributor Author

Added tests (and ensure they don't pass with the previous commit) and rebased from main
Hopefully that's good now!

@Pandry
Pandry marked this pull request as ready for review April 25, 2026 18:13
@potiuk

potiuk commented May 10, 2026

Copy link
Copy Markdown
Member

There are still errors/

@potiuk

potiuk commented May 18, 2026

Copy link
Copy Markdown
Member

@Pandry A few things need addressing before review — see our Pull Request quality criteria.

Issues found:

  • Pre-commit / static checks: CI image checks / Static checks is failing. Run prek run --from-ref main --stage pre-commit locally and fix anything that flags. See the static-checks docs.

What to do next:

  • Push a fix for the static-check failure.

No rush — take your time. We appreciate your contribution and are happy to wait for updates. If you have questions, feel free to ask on the Airflow Slack.


Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

@potiuk

potiuk commented May 24, 2026

Copy link
Copy Markdown
Member

@Pandry A few things need addressing before review — see our Pull Request quality criteria.

  • CI fails: CI image checks / Static checks (and possibly other checks — see the Checks tab for the full list).

No rush.


Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.


Drafted-by: Claude Code (Opus 4.7); reviewed by @potiuk before posting

@Pandry

Pandry commented May 24, 2026

Copy link
Copy Markdown
Contributor Author

Yeah, sorry, I've been very sloppy with this, got a lot going on and it's not pressing because I added the patch as part of my image pipeline


I was sure I tested everything and all tests passed (including the pre-commit) but I might have done that before the commit, so the pre-commit changed the files but I didn't commit them
I'll have a look now and hopefully fix it once and for all!

@potiuk potiuk added the ready for maintainer review Set after triaging when all criteria pass. label May 27, 2026
@potiuk
potiuk marked this pull request as draft July 2, 2026 18:45
@potiuk
potiuk marked this pull request as ready for review July 2, 2026 22:14

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good fix, and sorry it sat this long. _fetch_bare_repo wraps its fetch in custom_environment(GIT_SSH_COMMAND=...) so the configured deploy key is used, but _fetch_submodules did not — so git submodule update ran without it and private submodules could not be pulled at all.

Copying the established pattern is the right call here rather than inventing a new one, and swapping self.bare_repo for self.repo is correct since submodules update in the working repo.

Since this has been open a while, I checked the pattern against current main rather than just against the branch: _fetch_bare_repo still uses exactly this nullcontext() / custom_environment shape, _fetch_submodules is still the unguarded two-liner, and nullcontext is already imported. So the fix still applies to the code as it stands today.

The tests cover both entry paths — initialize() with a pinned version and without, which routes through refresh() — and each asserts the exact call, so a regression would be caught rather than silently passing.

Rebased onto main before merging, as the branch had drifted a long way behind.


Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting

The git dag bundle did not use the private key to pull submodules.
This lead to issues when working with private submodules.
This commit uses the same key used to pull the dag bundle to pull the
submodules.
@potiuk
potiuk merged commit 600d998 into apache:main Aug 1, 2026
155 checks passed
@boring-cyborg

boring-cyborg Bot commented Aug 1, 2026

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready for maintainer review Set after triaging when all criteria pass.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants