Log SQL query on its own line in BigQueryInsertJobOperator#69386
Merged
shahar1 merged 1 commit intoJul 6, 2026
Merged
Conversation
Co-authored-by: Lucho Formisano <22617963+LuchoFormisano@users.noreply.github.com>
Contributor
Author
|
Verified the change end-to-end against a real GCP project. Same DAG with a multiline query, run twice: first with the provider built from main (before) and then from this PR's head (eb53faf, after). Before: the SQL is only visible inside the single-line config dict with escaped \n. After: the new SQL query: line renders the query on real lines, readable and copy-pasteable straight from the task log. Job inserted and run succeeded in both cases — screenshots attached.
|
shahar1
approved these changes
Jul 4, 2026
Contributor
|
CC: @VladaZakharova @MaksYermak , WDYT? |
VladaZakharova
approved these changes
Jul 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



When
BigQueryInsertJobOperatorstarts, it logs the whole jobconfigurationdict in a single line. For query jobs this makes the SQL essentially unreadable in the task log — the query shows up embedded in the dict repr with escaped\nsequences:This change logs the SQL on its own log line (reusing the operator's existing
sqlproperty) so newlines render naturally and the query can be read — and copy-pasted into the BigQuery console — directly from the log:This restores a behaviour Airflow used to have rather than adding something new:
BigQueryExecuteQueryOperatorlogged the plain SQL string directly —self.log.info("Executing: %s", self.sql)— until the deprecated operator was removed in #43953, leaving the configuration-dict dump as the only trace of the query in the logs. This also matches how sibling operators log SQL today (e.g.BigQueryCheckOperator'sExecuting SQL check: %s). The stray trailing quote in the current log line is fixed along the way.Readable SQL in the log matters most after a failure or when logs are consumed remotely (CloudWatch/Stackdriver/grep), where the UI's rendered-template view isn't available. For non-query job types (load/copy/extract) the behaviour is unchanged.
No test added: the change is log-output-only, and the project testing standards discourage asserting on log output.
Verification performed:
BigQueryInsertJobOperatorunit tests pass in Breeze; mypy and static checks clean.example_bigquery_value_check.pysystem test end-to-end against a real GCP project: the Dag passed (1 passed in 188s), and the task log shows the newSQL query:line rendered as plain text.Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Fable 5) following the guidelines