Skip to content

[SPARK-57447][SQL] Escape the index name in (H2|MySQL|Postgres)Dialect.indexExists#56508

Closed
dongjoon-hyun wants to merge 1 commit into
apache:masterfrom
dongjoon-hyun:SPARK-57447
Closed

[SPARK-57447][SQL] Escape the index name in (H2|MySQL|Postgres)Dialect.indexExists#56508
dongjoon-hyun wants to merge 1 commit into
apache:masterfrom
dongjoon-hyun:SPARK-57447

Conversation

@dongjoon-hyun

@dongjoon-hyun dongjoon-hyun commented Jun 14, 2026

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

indexExists in H2Dialect, MySQLDialect, and PostgresDialect embeds the
index name as a SQL string literal in its lookup query. This PR wraps the index
name with the existing JdbcDialect.escapeSql helper so that single quotes are
properly escaped.

Note that the scope of this PR is only index names. For table names or TABLE_SCHEMA, we are going to handle independently.

Why are the changes needed?

If an index name contains a single quote, the lookup query becomes malformed
(e.g. ... INDEX_NAME = 'a'b'). JdbcUtils.checkIfIndexExists swallows the
resulting exception and returns false, so an existing index whose name
contains a single quote is incorrectly reported as not existing.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Pass the CIs.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code

@dongjoon-hyun dongjoon-hyun changed the title [SPARK-57447] Escape the index name in (H2|MySQL|Postgres)Dialect.indexExists [SPARK-57447][SQL] Escape the index name in (H2|MySQL|Postgres)Dialect.indexExists Jun 14, 2026
@dongjoon-hyun

Copy link
Copy Markdown
Member Author

cc @huaxingao

@dongjoon-hyun

Copy link
Copy Markdown
Member Author

All tests passed.

@dongjoon-hyun

Copy link
Copy Markdown
Member Author

Could you review this PR when you have some time, please, @viirya ?

@viirya viirya left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix itself is correct — escapeSql doubles single quotes, the three call sites all interpolate indexName inside a '...' literal, and I confirmed JdbcUtils.checkIfIndexExists swallows the exception and returns false, so the false-negative described in the PR is real. The Mockito test is nice: it captures the executed SQL and asserts the escaped form, so all three dialects are covered in plain CI without a live DB.

My one concern is scope vs. the bug class. The same false-negative pattern still exists in these methods for the other interpolated literals:

  • H2Dialect.indexExists still has TABLE_SCHEMA = '${tableIdent.namespace().last}' and TABLE_NAME = '${tableIdent.name()}' unescaped.
  • PostgresDialect.indexExists still has tablename = '${tableIdent.name()}' unescaped.

A table named o'brien reproduces the identical malformed-query -> swallowed-exception -> false "index does not exist" behavior. The same raw-literal pattern also appears in listIndexes (H2Dialect and PostgresDialect build ... = '${tableIdent.name()}' the same way).

Could we either escape the table/schema names in these queries too, or note in the PR description that they're intentionally out of scope for a follow-up? Leaning toward fixing them here since it's just a few more escapeSql calls and avoids a near-identical bug report later. Not blocking — the change as-is is safe and correct.

@huaxingao huaxingao left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dongjoon-hyun

Copy link
Copy Markdown
Member Author

Thank you, @viirya and @huaxingao .

To @viirya , I revise the PR description by mentioning the scope is only index names and the table names and TABLE_SCHEMA will be handled independently.

Could we either escape the table/schema names in these queries too, or note in the PR description that they're intentionally out of scope for a follow-up?

dongjoon-hyun added a commit that referenced this pull request Jun 15, 2026
…ct.indexExists`

### What changes were proposed in this pull request?

`indexExists` in `H2Dialect`, `MySQLDialect`, and `PostgresDialect` embeds the
index name as a SQL string literal in its lookup query. This PR wraps the index
name with the existing `JdbcDialect.escapeSql` helper so that single quotes are
properly escaped.

Note that the scope of this PR is only index names. For table names or TABLE_SCHEMA, we are going to handle independently.

### Why are the changes needed?

If an index name contains a single quote, the lookup query becomes malformed
(e.g. `... INDEX_NAME = 'a'b'`). `JdbcUtils.checkIfIndexExists` swallows the
resulting exception and returns `false`, so an existing index whose name
contains a single quote is incorrectly reported as not existing.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Pass the CIs.

### Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code

Closes #56508 from dongjoon-hyun/SPARK-57447.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit 86db727)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
@dongjoon-hyun

Copy link
Copy Markdown
Member Author

Merged to master/4.x for now because we have Apache Spark 4.2.0 RC3 vote currently.

If RC3 fails, I will consider to cherry-pick this to branch-4.2 and older.

@dongjoon-hyun dongjoon-hyun deleted the SPARK-57447 branch June 15, 2026 03:38
dongjoon-hyun added a commit that referenced this pull request Jun 17, 2026
…ct.indexExists`

### What changes were proposed in this pull request?

`indexExists` in `H2Dialect`, `MySQLDialect`, and `PostgresDialect` embeds the
index name as a SQL string literal in its lookup query. This PR wraps the index
name with the existing `JdbcDialect.escapeSql` helper so that single quotes are
properly escaped.

Note that the scope of this PR is only index names. For table names or TABLE_SCHEMA, we are going to handle independently.

### Why are the changes needed?

If an index name contains a single quote, the lookup query becomes malformed
(e.g. `... INDEX_NAME = 'a'b'`). `JdbcUtils.checkIfIndexExists` swallows the
resulting exception and returns `false`, so an existing index whose name
contains a single quote is incorrectly reported as not existing.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Pass the CIs.

### Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code

Closes #56508 from dongjoon-hyun/SPARK-57447.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit 86db727)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit ab289d6)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
dongjoon-hyun added a commit that referenced this pull request Jun 17, 2026
…ct.indexExists`

`indexExists` in `H2Dialect`, `MySQLDialect`, and `PostgresDialect` embeds the
index name as a SQL string literal in its lookup query. This PR wraps the index
name with the existing `JdbcDialect.escapeSql` helper so that single quotes are
properly escaped.

Note that the scope of this PR is only index names. For table names or TABLE_SCHEMA, we are going to handle independently.

If an index name contains a single quote, the lookup query becomes malformed
(e.g. `... INDEX_NAME = 'a'b'`). `JdbcUtils.checkIfIndexExists` swallows the
resulting exception and returns `false`, so an existing index whose name
contains a single quote is incorrectly reported as not existing.

No.

Pass the CIs.

Generated-by: Claude Code

Closes #56508 from dongjoon-hyun/SPARK-57447.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit 86db727)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit ab289d6)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit 85a0972)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
dongjoon-hyun added a commit that referenced this pull request Jun 17, 2026
…ct.indexExists`

`indexExists` in `H2Dialect`, `MySQLDialect`, and `PostgresDialect` embeds the
index name as a SQL string literal in its lookup query. This PR wraps the index
name with the existing `JdbcDialect.escapeSql` helper so that single quotes are
properly escaped.

Note that the scope of this PR is only index names. For table names or TABLE_SCHEMA, we are going to handle independently.

If an index name contains a single quote, the lookup query becomes malformed
(e.g. `... INDEX_NAME = 'a'b'`). `JdbcUtils.checkIfIndexExists` swallows the
resulting exception and returns `false`, so an existing index whose name
contains a single quote is incorrectly reported as not existing.

No.

Pass the CIs.

Generated-by: Claude Code

Closes #56508 from dongjoon-hyun/SPARK-57447.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit 86db727)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit ab289d6)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit 85a0972)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit c697510)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
dongjoon-hyun added a commit that referenced this pull request Jun 17, 2026
…ct.indexExists`

`indexExists` in `H2Dialect`, `MySQLDialect`, and `PostgresDialect` embeds the
index name as a SQL string literal in its lookup query. This PR wraps the index
name with the existing `JdbcDialect.escapeSql` helper so that single quotes are
properly escaped.

Note that the scope of this PR is only index names. For table names or TABLE_SCHEMA, we are going to handle independently.

If an index name contains a single quote, the lookup query becomes malformed
(e.g. `... INDEX_NAME = 'a'b'`). `JdbcUtils.checkIfIndexExists` swallows the
resulting exception and returns `false`, so an existing index whose name
contains a single quote is incorrectly reported as not existing.

No.

Pass the CIs.

Generated-by: Claude Code

Closes #56508 from dongjoon-hyun/SPARK-57447.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit 86db727)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit ab289d6)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit 85a0972)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit c697510)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit da40fdb)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
iemejia pushed a commit to iemejia/spark that referenced this pull request Jun 17, 2026
…ct.indexExists`

### What changes were proposed in this pull request?

`indexExists` in `H2Dialect`, `MySQLDialect`, and `PostgresDialect` embeds the
index name as a SQL string literal in its lookup query. This PR wraps the index
name with the existing `JdbcDialect.escapeSql` helper so that single quotes are
properly escaped.

Note that the scope of this PR is only index names. For table names or TABLE_SCHEMA, we are going to handle independently.

### Why are the changes needed?

If an index name contains a single quote, the lookup query becomes malformed
(e.g. `... INDEX_NAME = 'a'b'`). `JdbcUtils.checkIfIndexExists` swallows the
resulting exception and returns `false`, so an existing index whose name
contains a single quote is incorrectly reported as not existing.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Pass the CIs.

### Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code

Closes apache#56508 from dongjoon-hyun/SPARK-57447.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants