[Bug] PathUtils.equalsIgnoreSchemeIfOneIsS3 path comparison is inconsistent between same-scheme and cross-scheme branches

[LuciferYang]

Search before asking

• I had searched in the issues and found no similar issues.

Version

master (fe-foundation, org.apache.doris.foundation.util.PathUtils).

What's Wrong?

PathUtils.equalsIgnoreSchemeIfOneIsS3(p1, p2) compares two storage-location URIs treating the s3 scheme as interchangeable with other object-store schemes. Its two branches used inconsistent rules:

• Same scheme → p1.equalsIgnoreCase(p2): full-string, case-insensitive, trailing slash significant.
• Cross-scheme (one is s3) → compares normalize(authority)/normalize(path) with Objects.equals: case-sensitive, trailing slash stripped.

Consequences:

1. The result for one URI depends on the other URI's scheme. For example s3://bucket/path/ vs s3://bucket/path are unequal (same-scheme branch), but s3://bucket/path/ vs cos://bucket/path are equal (cross-scheme branch).
2. The same-scheme branch ignores case for the whole string, so it can wrongly equate case-sensitive S3 object keys (s3://b/A == s3://b/a).

The only caller is HMSTransaction.prepareInsertExistingTable (fe-core), which uses this to decide whether a Hive commit needs a rename — so inconsistent equality can lead to an incorrect rename decision.

What You Expected?

A single, consistent rule regardless of whether the two schemes match: compare authority + path (scheme ignored when equal or when one side is s3), with trailing slashes insignificant and the comparison case-sensitive (object-storage keys are case-sensitive).

How to Reproduce?

// same pair, different "other" scheme -> different answer (inconsistent):
PathUtils.equalsIgnoreSchemeIfOneIsS3("s3://bucket/path/", "s3://bucket/path");   // false
PathUtils.equalsIgnoreSchemeIfOneIsS3("s3://bucket/path/", "cos://bucket/path");  // true

// same-scheme comparison wrongly ignores case:
PathUtils.equalsIgnoreSchemeIfOneIsS3("s3://bucket/A", "s3://bucket/a");          // true (should be false)

Anything Else?

Fix proposed in the linked PR. It also hardens several edge cases surfaced during review (opaque URIs, percent-encoded slashes, triple-slash / network-path forms) by falling back to exact string comparison for inputs that are malformed for object storage.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

[zclllyybb]

Breakwater-GitHub-Analysis-Slot: slot_bc1fd4b9373a
This content is generated by AI for reference only.

Initial triage: this looks like a real consistency bug in current master.

I refreshed apache/doris upstream/master to e5f3badd0109e312167f242df5aa53adb86806d8 and checked the referenced code path. In fe/fe-foundation/src/main/java/org/apache/doris/foundation/util/PathUtils.java, equalsIgnoreSchemeIfOneIsS3 currently has two different equality contracts:

• Same-scheme URIs go through a full-string equalsIgnoreCase comparison, so object key case is ignored and trailing slashes remain significant.
• Cross-scheme URIs where either side is s3 compare normalized authority and path with Objects.equals, so the comparison is case-sensitive and strips trailing slashes.

That matches the issue's reproduction. It also matters to production behavior: the production use I found is HMSTransaction.prepareInsertExistingTable in fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HMSTransaction.java, where the result directly drives needRename = !PathUtils.equalsIgnoreSchemeIfOneIsS3(targetPath, writePath). Therefore:

• Same-scheme trailing-slash-only differences can trigger an unnecessary rename even though the cross-scheme s3 path treats the same location as equal.
• Same-scheme case-only differences such as s3://bucket/A versus s3://bucket/a can be incorrectly treated as equal, which is unsafe for object-storage keys.

I also found the linked public fix PR: #64768. It is currently open as a draft, based on the same master head, and its patch changes the utility to use one structural comparison rule for same-scheme and s3 cross-scheme cases. The added PathUtilsTest cases cover the reported trailing-slash consistency issue, case-sensitive path/authority comparisons, malformed object-storage URI forms, encoded slashes, query/fragment distinctions, and s3a/s3n behavior.

Suggested next steps:

1. Review PR [fix](fs) Make PathUtils.equalsIgnoreSchemeIfOneIsS3 path comparison consistent #64768 as the likely direct fix for this issue.
2. Keep the utility-level regression tests for both reported examples: same-scheme trailing slash and same-scheme case-only path differences.
3. If maintainers want stronger caller coverage, add a narrow FE test around the Hive insert path or rename-decision input pair to ensure targetPath and writePath follow the unified location-equality rule.

Missing information is not blocking the code-level conclusion here because the logic bug is visible from the current source and the issue includes direct method-level reproductions. A real-world incident assessment would still need the exact table location, write path, storage scheme pair, Doris build SHA, and FE commit/rename logs.