[SPARK-57354][SQL] Add ignoredPathSegmentRegex data source option and config#56374

Closed

CalvQ wants to merge 6 commits into

apache:masterfrom

CalvQ:list-hidden-files-option

Commits on Jun 18, 2026

[SPARK-57354][SQL] Add listHiddenFiles data source option
Calvin Qin
committed
update tests and docs, remove dead filestream codepath
Calvin Qin
committed
minor fix, pass bool through shouldFilterOut* functions
Calvin Qin
committed
refactor to ignoredPathSegmentRegex and address feedback
Calvin Qin
committed
Address review feedback: centralize regex compile, empty disables filter, version 4.3.0
Calvin Qin
committed

Commits on Jun 23, 2026

Fix scalastyle line-length violations (100-char limit)
Calvin Qin
committed