feat(local.file_match, loki.source.file): Match multiple files using doublestar {...} expressions

[ptodev]

Pull Request Details

Upgrading to doublestar/v4. The new version of the library is more performant and has the new {...} matching syntax.

However, it does have a few breaking changes. The breaking change which affects us the most is not documented - apparently the library is now case-sensitive by default even on Windows.

Issue(s) fixed by this Pull Request

Fixes #4423

Notes to the Reviewer

I checked the tests on Windows by removing the case sensitivity glob option and they indeed failed:

PR Checklist

• Documentation added
• Tests updated
• Config converters updated

[github-actions]

🔍 Dependency Review

github.com/bmatcuk/doublestar v1.3.4 -> v4.9.1 — ❌ Changes Needed

Summary

• This is a major upgrade to the v4 module (new module path) with API and behavior changes.
• Key changes you must make:
  • Replace uses of doublestar.Glob with doublestar/v4.FilepathGlob (optionally supply options like case-insensitivity on Windows).
  • Normalize patterns/paths for PathMatch to use OS-specific separators (e.g., filepath.FromSlash for exclusion patterns).
  • If you need case-insensitive globbing on Windows, use WithCaseInsensitive() with FilepathGlob (v4 adds option-based configuration).
  • Optionally add a thin abstraction for platform-dependent behavior (as done here with internal/util/glob).

Why

• v4 introduces an options-based API for globbing and formalizes platform-sensitive behavior (e.g., case-insensitivity on Windows).
• v4’s FilepathGlob replaces direct usage of Glob to better reflect OS file path semantics; it also accepts options such as WithCaseInsensitive().

Changelog highlights (v1.3.4 → v4.9.1)

• New options-based API (e.g., FilepathGlob(pattern string, opts ...GlobOption)) for controlling behavior like case sensitivity.
• Improvements and clarifications to separator handling and OS-specific semantics.
• Pattern features such as brace expansion {a,b,c} are supported and documented.
• Backward-incompatible changes warrant replacing old calls and normalizing separators where appropriate.

Code changes to apply

• Replace Glob with FilepathGlob:

- matches, err := doublestar.Glob(targetPath)
+ matches, err := doublestar.FilepathGlob(targetPath)

• Ensure exclude patterns use OS-specific separators for PathMatch:

-import "github.com/bmatcuk/doublestar"
+import (
+  "path/filepath"
+  "github.com/bmatcuk/doublestar/v4"
+)

- if match, _ := doublestar.PathMatch(exclude, m); match {
+ if match, _ := doublestar.PathMatch(filepath.FromSlash(exclude), m); match {
    continue
 }

• Enable case-insensitive globbing on Windows (v4 option):

- matches, err := doublestar.FilepathGlob(pattern)
+ matches, err := doublestar.FilepathGlob(pattern, doublestar.WithCaseInsensitive())

• Optional: Introduce a small abstraction to encapsulate platform behavior (matches what this PR does):

// internal/util/glob/glob.go
type Globber interface {
  FilepathGlob(pattern string) ([]string, error)
  PathMatch(pattern, path string) (bool, error)
}

// internal/util/glob/glob_default.go (Unix – case-sensitive)
func NewGlobber() Globber { return &defaultGlobber{} }
func (g *defaultGlobber) FilepathGlob(p string) ([]string, error) { return doublestar.FilepathGlob(p) }
func (g *defaultGlobber) PathMatch(pat, path string) (bool, error) { return doublestar.PathMatch(pat, path) }

// internal/util/glob/glob_windows.go (Windows – case-insensitive glob)
func NewGlobber() Globber { return &windowsGlobber{} }
func (g *windowsGlobber) FilepathGlob(p string) ([]string, error) {
  return doublestar.FilepathGlob(p, doublestar.WithCaseInsensitive())
}
func (g *windowsGlobber) PathMatch(pat, path string) (bool, error) {
  return doublestar.PathMatch(pat, path)
}

• Refactor call sites to use the abstraction (illustrative):

- matches, err := doublestar.Glob(w.getPath())
+ matches, err := w.globber.FilepathGlob(w.getPath())

- if match, _ := doublestar.PathMatch(exclude, m); match {
+ if match, _ := w.globber.PathMatch(filepath.FromSlash(exclude), m); match {
    continue
 }

Behavioral notes and tests to (re)confirm

• Case sensitivity:
  • Unix: case-sensitive matching (e.g., *.LOG will NOT match test.log).
  • Windows: case-insensitive matching recommended via WithCaseInsensitive() for FilepathGlob.
• Brace expansion:
  • {a,b,c} patterns (e.g., /*.{log,txt,json} and /var/log/{nginx,apache}/*.log) are supported; add tests and docs.
• Separator normalization:
  • Use filepath.FromSlash for patterns you persist with / to ensure correct behavior on Windows.

References (click to view)

• Relevant APIs in v4:
  • FilepathGlob(pattern string, opts ...GlobOption) ([]string, error)
  • PathMatch(pattern, name string) (bool, error)
• See: bmatcuk/doublestar README and release notes for v4 for details on options and semantics.

Notes

• No other dependency changes in go.mod require review here.
• This PR already applies the necessary code and test updates (glob abstraction, case-sensitivity tests, and docs for brace expansion).

[github-actions]

💻 Deploy preview available (chore: Upgrade to doublestar/v4):

• https://deploy-preview-alloy-5470-zb444pucvq-vp.a.run.app/docs/alloy/latest/

[github-actions]

💻 Deploy preview deleted (feat(local.file_match, loki.source.file): Match multiple files using doublestar {...} expressions).

[kalleep]

Nice, great work. Just a few comments

[kalleep]

Should we move filepath.FromSlash usage into globber(s)?

[ptodev]

I'm not sure. The reason I left it like this is so that it's more flexible if someone doesn't want the FromSlash in the future.

[x1unix]

Why an interface is required here? Is it necessary to do tests/mocks?

If the iface isn't strictly necessary, would suggest having a wrapper func that checks runtime.GOOS for case sensitivity option.

Downside of having separate build-constrained files is that syntax errors/linter/formatters may not run locally against build-constrained files for windows.

[ptodev]

It feels more defensive than using the doublestar functions directly.

If we use the functions directly and only switch to swapper funcs wherever is needed, then it would be very easy to mistakenly call the wrong function in case the code is refactored or extended.

[clayton-cornell]

Docs are fine as-is

[kalleep]

LGTM, one nit about the naming of glob_default but I don't feel too strongly about that one