feat(drive): add sync filtering controls#892
Closed
fangshuyu-768 wants to merge 4 commits into
Closed
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAdds a driveSyncFilter with CLI flags ChangesDrive sync filtering feature
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
added
domain/ccm
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (1)
shortcuts/drive/filter_test.go (1)
57-65: ⚡ Quick winAdd a nested-ancestor include case for wildcard patterns.
Please also assert that
MatchDir("docs/sub")is included forincludes: []string{"docs/**/*.md"}. Right now the test only checksdocs, which can miss nested-dir regressions in include-ancestor handling.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@shortcuts/drive/filter_test.go` around lines 57 - 65, Update the test TestDriveSyncFilterMatchDirIncludeAncestor to also assert that a nested descendant directory is treated as an included ancestor for the include pattern "docs/**/*.md": after creating filter := &driveSyncFilter{includes: []string{"docs/**/*.md"}}, add an assertion that filter.MatchDir("docs/sub").Included is true (similar to the existing check for "docs") while keeping the negative check for "vendor"; this ensures driveSyncFilter.MatchDir handles nested ancestors correctly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@shortcuts/drive/drive_status_test.go`:
- Around line 137-188: The test currently can pass from local-only results;
update TestDriveStatusFiltersLocalAndRemoteByExt to assert the remote stub was
actually exercised: after calling mountAndRunDrive, assert the download stub for
"/open-apis/drive/v1/files/tok_a/download" was invoked (use the httpmock
registry assertion API, e.g., reg.AssertCalled or reg.AssertAllCalled) and keep
the existing checks that "a.md" appears and "b.txt"/"c.txt" do not; this ensures
mountAndRunDrive and the remote GET stub (registered via reg.Register with token
"tok_a") were consumed.
In `@shortcuts/drive/filter.go`:
- Around line 213-221: The include-ancestor check currently only does raw prefix
matches on f.includes so patterns with wildcards like "docs/**/*.md" won't mark
"docs/sub" as included; update the logic in the code that iterates f.includes
(the block that returns driveFilterDecision with Reason
"flag_include_ancestor"/"flag_include_miss") to treat wildcard include patterns
as matching ancestor directories: for each pattern derive the directory portion
(e.g., strip a trailing "/**" and any file-glob component by taking the path
directory of the pattern), then check if that directory is a prefix of rel (with
a "/" suffix check) and if so return Included=true with Reason
"flag_include_ancestor"; preserve existing raw-prefix checks for non-wildcard
patterns and ensure you still return flag_include_miss when nothing matches.
In `@tests/cli_e2e/drive/drive_pull_dryrun_test.go`:
- Around line 70-97: The test TestDrive_PullDryRunAcceptsFilterFlags currently
only checks the exit code; update it to assert the dry-run rendered request
envelope in the command output (result.Stdout) so regressions in request
rendering are caught. After clie2e.RunCmd returns, add assertions that the
dry-run output contains the HTTP method and URL/path showing the folder token
"fldcnE2E001" and that the query or body includes the filter parameters (ext=md,
include=docs/**, exclude=docs/private/**); use the existing result assertion
helpers (e.g., result.AssertStdoutContains or similar) alongside the existing
result.AssertExitCode call to validate method, URL/folder token, and filter
params are present. Ensure you only read the dry-run output and do not perform
real API calls.
In `@tests/cli_e2e/drive/drive_push_dryrun_test.go`:
- Around line 73-100: The test TestDrive_PushDryRunAcceptsFilterFlags currently
only asserts exit code; enhance it to validate the dry-run request envelope
emitted by the CLI by capturing the command output (from clie2e.RunCmd result),
parsing the emitted JSON/request envelope, and asserting that fields match the
flags passed (e.g., local_dir="local", folder_token="fldcnE2E001", ext includes
["md","mdx"], include="docs/**", exclude="docs/private/**", and dry_run=true);
keep using --dry-run so no real API calls occur and add assertions on the parsed
envelope structure to fail the test if the request shape changes.
In `@tests/cli_e2e/drive/drive_status_dryrun_test.go`:
- Around line 73-100: The test TestDrive_StatusDryRunAcceptsFilterFlags
currently only asserts exit code; update it to also parse and validate the
dry-run envelope emitted by clie2e.RunCmd (inspect result stdout/stderr for the
dry-run JSON/payload) and assert the request structure contains the expected
command/action ("drive" / "+status" or equivalent field), localDir "local",
folderToken "fldcnE2E001", ext "md", include "docs/**", exclude
"docs/private/**" and that a dryRun flag is set; use the existing result object
returned from clie2e.RunCmd and add assertions that the parsed payload fields
match these values so renderer/request regressions are caught without calling
real APIs.
---
Nitpick comments:
In `@shortcuts/drive/filter_test.go`:
- Around line 57-65: Update the test TestDriveSyncFilterMatchDirIncludeAncestor
to also assert that a nested descendant directory is treated as an included
ancestor for the include pattern "docs/**/*.md": after creating filter :=
&driveSyncFilter{includes: []string{"docs/**/*.md"}}, add an assertion that
filter.MatchDir("docs/sub").Included is true (similar to the existing check for
"docs") while keeping the negative check for "vendor"; this ensures
driveSyncFilter.MatchDir handles nested ancestors correctly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 54350be0-467c-4340-b73c-bf964b471303
⛔ Files ignored due to path filters (1)
go.sumis excluded by!**/*.sum
📒 Files selected for processing (12)
go.modshortcuts/drive/drive_pull.goshortcuts/drive/drive_pull_test.goshortcuts/drive/drive_push.goshortcuts/drive/drive_push_test.goshortcuts/drive/drive_status.goshortcuts/drive/drive_status_test.goshortcuts/drive/filter.goshortcuts/drive/filter_test.gotests/cli_e2e/drive/drive_pull_dryrun_test.gotests/cli_e2e/drive/drive_push_dryrun_test.gotests/cli_e2e/drive/drive_status_dryrun_test.go
Support --ext, --include, --exclude, and .larkignore across drive +push, +pull, and +status so users can scope sync operations safely without deleting files outside the filtered view.
fangshuyu-768
force-pushed
the
feat/drive-filtering
branch
from
May 15, 2026 02:58
1b17d1f to
436b362
Compare
There was a problem hiding this comment.
♻️ Duplicate comments (1)
shortcuts/drive/drive_status_test.go (1)
298-305:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winAdd missing
reg.Verify(t)and remote-token assertion.The test can pass even if remote filtering is not exercised, because
"a.md"could come from the local walk alone. All other similar tests in this file callreg.Verify(t)to ensure stubs were consumed (lines 153, 250, 355, 443, 544, 575). Additionally, asserting the presence of"tok_a"would prove the remote stub was actually used.🔍 Suggested fix
out := stdout.String() if !strings.Contains(out, `"a.md"`) { t.Fatalf("expected a.md in status output: %s", out) } + if !strings.Contains(out, `"file_token": "tok_a"`) { + t.Fatalf("expected remote token tok_a in status output: %s", out) + } if strings.Contains(out, `"b.txt"`) || strings.Contains(out, `"c.txt"`) { t.Fatalf("filtered-out txt files should not appear in output: %s", out) } + + reg.Verify(t) }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@shortcuts/drive/drive_status_test.go` around lines 298 - 305, The test is missing verification that the remote stub was used and that all registry stubs were consumed; add a call to reg.Verify(t) at the end of the test and assert that stdout contains the remote-token string (e.g., "tok_a") in addition to checking for `"a.md"`, which proves the remote filtering ran. Locate the assertions near the end of this test (the block that inspects out := stdout.String()) and append the reg.Verify(t) call and a check that out contains "tok_a" so the remote stub is exercised and validated.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In `@shortcuts/drive/drive_status_test.go`:
- Around line 298-305: The test is missing verification that the remote stub was
used and that all registry stubs were consumed; add a call to reg.Verify(t) at
the end of the test and assert that stdout contains the remote-token string
(e.g., "tok_a") in addition to checking for `"a.md"`, which proves the remote
filtering ran. Locate the assertions near the end of this test (the block that
inspects out := stdout.String()) and append the reg.Verify(t) call and a check
that out contains "tok_a" so the remote stub is exercised and validated.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 0c491d34-0bb1-46c3-b46a-c75fa988260a
⛔ Files ignored due to path filters (1)
go.sumis excluded by!**/*.sum
📒 Files selected for processing (12)
go.modshortcuts/drive/drive_pull.goshortcuts/drive/drive_pull_test.goshortcuts/drive/drive_push.goshortcuts/drive/drive_push_test.goshortcuts/drive/drive_status.goshortcuts/drive/drive_status_test.goshortcuts/drive/filter.goshortcuts/drive/filter_test.gotests/cli_e2e/drive/drive_pull_dryrun_test.gotests/cli_e2e/drive/drive_push_dryrun_test.gotests/cli_e2e/drive/drive_status_dryrun_test.go
✅ Files skipped from review due to trivial changes (1)
- go.mod
🚧 Files skipped from review as they are similar to previous changes (10)
- tests/cli_e2e/drive/drive_pull_dryrun_test.go
- shortcuts/drive/filter_test.go
- shortcuts/drive/drive_status.go
- shortcuts/drive/drive_pull_test.go
- shortcuts/drive/drive_push.go
- shortcuts/drive/filter.go
- shortcuts/drive/drive_pull.go
- shortcuts/drive/drive_push_test.go
- tests/cli_e2e/drive/drive_push_dryrun_test.go
- tests/cli_e2e/drive/drive_status_dryrun_test.go
🚀 PR Preview Install Guide🧰 CLI updatenpm i -g https://pkg.pr.new/larksuite/cli/@larksuite/cli@8f3c69869c7175a280cc6582805a572075564ae3🧩 Skill updatenpx skills add larksuite/cli#feat/drive-filtering -y -g |
- Fix MatchDir ancestor logic for wildcard include patterns by
extracting concrete directory prefix (e.g. "docs/" from
"docs/**/*.md") so intermediate dirs like "docs/sub" are
correctly included
- Add docstrings to all filter.go functions/types to meet 80%
docstring coverage CI threshold
- Strengthen E2E dry-run filter tests with envelope assertions
(method, URL, folder_token) instead of exit-code-only checks
- Add reg.Verify(t) and remote token assertion to
TestDriveStatusFiltersLocalAndRemoteByExt
- Add nested ancestor include test case (MatchDir("docs/sub")
for pattern "docs/**/*.md")
- Restore accidentally deleted blank line in
TestDrive_StatusDryRunQuick
- Remove unused filterDriveStatusLocalHashes dead code
- Run go mod tidy to promote doublestar to direct dependency
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@shortcuts/drive/filter.go`:
- Around line 229-259: Move the "ancestor include" logic so it runs before the
ignore/builtin-exclude checks in MatchDir when any f.includes are present: if
len(f.includes) > 0, perform the loop that checks strings.HasPrefix(pattern,
prefix) and driveConcreteDirPrefix(pattern) first and return
driveFilterDecision{Included:true, Reason:"flag_include_ancestor"} when matched,
and only after that run matchedAny(rel, f.ignoreRules) and matchedAny(rel,
f.builtinExcludes); update references in MatchDir to ensure f.includes take
precedence over f.ignoreRules and f.builtinExcludes while preserving the
existing fallback returns (flag_include_miss, ignore_file, builtin) and keeping
driveConcreteDirPrefix and the existing Reason strings unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: e4c81969-d721-4989-ac6e-d73ede4d1d85
📒 Files selected for processing (7)
go.modshortcuts/drive/drive_status_test.goshortcuts/drive/filter.goshortcuts/drive/filter_test.gotests/cli_e2e/drive/drive_pull_dryrun_test.gotests/cli_e2e/drive/drive_push_dryrun_test.gotests/cli_e2e/drive/drive_status_dryrun_test.go
✅ Files skipped from review due to trivial changes (1)
- go.mod
🚧 Files skipped from review as they are similar to previous changes (3)
- shortcuts/drive/filter_test.go
- tests/cli_e2e/drive/drive_status_dryrun_test.go
- tests/cli_e2e/drive/drive_push_dryrun_test.go
Comment on lines
+229
to
+259
| if matchedAny(rel, f.ignoreRules) { | ||
| return driveFilterDecision{Included: false, Reason: "ignore_file"} | ||
| } | ||
| if matchedAny(rel, f.builtinExcludes) { | ||
| return driveFilterDecision{Included: false, Reason: "builtin"} | ||
| } | ||
| if len(f.includes) > 0 { | ||
| prefix := rel + "/" | ||
| for _, pattern := range f.includes { | ||
| // Direct prefix match: pattern starts with "docs/sub/". | ||
| if strings.HasPrefix(pattern, prefix) { | ||
| return driveFilterDecision{Included: true, Reason: "flag_include_ancestor"} | ||
| } | ||
| // Extract the leading non-wildcard directory prefix from the | ||
| // pattern. For "docs/**/*.md" the concrete prefix is "docs/", | ||
| // so "docs" and "docs/sub" are ancestors. For "src/lib/*.go" | ||
| // the concrete prefix is "src/lib/". | ||
| concretePrefix := driveConcreteDirPrefix(pattern) | ||
| if concretePrefix == "" { | ||
| continue | ||
| } | ||
| // rel is an ancestor if its path falls within the concrete | ||
| // prefix (e.g. "docs" is a prefix of "docs/") or the | ||
| // concrete prefix falls within rel (e.g. "docs/" is a | ||
| // prefix of "docs/sub/"). | ||
| if strings.HasPrefix(prefix, concretePrefix) || strings.HasPrefix(concretePrefix, prefix) { | ||
| return driveFilterDecision{Included: true, Reason: "flag_include_ancestor"} | ||
| } | ||
| } | ||
| return driveFilterDecision{Included: false, Reason: "flag_include_miss"} | ||
| } |
There was a problem hiding this comment.
Ancestor include logic is evaluated after .larkignore, violating stated precedence.
The struct docstring (line 45-46) states "CLI flags (--exclude > --include > --ext) > .larkignore", but MatchDir checks ignore rules (line 229) and builtin excludes (line 232) before the ancestor include logic (line 235). This causes .larkignore to block directory traversal even when --include explicitly targets descendants.
Example: with .larkignore containing docs and --include "docs/api/*.md", MatchDir("docs") returns excluded at line 230 because the ignore check precedes the ancestor check—local walk never descends into docs, so files under docs/api/ are never discovered.
Proposed fix: move ancestor check before ignore/builtin checks
func (f *driveSyncFilter) MatchDir(rel string) driveFilterDecision {
rel = filepath.ToSlash(rel)
if rel == "" || rel == "." {
return driveFilterDecision{Included: true, Reason: "default"}
}
if matchedAny(rel, f.excludes) {
return driveFilterDecision{Included: false, Reason: "flag_exclude"}
}
- if len(f.includes) > 0 && matchedAny(rel, f.includes) {
- return driveFilterDecision{Included: true, Reason: "flag_include"}
- }
- if matchedAny(rel, f.ignoreRules) {
- return driveFilterDecision{Included: false, Reason: "ignore_file"}
- }
- if matchedAny(rel, f.builtinExcludes) {
- return driveFilterDecision{Included: false, Reason: "builtin"}
- }
if len(f.includes) > 0 {
+ if matchedAny(rel, f.includes) {
+ return driveFilterDecision{Included: true, Reason: "flag_include"}
+ }
prefix := rel + "/"
for _, pattern := range f.includes {
- // Direct prefix match: pattern starts with "docs/sub/".
if strings.HasPrefix(pattern, prefix) {
return driveFilterDecision{Included: true, Reason: "flag_include_ancestor"}
}
- // Extract the leading non-wildcard directory prefix from the
- // pattern. For "docs/**/*.md" the concrete prefix is "docs/",
- // so "docs" and "docs/sub" are ancestors. For "src/lib/*.go"
- // the concrete prefix is "src/lib/".
concretePrefix := driveConcreteDirPrefix(pattern)
if concretePrefix == "" {
continue
}
- // rel is an ancestor if its path falls within the concrete
- // prefix (e.g. "docs" is a prefix of "docs/") or the
- // concrete prefix falls within rel (e.g. "docs/" is a
- // prefix of "docs/sub/").
if strings.HasPrefix(prefix, concretePrefix) || strings.HasPrefix(concretePrefix, prefix) {
return driveFilterDecision{Included: true, Reason: "flag_include_ancestor"}
}
}
return driveFilterDecision{Included: false, Reason: "flag_include_miss"}
}
+ if matchedAny(rel, f.ignoreRules) {
+ return driveFilterDecision{Included: false, Reason: "ignore_file"}
+ }
+ if matchedAny(rel, f.builtinExcludes) {
+ return driveFilterDecision{Included: false, Reason: "builtin"}
+ }
return driveFilterDecision{Included: true, Reason: "default"}
}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@shortcuts/drive/filter.go` around lines 229 - 259, Move the "ancestor
include" logic so it runs before the ignore/builtin-exclude checks in MatchDir
when any f.includes are present: if len(f.includes) > 0, perform the loop that
checks strings.HasPrefix(pattern, prefix) and driveConcreteDirPrefix(pattern)
first and return driveFilterDecision{Included:true,
Reason:"flag_include_ancestor"} when matched, and only after that run
matchedAny(rel, f.ignoreRules) and matchedAny(rel, f.builtinExcludes); update
references in MatchDir to ensure f.includes take precedence over f.ignoreRules
and f.builtinExcludes while preserving the existing fallback returns
(flag_include_miss, ignore_file, builtin) and keeping driveConcreteDirPrefix and
the existing Reason strings unchanged.
Codecov Report❌ Patch coverage is ❌ Your patch check has failed because the patch coverage (59.49%) is below the target coverage (60.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #892 +/- ##
==========================================
- Coverage 66.40% 66.37% -0.04%
==========================================
Files 560 561 +1
Lines 51548 51784 +236
==========================================
+ Hits 34230 34370 +140
- Misses 14455 14513 +58
- Partials 2863 2901 +38 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Cover normalizeDrivePattern, normalizeDrivePatterns, parseDriveIgnore error paths, MatchFile ignore/builtin/default branches, MatchDir exclude/ignore/builtin/default branches, driveConcreteDirPrefix, driveMatchPattern, filterDriveRemoteEntries, filterDrivePushLocalView, sortedDriveDirs, filterDriveStatusLocalFiles, filterDrivePullLocalAbsPaths.
Collaborator
Author
|
关闭此 PR，所有修复已合入 feat/drive-filtering 分支。 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add reusable filtering controls for drive sync flows so users can scope
+push,+pull, and+statusby extension, glob include/exclude rules, and a root.larkignorefile. This keeps sync operations aligned with the requested subset and prevents delete phases from touching files outside the filtered view.Changes
.larkignoreparsing--ext/--include/--excludeinto drive+push,+pull, and+statuswith consistent precedence--delete-local/--delete-remoteto the filtered sync view and add unit + dry-run E2E coverageTest Plan
go test ./shortcuts/drive)lark xxxcommand works as expected (go build -o lark-cli .andLARK_CLI_BIN=/Users/bytedance/work/larksuite-cli/lark-cli go test ./tests/cli_e2e/drive -run 'TestDrive_(Push|Pull|Status)')Related Issues
Summary by CodeRabbit
New Features
Tests