Skip to content

sweep: slow reprovide alerts#11021

Merged
lidel merged 31 commits intomasterfrom
reprovide-alert
Oct 24, 2025
Merged

sweep: slow reprovide alerts#11021
lidel merged 31 commits intomasterfrom
reprovide-alert

Conversation

@guillaumemichel
Copy link
Copy Markdown
Contributor

@guillaumemichel guillaumemichel commented Oct 9, 2025

Builds on top of #11019 (because we need to access prov.Stats().

Regular polls, checking the reprovide queue size, and printing a warning if the queue keeps increasing over a few intervals (>30min) and all workers are busy.

Warning

#11019 must be merged first.

guillaumemichel and others added 8 commits October 15, 2025 13:47
- add extractSweepingProvider() helper to reduce nested type switching
- extract lowWorkerThreshold constant for worker availability check
- fix --lan error handling to work with buffered providers
- prevent panic when both columns are empty
- fix column alignment with UTF-8 characters
- only track col0MaxWidth for first column (as intended)
- test basic functionality, flags, JSON output
- test legacy provider behavior
- test integration with content scheduling
- test disabled provider configurations
- add parseSweepStats helper with t.Helper()
- update tagline to "Control and monitor content providing"
- simplify help descriptions
- make error messages more consistent
- update tests to match new error messages
```
Next reprovide at:
Next prefix:
```
updated to:
```
Next region prefix:
Next region reprovide:
```
Enhance documentation for the Provide system to better explain how provider
records work and the differences between sweep and legacy modes.

Changes to docs/config.md:
- Provide section: add clear explanation of provider records and their role
- Provide.DHT: add provider record lifecycle and two provider systems overview
- Provide.DHT.Interval: explain relationship to expiration, contrast sweep vs legacy behavior
- Provide.DHT.SweepEnabled: rewrite to explain legacy problem, sweep solution, and efficiency gains
- Monitoring section: prioritize command-line tools (ipfs provide stat) before Prometheus

Changes to core/commands/provide.go:
- ipfs provide stat help: add explanation of provider records, TTL expiration, and how sweep batching works

Changes to docs/changelogs/v0.39.md:
- Add context about why stats matter for monitoring provider health
- Emphasize real-time monitoring workflow with watch command
- Explain what users can observe (rates, queues, worker availability)
Merged latest changes from origin/master while preserving go-libp2p-kad-dht v0.35.2-0.20251009080632-2c5b3769ca70 from this branch.
Copy link
Copy Markdown
Member

@lidel lidel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, just fix the indentation (probably prefer spaces as they render the same for everyone)

Base automatically changed from provide-stats to master October 23, 2025 18:29
@lidel lidel self-assigned this Oct 24, 2025
lidel added 2 commits October 24, 2025 22:21
resolved conflicts:
- go.mod/go.sum: prefer versions from master (PR #11019 merged first)
- docs/config.md: prefer clearer wording from master
- docs/changelogs/v0.39.md: keep reprovide-alert warning feature

updates from make mod_tidy:
- go-log/v2: v2.8.1 -> v2.8.2
- go-dsqueue: v0.0.5 -> v0.1.0
- extract reprovideAlertPollInterval and consecutiveAlertsThreshold constants
- add extractSweepingProvider helper matching provide.go style
- add defer ticker.Stop() for proper cleanup
- improve variable grouping and documentation
@lidel
Copy link
Copy Markdown
Member

lidel commented Oct 24, 2025

Resolved conflicts + small cleanup in 3433609, lgtm, merging.

@lidel lidel merged commit 5d4c40e into master Oct 24, 2025
16 checks passed
@lidel lidel deleted the reprovide-alert branch October 24, 2025 21:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants