Background
We've been talking about setting up nightly CI to keep things healthy. The idea is that foc-localnet could help with the first leg of tests—basically sanity checks to make sure the local network can at least start cold. That part makes sense and we can extend it later.
But before we commit to an approach, there are some fundamental questions we need to answer as a team.
Key Questions
1. What Does "Healthy" Actually Mean?
Here's the issue: foc-localnet pulls together multiple repos (curio, lotus, synapse-sdk, filecoin-services, etc.). For any test run, we need to explicitly say which commit/branch/tag of each repo we're testing.
So what commits should nightly pick?
- Do we always test the latest
main of everything?
- Do we pin to specific "known good" versions?
- Something else?
2. Cross-Repo Health is Complicated
This is the trickier part: a commit that's healthy in isolation might break when combined with other repos.
For example:
- Commit
5bd112ef... might be perfectly fine for lotus standalone
- But could break the entire setup when combined with a specific curio version
When cross-repo changes are needed, how do we handle that? Most significant changes need coordinated PRs across multiple repos to be stable. How does our nightly strategy account for this?
3. Where Should Test Scenarios Live?
I've shared my thoughts before that foc-localnet should just be a utility for starting/stopping clusters, not a full testing framework.
If we're building test scenarios for nightly, where do they belong?
- In synapse-sdk?
- A separate testing repo?
- Somewhere else?
What We Need
Before we design the nightly CI, we need alignment on:
- How we define the "healthy point" to test across multiple repos
- How we handle cross-repo dependencies
- Where testing scenarios should live
Thoughts?
Background
We've been talking about setting up nightly CI to keep things healthy. The idea is that
foc-localnetcould help with the first leg of tests—basically sanity checks to make sure the local network can at least start cold. That part makes sense and we can extend it later.But before we commit to an approach, there are some fundamental questions we need to answer as a team.
Key Questions
1. What Does "Healthy" Actually Mean?
Here's the issue:
foc-localnetpulls together multiple repos (curio, lotus, synapse-sdk, filecoin-services, etc.). For any test run, we need to explicitly say which commit/branch/tag of each repo we're testing.So what commits should nightly pick?
mainof everything?2. Cross-Repo Health is Complicated
This is the trickier part: a commit that's healthy in isolation might break when combined with other repos.
For example:
5bd112ef...might be perfectly fine for lotus standaloneWhen cross-repo changes are needed, how do we handle that? Most significant changes need coordinated PRs across multiple repos to be stable. How does our nightly strategy account for this?
3. Where Should Test Scenarios Live?
I've shared my thoughts before that
foc-localnetshould just be a utility for starting/stopping clusters, not a full testing framework.If we're building test scenarios for nightly, where do they belong?
What We Need
Before we design the nightly CI, we need alignment on:
Thoughts?