feat(simlibp2p): Simulated libp2p Networks#3262
Conversation
|
hah these tests will be infinitely flaky in CI. but should be perfect once we can use go 1.24's synctest here (as a buildtag, then another command for just tests with those build tags, just as we do with nocover) |
|
Fixed flaky tests by using synctest. Set an extra CI step to run these synctest tests, like we do for NoCover. |
|
cc @Wondertan since you are (were?) a user of mocknet. I'm curious if you would find this useful. It's more realistic than mocknet (you can set bandwidth limits and latencies) and you exercise all the same quic code you would in production. Combining this with synctest means you should be able to run large simulated networks accurately. |
|
Hey @MarcoPolo, thanks for the ping. Yes, we rely on mocknet and I was hoping that in memory transport is gonna replace the need for it. I guess the simulated network has even more capabilities with bandwidth/latency tunables as its a lower layer construct, but I wouldnt say this is something we need atm. Although, definetly useful Afaiu, the synctest would help with flakes for both. |
Correct |
|
I've been playing with this a lot, and I think it is quite useful. Worth merging |
sukunrt
left a comment
sukunrt
left a comment
There was a problem hiding this comment.
Partial review. Mostly looks good except a few nits.
| // When we pass in our own conn manager, we need to close it manually (??) | ||
| // TODO: this seems like a bug |
There was a problem hiding this comment.
Is this true? Can you open an issue?
There was a problem hiding this comment.
True. Not sure it's a bug though: https://github.com/libp2p/go-libp2p/blob/master/config/config.go#L412
maybe the lifecycle should be handled here (?)
| return m, nil | ||
| }), | ||
| quicreuse.OverrideListenUDP(func(network string, address *net.UDPAddr) (net.PacketConn, error) { | ||
| m.ip.Store(&address.IP) |
There was a problem hiding this comment.
Does this mean we will only dial out from the last Listened IP?
| h, err = libp2p.New( | ||
| libp2p.ListenAddrStrings(addr), | ||
| QUICSimConnSimpleNet(nw, l.LinkSettings, quicReuseOpts...), | ||
| libp2p.DisableIdentifyAddressDiscovery(), |
There was a problem hiding this comment.
Why do we need to disable address discovery?
There was a problem hiding this comment.
We likely don't need to. This was probably me reducing the number of goroutines per peer.
There was a problem hiding this comment.
Just tried it. We do need to disable it because something in that service stalls it.
Going to leave it disabled for now because there are a lot of changes happening there. I'll leave it as a todo to come back to it later.
MarcoPolo
changed the title
This replaces the custom rate limiter implementation with the standard library's rate.Limiter
MarcoPolo
force-pushed
the
marco/simnet
branch
from
cce272b to
5556df7
Compare
3 participants
This adds a new Simulated Link to extend the Simulated UDP Packet conn. Simulated links can emulate bandwidth and latency over a connection, and can be used to connect two simulated conns together.
This also adds a simulated network helper to easily create a libp2p network and route the various links.
As-is, this works as a simple real-time simulator that may lose precision as computation gets bigger. The real benefit is when this is combined with Go 1.24's new
testing/synctestpackage that allows the simulated network to run as a discrete event simulator. For this use case, only QUIC is supported.This should serve to replace mocknet with a more accurate environment.
In draft until I clean up the tests a bit and use it a bit more.