Goroutine accumulation in DHT

[Kubuxu]

Version information:

0.4.3 - official binary

Description:

Usingsing Files API bug number of goroutines accumulate in the DHT SendMessage function as reported by @mattseh.
This affects the performance and memory requirements of go-ipfs as accumulating goroutines use memory.

Stack frequency:

  3341 sync.runtime_Semacquire(
    491 github.com/ipfs/go-ipfs/routing/dht.(*IpfsDHT).Provide(
    491 github.com/ipfs/go-ipfs/routing/dht.(*dhtQueryRunner).Run(
    406 gx/ipfs/QmQopLATEYMNg7dVqZRNDfeE2S1yKy8zrRh5xnYiuqeZBn/goprocess/context.CloseAfterContext.func1(
    399 gx/ipfs/QmZy2y8t9zQH2a1b8q2ZSLKp17ATuJoCNxxyMFG5qFExpt/go-net/context.propagateCancel.func1(
    399 gx/ipfs/QmQdnfvZQuhdT93LNc5bos52wAmdr3G2p6G8teLJMEN32P/go-libp2p-peerstore/queue.(*ChanQueue).process.func1(
    399 github.com/ipfs/go-ipfs/routing/dht.(*dhtQueryRunner).spawnWorkers(
    372 gx/ipfs/QmQopLATEYMNg7dVqZRNDfeE2S1yKy8zrRh5xnYiuqeZBn/goprocess.(*process).doClose(
    300 gx/ipfs/QmZ8MMKFwA95viWULoSYFZpA4kdFa8idmFSrP12YJwjjaL/yamux.(*Stream).Read(
    255 net.runtime_pollWait(
    248 gx/ipfs/QmZ8MMKFwA95viWULoSYFZpA4kdFa8idmFSrP12YJwjjaL/yamux.(*Session).AcceptStream(
    247 gx/ipfs/QmZ8MMKFwA95viWULoSYFZpA4kdFa8idmFSrP12YJwjjaL/yamux.(*Session).keepalive(
    247 github.com/ipfs/go-ipfs/exchange/bitswap.(*msgQueue).runQueue(
    245 gx/ipfs/QmZ8MMKFwA95viWULoSYFZpA4kdFa8idmFSrP12YJwjjaL/yamux.(*Session).send(
    106 gx/ipfs/QmX6DhWrpBB5NtadXmPSXYNdVvuLfJXoFNMvUMoVvP5UJa/go-context/io.(*ctxReader).Read(
     92 gx/ipfs/QmQopLATEYMNg7dVqZRNDfeE2S1yKy8zrRh5xnYiuqeZBn/goprocess.(*process).Close(
     67 github.com/ipfs/go-ipfs/routing/dht.(*messageSender).ctxReadMsg(
      8 github.com/ipfs/go-ipfs/exchange/bitswap.(*Bitswap).taskWorker(

Full goroutine stack: https://gist.github.com/mattseh/5f7756d2330fe0788938d17a43cc2c20/raw/b4bc594a66349e6bc1ec36cdd6adc0f5b97f1d81/gistfile1.txt

[matthewrobertbell]

By the way, the daemon was using ~1.3GB when this stack trace was recorded.

[Kubuxu]

It should be resolved in 0.4.4-dev. If you are having problems with it give the current master a try.

If it shows up on the 0.4.4-dev please create a new issue.