swarm/test: Add integration test for 'swarm up'

[lmars]

This is a proposal for adding swarm "integration tests" which use a compiled swarm binary to boot clusters and upload files, and then make assertions against the HTTP and IPC APIs.

This helps us write tests covering the desired behaviour of swarm from the user's point of view (e.g. what does swarm up file.txt actually do, see #14342).

This is as discussed in this weeks swarm standup.

/cc @zelig

[lmars]

I had to update build/ci.go to make sure GOBIN is in PATH, and now the test passes in Travis:

ok  	github.com/ethereum/go-ethereum/swarm/test	10.381s	coverage: 72.2% of statements

[zelig]

eek, do we need to do this?

[lmars]

There is a non trivial amount of time between exec'ing the process and it listening on the IPC socket, a retry dial loop seems to be the easiest way to wait for that to happen.

[zelig]

So these are already the kinds of tests i want the network testing framework to do.
Here i assume your objective is just to have basic tests for the CLI itself?

Just wonder how this will align with our scenario driven test framework and if it is worth the effort
maintaining this. Let's have a chat

[lmars]

These tests are basically just CLI tests yes, which I don't think will be covered by the scenario tests (they most likely won't be calling swarm up, swarm manifest etc.).

My vision is that we have three types of tests:

• Unit tests - testing the behaviour of individual functions / objects
• Integration tests - testing the behaviour from the end user's perspective using compiled binaries / HTTP / IPC requests
• Network scenario tests - testing how the nodes interact with each other at a large scale, running either in-memory / rlpx with unique ports / docker containers / kvm

WDYT?

[lmars]

I have moved the test geth child process runner into a package (cmd/testutils) and used it here to run swarm without requiring the binary to be in PATH.

/cc @fjl @zelig

[fjl]

Please fix the cmd/swarm test on windows.

[fjl]

Also, please move "cmd/testutils" to "internal/cmdtest".

[lmars]

@zelig I have just seen an odd failure in Travis (https://travis-ci.org/ethereum/go-ethereum/jobs/227099226):

• swarm up uploads some data
• a manifest is generated with key d1967515
• it is recognised as a new chunk and a store request is made
• the HTTP request fails with:

error creating manifest: error loading manifest d19675150bfa4419a2ea4344df06d6be810c922f31892d6ca0add96c845e4fbc: Manifest not Found

Here is the output from the logs:

DEBUG[04-29|09:32:07] [BZZ] HTTP: HTTP POST request URL: '/bzz:/', Host: '', Path: '/bzz:/', Referer: '', Accept: '' 
DEBUG[04-29|09:32:07] [BZZ] HTTP: POST request received for bzz:// 
TRACE[04-29|09:32:07] manifest lookup key: 'd1967515'. 
TRACE[04-29|09:32:07] DPA.Put: d1967515 new chunk. call netStore.Put 
TRACE[04-29|09:32:07] dpa: retrieve loop : chunk d1967515 
TRACE[04-29|09:32:07] NetStore.Get: d19675150bfa4419a2ea4344df06d6be810c922f31892d6ca0add96c845e4fbc not found locally. open new request 
TRACE[04-29|09:32:07] serving 0 nodes at 0 (PO00) 
TRACE[04-29|09:32:07] add 0 -> 0 (PO00, PO000) 
TRACE[04-29|09:32:07] add 1 -> 1 (PO01, PO000) 
TRACE[04-29|09:32:07] serve 1 (<=0) nodes for target lookup d19675150bfa4419a2ea4344df06d6be810c922f31892d6ca0add96c845e4fbc (PO000) 
TRACE[04-29|09:32:07] forwarder.Retrieve: d1967515 - received 1 peers from KΛÐΞMLIΛ... 
TRACE[04-29|09:32:07] DPA.Put 0: d1967515 
TRACE[04-29|09:32:07] forwarder.Retrieve: sending retrieveRequest d1967515 to peer [enode://3eeb8ca971d9e20a83f4d1d296618dc9aea0bd1d0dab40f7d5188e1b6386045f6fb84e1e5fb7053f69f18eae95995bbdd4ea8df0c38cd5599e0403b13c9dd0cd@127.0.0.1:43890?discport=0] 
TRACE[04-29|09:32:07] NetStore.Put: localStore.Put d1967515 hit existing request...delivering 
TRACE[04-29|09:32:07] -> 2: from: ourselves, Key: d1967515; ID: 511551311682771724, MaxSize: 0, MaxPeers: 0 (*network.retrieveRequestMsgData) to enode://3eeb8ca971d9e20a83f4d1d296618dc9aea0bd1d0dab40f7d5188e1b6386045f6fb84e1e5fb7053f69f18eae95995bbdd4ea8df0c38cd5599e0403b13c9dd0cd@127.0.0.1:43890?discport=0 
TRACE[04-29|09:32:07] DbStore.Put: d1967515. db storage counter: 1  
TRACE[04-29|09:32:07] DPA.Get: d1967515 retrieved, 0 bytes (0xc420026a80) 
ERROR[04-29|09:32:07] [BZZ] HTTP: error serving POST bzz://: error creating manifest: error loading manifest d19675150bfa4419a2ea4344df06d6be810c922f31892d6ca0add96c845e4fbc: Manifest not Found

That doesn't seem to make much sense to me, thoughts?

[lmars]

@fjl I have rebased this and fixed the test on Windows.