Skip to content

Added named parameters for base_port and base_rpcport. #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
winksaville wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Added named parameters for base_port and base_rpcport. #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 28 additions & 16 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ Welcome to the FRONTIER
Running a cluster of 8 instances under dir `tmp/eth/` isolated on local eth network (id 3301), launch 05. Give external IP and pass extra param `--mine`.

```
GETH=./geth bash gethcluster.sh <root> <n> <network_id> <runid> <IP> [[params]...]
GETH=./geth bash gethcluster.sh <root> <network_id> <n> <runid> <IP> [[params]...]
```

This will set up a local cluster of nodes
Expand All @@ -56,27 +56,39 @@ This will set up a local cluster of nodes
- the nodes can be restarted from the same state individually using the `gethup.sh` script
- if you want to interact with the nodes, use a json-rpc client
- you can supply additional params on the command line which will be passed
to `gethup.sh` and eventually to `geth` for each node, for instance `-vmodule=http=6 -mine -minerthreads=8` is a good one.
to `gethup.sh` and eventually to `geth` for each node, for instance `--vmodule http=6 --miner --minerthreads 8` is a good one.

```
GETH=./geth bash gethcluster.sh ./leagues/3301/cicada 2 3301 05 77.160.58.3 -mine
launching node 0/2 ---> tail -f ./leagues/3301/cicada/00.05.log
GETH=./geth bash gethcluster.sh ./leagues/3301/cicada 3301 2 05 77.160.58.3 --miner
launching node 0/2 ---> tail -f ./leagues/3301/cicada/3301/log/00.log
Welcome to the FRONTIER
launching node 1/2 ---> tail -f ./leagues/3301/cicada/01.05.log
launching node 1/2 ---> tail -f ./leagues/3301/cicada/3301/log/01.log
Welcome to the FRONTIER
```

fill create:
```
./leagues/3301/cicada/
./leagues/3301/cicada/3301/
./leagues/3301/cicada/3301/00/
./leagues/3301/cicada/3301/00.05.log
./leagues/3301/cicada/3301/00.05.glog
./leagues/3301/cicada/3301/01/
./leagues/3301/cicada/3301/01.05.log
./leagues/3301/cicada/3301/01.05.glog
./leagues/3301/cicada/3301/
will create:
```
$ tree -d ./leagues/3301/cicda
.
├── data
│   ├── 00
│   │   ├── geth
│   │   │   ├── chaindata
│   │   │   └── nodes
│   │   └── keystore
│   │   └── keystore
│   └── 01
│   ├── geth
│   │   ├── chaindata
│   │   └── nodes
│   └── keystore
│   └── keystore
├── keystore
│   ├── 00
│   │   └── keystore
│   └── 01
│   └── keystore
└── log
```

You can kill and restart individual nodes or the entire cluster safely, by using different runid you can separate logs for the individual runs in a neat way.
Expand Down
130 changes: 96 additions & 34 deletions gethcluster.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,68 +1,130 @@
# !/bin/bash
# bash cluster <root> <network_id> <number_of_nodes> <runid> <local_IP> [[params]...]
# bash cluster <root> <network_id> <number_of_nodes> <local_IP> [[params]...]
# https://github.com/ethereum/go-ethereum/wiki/Setting-up-monitoring-on-local-cluster

# sets up a local ethereum network cluster of nodes
# - <number_of_nodes> is the number of nodes in cluster
# - <root> is the root directory for the cluster, the nodes are set up
# with datadir `<root>/<network_id>/00`, `<root>/ <network_id>/01`, ...
# - new accounts are created for each node
# - they launch on port 30300, 30301, ...
# - they star rpc on port 8100, 8101, ...
# - by collecting the nodes nodeUrl, they get connected to each other
# - if enode has no IP, `<local_IP>` is substituted
# - if `<network_id>` is not 0, they will not connect to a default client,
# resulting in a private isolated network
# - the nodes log into `<root>/00.<runid>.log`, `<root>/01.<runid>.log`, ...
# - The nodes launch in mining mode
# - the cluster can be killed with `killall geth` (FIXME: should record PIDs)
# and restarted from the same state
# - if you want to interact with the nodes, use rpc
# - you can supply additional params on the command line which will be passed
# to each node, for instance `-mine`
# Defaults for our two optional parameters
DFLT_BASE_PORT=311
DFLT_BASE_RPCPORT=82

# Other args if not named based
other=""


usage() {
echo "Usage: $0 <root> <network_id> <number_of_nodex> <local_IP> [ base_port=xx ] [ base_rpcport=yy ] [[params] ...]"
echo " Bring up a local ethereum network cluster of nodes."
echo " killall -q geth can be used to kill them, obviously be careful :)"
echo ""
echo " Two required parameters:"
echo " root is root directory of the cluster nodes are setup"
echo " with data_dir=<root>/<network_id/00, <root>/<network_id>/01, .."
echo " nework_id is an identifier unique for this network"
echo " number_of_nodes is the number of nodes to creeate"
echo " local_IP is the IP address of for this computer"
echo ""
echo " Two optional parameters:"
echo " base_port=yy default is ${DFLT_BASE_PORT}"
echo " Where the rpcport will be concatonated with to xx"
echo " so base_rpcport=${DFLT_BASE_PORT} and instance_name is 03"
echo " port == ${DFLT_PORT}03"
echo " base_rpcport=yy default is ${DFLT_BASE_RPCPORT}"
echo " Where the rpcport will be concatonated with to yy"
echo " so base_rpcport=${DFLT_BASE_RPCPORT} and instance_name is 03"
echo " port == ${DFLT_BASE_RPCPORT}03"
echo ""
echo " \[\[params\] ...] are additional parameters passed to gethup.sh such as --miner --minerthreads 2"
}

if (( $# == 0 )); then
usage
exit 1
fi

root=$1
shift
network_id=$1
dir=$root/$network_id
mkdir -p $dir/data
mkdir -p $dir/log
data_dir=$root/$network_id
mkdir -p $data_dir/data
mkdir -p $data_dir/log
shift
N=$1
shift
ip_addr=$1
shift

# Declare an array of named arguments
declare -A args

function print_args() {
echo "root=$root"
echo "network_id=$network_id"
echo "N=$N"
echo "ip_addr=$ip_addr"

echo "base_port=${args[base_port]}"
echo "base_rpcport=${args[base_rpcport]}"
echo "other=${other}"
}

# Default values for named parameters
args[base_port]=$DFLT_BASE_PORT
args[base_rpcport]=$DFLT_BASE_RPCPORT

# Parse the named arguments
function parse_named_args() {
for a in $*
do
#echo "a=$a"
case $a in
base_port=*)
local $a
args[base_port]="$base_port"
;;
base_rpcport=*)
local $a
args[base_rpcport]="$base_rpcport"
;;
*)
[[ "$other" == "" ]] && other=$a || other="$other $a"
;;
esac
done
}

parse_named_args $*

#print_args


# GETH=geth

if [ ! -f "$dir/nodes" ]; then
if [ ! -f "$data_dir/nodes" ]; then

echo "[" >> $dir/nodes
echo "[" >> $data_dir/nodes
for ((i=0;i<N;++i)); do
id=`printf "%02d" $i`
if [ ! $ip_addr="" ]; then
ip_addr="[::]"
fi

echo "getting enode for instance $id ($i/$N)"
eth="$GETH --datadir $dir/data/$id --port 303$id --networkid $network_id"
eth="$GETH --datadir $data_dir/data/$id --port ${args[base_port]}${id} --networkid $network_id"
cmd="$eth js <(echo 'console.log(admin.nodeInfo.enode); exit();') "
echo $cmd
bash -c "$cmd" 2>/dev/null |grep enode | perl -pe "s/\[\:\:\]/$ip_addr/g" | perl -pe "s/^/\"/; s/\s*$/\"/;" | tee >> $dir/nodes
bash -c "$cmd" 2>/dev/null |grep enode | perl -pe "s/\[\:\:\]/$ip_addr/g" | perl -pe "s/^/\"/; s/\s*$/\"/;" | tee >> $data_dir/nodes
if ((i<N-1)); then
echo "," >> $dir/nodes
echo "," >> $data_dir/nodes
fi
done
echo "]" >> $dir/nodes
echo "]" >> $data_dir/nodes
fi

for ((i=0;i<N;++i)); do
id=`printf "%02d" $i`
# echo "copy $dir/data/$id/static-nodes.json"
mkdir -p $dir/data/$id
# cp $dir/nodes $dir/data/$id/static-nodes.json
echo "launching node $i/$N ---> tail-f $dir/log/$id.log"
echo GETH=$GETH bash ./gethup.sh $dir $id --networkid $network_id $*
GETH=$GETH bash ./gethup.sh $dir $id --networkid $network_id $*
# echo "copy $data_dir/data/$id/static-nodes.json"
mkdir -p $data_dir/data/$id
# cp $data_dir/nodes $data_dir/data/$id/static-nodes.json
echo "launching node $i/$N ---> tail-f $data_dir/log/$id.log"
echo "GETH=$GETH bash ./gethup.sh $data_dir $id base_port=${args[base_port]} base_rpcport=${args[base_rpcport]} --networkid $network_id $other"
GETH=$GETH bash ./gethup.sh $data_dir $id base_port=${args[base_port]} base_rpcport=${args[base_rpcport]} --networkid $network_id $other
done
97 changes: 80 additions & 17 deletions gethup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,81 @@
#!/bin/bash
# Usage:
# bash /path/to/eth-utils/gethup.sh <datadir> <instance_name>
# bash /path/to/eth-utils/gethup.sh <datadir> <instance_name> <base_port=xx> <base_rpcport=yyy>

# Defaults for our two optional parameters
DFLT_BASE_PORT=311
DFLT_BASE_RPCPORT=82
other=""


usage() {
echo "Usage: $0 datadir instance_name [ base_port=xx ] [ base_rpcport=yy ]"
echo " Bring up an eth node"
echo ""
echo " Two required parameters:"
echo " datadir is the data directory"
echo " instance_name is the node instance name"
echo ""
echo " Two optional parameters:"
echo " base_port=yy default is ${DFLT_BASE_PORT}"
echo " Where the rpcport will be concatonated with to xx"
echo " so base_rpcport=${DFLT_BASE_PORT} and instance_name is 03"
echo " port == ${DFLT_PORT}03"
echo " base_rpcport=yy default is ${DFLT_BASE_RPCPORT}"
echo " Where the rpcport will be concatonated with to yy"
echo " so base_rpcport=${DFLT_BASE_RPCPORT} and instance_name is 03"
echo " port == ${DFLT_BASE_RPCPORT}03"
}

if (( $# == 0 )); then
usage
exit 1
fi

root=$1 # base directory to use for datadir and logs
shift
dd=$1 # double digit instance id like 00 01 02
shift

# Declare an array of named arguments
declare -A args

function print_args() {
echo "root=$root"
echo "dd=$dd"
echo "base_port=${args[base_port]}"
echo "base_rpcport=${args[base_rpcport]}"
echo "other=${other}"
}

# Default values for named parameters
args[base_port]=$DFLT_BASE_PORT
args[base_rpcport]=$DFLT_BASE_RPCPORT

# Parse the named arguments
function parse_named_args() {
for a in $*
do
#echo "a=$a"
case $a in
base_port=*)
local $a
args[base_port]="$base_port"
;;
base_rpcport=*)
local $a
args[base_rpcport]="$base_rpcport"
;;
*)
[[ "$other" == "" ]] && other=$a || other="$other $a"
;;
esac
done
}

parse_named_args $*

#print_args

# logs are output to a date-tagged file for each run , while a link is
# created to the latest, so that monitoring be easier with the same filename
Expand All @@ -20,8 +89,8 @@ log=$root/log/$dd.$datetag.log # /tmp/eth/04.09.log
linklog=$root/log/$dd.current.log # /tmp/eth/04.09.log
stablelog=$root/log/$dd.log # /tmp/eth/04.09.log
password=$dd # 04
port=311$dd # 30304
rpcport=82$dd # 8104
port=${args[base_port]}${dd} # 31104
rpcport=${args[base_rpcport]}${dd} # 8204

mkdir -p $root/data
mkdir -p $root/log
Expand Down Expand Up @@ -51,30 +120,24 @@ echo "copying keys $root/keystore/$dd $datadir/keystore"
cp -R $root/keystore/$dd/keystore/ $datadir/keystore/
# fi

BZZKEY=`$GETH --datadir=$datadir account list|head -n1|perl -ne '/([a-f0-9]{40})/ && print $1'`

# bring up node `dd` (double digit)
# - using <rootdir>/<dd>
# - listening on port 303dd, (like 30300, 30301, ...)
# - listening on port <base_port>dd, (like 31100, 31101, ...)
# - with the account unlocked
# - launching json-rpc server on port 81dd (like 8100, 8101, 8102, ...)
# - launching json-rpc server on port <base_rpcport>dd (like 8200, 8201, 8202, ...)
echo "$GETH --datadir=$datadir \
--identity="$dd" \
--bzzaccount=$BZZKEY --bzzport=86$dd \
--port=$port \
--unlock=$BZZKEY \
--password=<(echo -n $dd) \
--rpc --rpcport=$rpcport --rpccorsdomain='*' $* \
--port $port \
--password <(echo -n $dd) \
--rpc --rpcport=$rpcport --rpccorsdomain='*' $other \
2>&1 | tee "$stablelog" > "$log" & # comment out if you pipe it to a tty etc.
"

$GETH --datadir=$datadir \
--identity="$dd" \
--bzzaccount=$BZZKEY --bzzport=86$dd \
--port=$port \
--unlock=$BZZKEY \
--identity "$dd" \
--port $port \
--password=<(echo -n $dd) \
--rpc --rpcport=$rpcport --rpccorsdomain='*' $* \
--rpc --rpcport $rpcport --rpccorsdomain '*' $other \
2>&1 | tee "$stablelog" > "$log" & # comment out if you pipe it to a tty etc.

# to bring up logs, uncomment
Expand Down