Recover from unparseable onion replies from routing failures somehow

[ZmnSCPxj]

Unparseable onion replies are currently not reported to gossipd. This means if we get an unparseable onion reply from routing failure, we do not update the gossipd routemap and would get the same route again.

One simple solution would be to simply randomly select one of the nodes on the route and deactivate its channels (or if that seems too harsh, just one of the channels on the route). This will at least make gossipd return a different route afterwards, at least guaranteeing some sort of forward progress. If we guessed wrong,there is the chance the same unparseable node gets selected again, but at least we can get to try some other route.

xref. #638 (comment)

[cdecker]

I like the idea of temporarily disabling one channel, we could just have a timer that re-enables the channel after a few seconds, or once the route succeeded.

[ZmnSCPxj]

I sketch this here.

1. Add a new gossip_disable_channel_temporarily that accepts only a source node and the out channel from that node. Implement this by disabling temporarily (+20 seconds, like in Make gossipd more forgiving of temporary routing failures #886) this specific out-going channel from that node. (We could reuse the existing gossip_report_routing_failure but I feel it is cleaner to make a separate message for this).
2. In lightningd/pay.c, in case of unparseable onion error, randomly select i from 0 to n_hops - 1 inclusive (if possible, it should be skewed towards earlier hops). The channel is route_channels[i] If 0, the source node is the local node, otherwise it is route_nodes[i - 1]. Report to gossipd this via gossip_disable_channel_temporarily.
3. For testing, maybe implement some dev-malformed-onion-reply command (or command-line flag?) and have channeld deliberately corrupt onion replies (e.g. invert the first byte) if this flag is set.