[Bug] 0.7.2: UNIQUE constraint failed: `block.height`

[NicolasDorier]

When I am running the tests for btcpay on the new version of clightning, it crashes.

2019-08-17T06:52:16.254Z DEBUG lightningd(7): 03b02f952b85ee87bc53235102a393e6d218712f6dfaba2aa327d5b0cf66d89236 chan #1: Balance 1204000msat -> 1205000msat
2019-08-17T06:52:16.258Z DEBUG lightningd(7): lightning_channeld-03b02f952b85ee87bc53235102a393e6d218712f6dfaba2aa327d5b0cf66d89236 chan #1: ... , awaiting 1121
2019-08-17T06:52:16.259Z DEBUG lightningd(7): lightning_channeld-03b02f952b85ee87bc53235102a393e6d218712f6dfaba2aa327d5b0cf66d89236 chan #1: Got it!
2019-08-17T06:52:16.259Z DEBUG lightning_hsmd(19): Client: Received message 18 from client
2019-08-17T06:52:16.259Z DEBUG lightningd(7): lightning_channeld-03b02f952b85ee87bc53235102a393e6d218712f6dfaba2aa327d5b0cf66d89236 chan #1: Sending revoke_and_ack
2019-08-17T06:52:16.259Z DEBUG lightningd(7): lightning_channeld-03b02f952b85ee87bc53235102a393e6d218712f6dfaba2aa327d5b0cf66d89236 chan #1: htlc 10: RCVD_REMOVE_ACK_COMMIT->SENT_REMOVE_ACK_REVOCATION
2019-08-17T06:52:16.259Z DEBUG lightningd(7): lightning_channeld-03b02f952b85ee87bc53235102a393e6d218712f6dfaba2aa327d5b0cf66d89236 chan #1: sending_revoke_and_ack: HTLC REMOTE 10 = SENT_REMOVE_ACK_REVOCATION/RCVD_REMOVE_ACK_REVOCATION FULFILLED
2019-08-17T06:52:16.261Z DEBUG lightningd(7): lightning_channeld-03b02f952b85ee87bc53235102a393e6d218712f6dfaba2aa327d5b0cf66d89236 chan #1: peer_out WIRE_REVOKE_AND_ACK
2019-08-17T06:52:17.187Z DEBUG lightningd(7): Adding block 119: 3c39dadb926fce4f547cb0208aba55760c6d35c555ad0f67b14231a4c9a1d362
2019-08-17T06:52:17.187Z **BROKEN** lightningd(7): wallet_block_add: UNIQUE constraint failed: blocks.height
wallet_block_add: UNIQUE constraint failed: blocks.height
lightningd: FATAL SIGNAL 6 (version basedon-v0.7.2)
0x55656772f2c1 send_backtrace
        common/daemon.c:40
0x55656772f367 crashdump
        common/daemon.c:53
0x7f218612b0e9 ???
        ???:0
2019-08-17T06:52:17.215Z **BROKEN** lightningd(7): FATAL SIGNAL 6 (version basedon-v0.7.2)
2019-08-17T06:52:17.215Z **BROKEN** lightningd(7): backtrace: common/daemon.c:45 (send_backtrace) 0x55656772f317
2019-08-17T06:52:17.215Z **BROKEN** lightningd(7): backtrace: common/daemon.c:53 (crashdump) 0x55656772f367
2019-08-17T06:52:17.215Z **BROKEN** lightningd(7): backtrace: (null):0 ((null)) 0x7f218612b0e9
Log dumped in crash.log.20190817065217
./entrypoint.sh: line 98:     7 Aborted                 (core dumped) lightningd "$@"

[NicolasDorier]

Same tests on 0.7.1 pass.

[NicolasDorier]

The logs does not show block 119 mined twice.

[NicolasDorier]

Made a pastebin on https://pastebin.com/P4zfeAnH (on another run)

[cdecker]

Might be because of the new backfilling mechanism inserting the filteredblock, and then the normal sync catching up.

Can you reproduce this? I'm assuming there is a quick succession of restarts involved (restart backing off the sync height, causing a block to be left inbetween).

[NicolasDorier]

I don't know exactly how to reproduce it, but I know that it break my tests consistently.

[NicolasDorier]

I don't really understand what you mean by "restarts" as you can see in the logs, there is no restart. Not of bitcoind and not of clightning.

[NicolasDorier]

Also, as you can see in the log, it is the same run as when the database get created.

[NicolasDorier]

Will update btcpay version of clightning when this one has a commit to fix it.

I can put additional logs or anything to help diagnose the issue, just let me know.

[rustyrussell]

I'm holding the release for this :(