bdk + esplora not seeing an old transaction as confirmed

[danielabrozzoni]

Describe the bug
When syncing using bdk and esplora, we might not see that a certain tx is confirmed, and consider it unconfirmed forever

To Reproduce
See https://github.com/sebastianmontero/bdk-esplora-test

Expected behavior
bdk correctly seeing a transaction as confirmed when it is

Build environment

• BDK tag/commit: 4ee11aa

[LLFourn]

The bug is in the above code and in the wallet example. You're meant to ask the update what heights are missing from the current chain not the wallet. Though I'm not sure why it would be unconfirmed forever. The transaction should be added with its anchors anyway so it should be missing the next time I would have thought. Can we confirm why the anchor is not returned from missing_heights in the future? Is it never inserted? If it's there and not returned it's probably a bug in missing_heights.

[danielabrozzoni]

@LLFourn missing_heights would return the anchor, but update_local_chain wouldn't insert it as it's older than ASSUME_FINAL_DEPTH:

bdk/crates/esplora/src/blocking_ext.rs

Lines 134 to 140 in 4a65a12

	let updated_hash = match fetched_blocks.entry(local_block.height) {
	btree_map::Entry::Occupied(entry) => *entry.get(),
	btree_map::Entry::Vacant(entry) => *entry.insert(
	if local_tip_height - local_block.height >= ASSUME_FINAL_DEPTH {
	local_block.hash
	} else {
	self.get_block_hash(local_block.height)?

[LLFourn]

@LLFourn missing_heights would return the anchor, but update_local_chain wouldn't insert it as it's older than ASSUME_FINAL_DEPTH:

bdk/crates/esplora/src/blocking_ext.rs

Lines 134 to 140 in 4a65a12

	let updated_hash = match fetched_blocks.entry(local_block.height) {
	btree_map::Entry::Occupied(entry) => *entry.get(),
	btree_map::Entry::Vacant(entry) => *entry.insert(
	if local_tip_height - local_block.height >= ASSUME_FINAL_DEPTH {
	local_block.hash
	} else {
	self.get_block_hash(local_block.height)?

But it should be part of request_heights is what I mean. The things in request_heights are always fetched:

bdk/crates/esplora/src/blocking_ext.rs

Lines 107 to 117 in 4a65a12

	for height in request_heights {
	// do not fetch blocks higher than remote tip
	if height > new_tip_height {
	continue;
	}
	// only fetch what is missing
	if let btree_map::Entry::Vacant(entry) = fetched_blocks.entry(height) {
	let hash = self.get_block_hash(height)?;
	entry.insert(hash);
	}
	}

So more succinct question why isn't the things in missing_heights going into request_heights and then being returned.
Maybe something going wrong here:

bdk/crates/esplora/src/blocking_ext.rs

Lines 178 to 186 in 4a65a12

	fetched_blocks
	// we exclude anything at or below the first cp of the update chain otherwise
	// building the chain will fail
	.split_off(&(first_cp.height() + 1))
	.into_iter()
	.map(|(height, hash)| BlockId { height, hash })
	.fold(first_cp, |prev_cp, block| {
	prev_cp.push(block).expect("must extend checkpoint")
	})

[danielabrozzoni]

Yeah, I can look better into it tomorrow, but as far as I can tell the block is fetched as part of the request heights, but then not added to the chain because the earliest agreement checkpoint is not early enough. I initially thought it was because of ASSUME_FINAL_DEPTH, but I might be wrong. -------- Original Message --------

…

On Oct 10, 2023, 02:02, Lloyd Fournier wrote: > ***@***.***(https://github.com/LLFourn) missing_heights would return the anchor, but update_local_chain wouldn't insert it as it's older than ASSUME_FINAL_DEPTH: > > https://github.com/bitcoindevkit/bdk/blob/4a65a12c4fe9bb419906d5c921bbff8b8b4e5379/crates/esplora/src/blocking_ext.rs#L134-L140 But it should be part of request_heights is what I mean. The things in request_heights are always fetched: https://github.com/bitcoindevkit/bdk/blob/4a65a12c4fe9bb419906d5c921bbff8b8b4e5379/crates/esplora/src/blocking_ext.rs#L107-L117 So more succinct question why isn't the things in missing_heights going into request_heights and then being returned. Maybe something going wrong here: https://github.com/bitcoindevkit/bdk/blob/4a65a12c4fe9bb419906d5c921bbff8b8b4e5379/crates/esplora/src/blocking_ext.rs#L178-L186 — Reply to this email directly, [view it on GitHub](#1151 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AF7B4KP2RUA23JKPRJQHRCLX6SF77AVCNFSM6AAAAAA5V5GK5GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJUGA3TOMRWGE). You are receiving this because you were assigned.Message ID: ***@***.***>