Add latency stats for entry location index lookup so that possible RocksDB bottleneck can be detected

[lhotari]

Motivation

• this metric will help to detect when the bottleneck is in the entry location index lookup operations
  and RocksDB tuning is needed

Changes

Add new operation latency metrics lookup-entry-location which will be provided at the same level of metrics as the existing entries-count metric.

[StevenLuMT]

LGTM

[hangc0276]

We already have the index lookup stats read-locations-index-time, does it has any case this metric can't cover?

bookkeeper/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/storage/ldb/DbLedgerStorageStats.java

Line 48 in de5d45f

private static final String READ_ENTRY_LOCATIONS_INDEX_TIME = "read-locations-index-time";

[lhotari]

We already have the index lookup stats read-locations-index-time, does it has any case this metric can't cover?

bookkeeper/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/storage/ldb/DbLedgerStorageStats.java

Line 48 in de5d45f

private static final String READ_ENTRY_LOCATIONS_INDEX_TIME = "read-locations-index-time";

thanks for pointing that out @hangc0276 .

The problem with the read-locations-index-time is that it's not a latency metric. it's a counter metric that records the total time spent for getLocationIndex calls.
I also noticed that the metric is not used for the getLocation call in entryExists method:

bookkeeper/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/storage/ldb/SingleDirectoryDbLedgerStorage.java

Lines 344 to 348 in 4ee8c04

	// Read from main storage
	long entryLocation = entryLocationIndex.getLocation(ledgerId, entryId);
	if (entryLocation != 0) {
	return true;
	}

.

[eolivelli]

LGTM