Ensure locking patterns don't prevent parallel requests

[fairclothjm]

Description

Allow Create, Update and Delete operations to be performed in parallel.

This PR fixes an issue in which the locking pattern did not allow parallel requests. The lease revocations were being performed sequentially which would block until completion. This would eventually exceed the default timeout of 90s, causing lease revocations to fail with err: failed to make connection: error in Connection waiting for cluster: unambiguous timeout. Additionally, this could block concurrent Create and Update operations.

Manual test

• cd to couchbase plugin dir
  • checkout this branch
  • make dev
• then run the test script
• observe the lease revocations are completed and there are no read errors

test script

#!/usr/bin/env bash

set -e

docker run \
  --name couchbase-server \
  --publish 8091-8094:8091-8094 \
  --publish 11210-11211:11210-11211 \
  --detach \
  --rm \
  couchbase/server-sandbox:6.5.0

docker ps -f name=couchbase-server --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"

vault write database/config/cb \
  plugin_name="couchbase" \
  hosts="localhost" \
  username="Administrator" \
  password="password" \
  allowed_roles="role*"

echo "===> Create roles for testing..."

for i in {1..10}; do
  vault write database/roles/role${i} \
    db_name=cb \
    default_ttl="30s" \
    max_ttl="30s"
done

echo "===> Create numerous new credentials..."

read_creds() {
  for i in {1..10}; do
    vault read database/creds/role${i} > /dev/null
  done
}

for r in {1..100}; do
  echo -n "${r} "
  read_creds "$r"
done

vault lease revoke -prefix database/creds