Skip to content
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
03400c4
feat: support transaction isolation level in dbapi (#1327)
olavloite Apr 2, 2025
b3c259d
fix: improve client-side regex statement parser (#1328)
olavloite Apr 3, 2025
3ac0f91
feat: add SQL statement for begin transaction isolation level (#1331)
olavloite Apr 14, 2025
beb33d2
chore(python): remove noxfile.py from templates (#1335)
gcf-owl-bot[bot] Apr 14, 2025
ca76108
feat: add sample for pre-split feature (#1333)
aksharauke Apr 22, 2025
a8f38cd
chore: sample fix with increased timeout (#1339)
harshachinta Apr 22, 2025
6ca9b43
feat: add interval type support (#1340)
rahul2393 Apr 28, 2025
9331146
chore(main): release 3.54.0 (#1330)
release-please[bot] Apr 28, 2025
e064474
fix: remove setup.cfg configuration for creating universal wheels (#1…
parthea Apr 29, 2025
3943885
fix: E2E tracing metadata append issue (#1357)
surbhigarg92 May 2, 2025
c55fb36
test: fix retry helpers currently causing flaky test failures (#1369)
olavloite May 5, 2025
aae8d61
fix: pass through kwargs in dbapi connect (#1368)
olavloite May 6, 2025
e53eaa2
build: reduce build time (#1370)
olavloite May 6, 2025
686bda6
chore(x-goog-request-id): commit testing scaffold (#1366)
odeke-em May 7, 2025
064d9dc
feat: support fine-grained permissions database roles in connect (#1338)
waltaskew May 7, 2025
1d36f4d
test: add explicit credentials and project to test (#1372)
olavloite May 7, 2025
fd4ee67
build: exclude presubmit.cfg from owlbot generation (#1373)
olavloite May 7, 2025
3a91671
chore(x-goog-spanner-request-id): plug in functionality after test sc…
odeke-em May 17, 2025
1f4e98c
test: add a test for unary retries of UNAVAILABLE (#1376)
olavloite May 20, 2025
de322f8
docs: fix markdown formatting in transactions page (#1377)
rahul2393 May 20, 2025
bae395d
chore(x-goog-spanner-request-id): add x_goog_spanner_request_id as an…
odeke-em May 21, 2025
d532d57
chore: Update gapic-generator-python to 1.24.1 (#1314)
gcf-owl-bot[bot] May 21, 2025
b433281
chore(x-goog-spanner-request-id): more updates for batch_write + mock…
odeke-em May 22, 2025
aa4880b
chore: add samples for transaction timeout configuration (#1380)
alkatrivedi May 27, 2025
97d7268
feat: Add support for multiplexed sessions (#1381)
rahul2393 May 28, 2025
234135d
chore(main): release 3.55.0 (#1363)
release-please[bot] May 28, 2025
f9fd347
chore: add sample to set statement timeout within a transaciton (#1384)
alkatrivedi Jun 4, 2025
d35a0cc
Update `SessionOptions` to support `GOOGLE_CLOUD_SPANNER_FORCE_DISABL…
currantw May 29, 2025
34baadf
feat: Multiplexed sessions - Remove handling of `MethodNotImplemented…
currantw May 29, 2025
998f23f
feat: Multiplexed sessions - Update `Connection` to use multiplexed s…
currantw May 30, 2025
ec19f2d
cleanup: Rename `beforeNextRetry` to `before_next_retry`.
currantw May 30, 2025
25d0943
cleanup: Fix a few unrelated typos.
currantw May 30, 2025
fca6f06
feat: Multiplexed sessions - Add ingest of precommit tokens to `_Snap…
currantw May 30, 2025
56001b9
feat: Multiplexed sessions - Deprecate `StreamedResultSet._source` (r…
currantw May 30, 2025
b4eadca
feat: Multiplexed sessions - Move `_session_options` from `Database` …
currantw May 30, 2025
68e9b67
feat: Multiplexed sessions - Deprecate `SessionCheckout` and update `…
currantw May 30, 2025
6ca0d3f
feat: Multiplexed sessions - Deprecate `Database.session()` and minor…
currantw May 31, 2025
9057a64
feat: Multiplexed sessions - Update `BatchSnapshot` to use database s…
currantw May 31, 2025
c9dd818
feat: Multiplexed sessions - Move `Batch` and `Transaction` attribute…
currantw Jun 2, 2025
599939a
feat: Multiplexed sessions - Update pools so they don't use deprecate…
currantw Jun 2, 2025
2065e52
feat: Multiplexed sessions - Update session to remove class attribute…
currantw Jun 2, 2025
7b925b3
feat: Multiplexed sessions - Move begin transaction logic from `Snaps…
currantw Jun 3, 2025
9246dd2
feat: Multiplexed sessions - Remove begin transaction logic from `Tra…
currantw Jun 3, 2025
f1b3fdb
feat: Multiplexed sessions - Add logic for beginning mutations-only t…
currantw Jun 3, 2025
98c477d
feat: Multiplexed sessions - Cleanup and improve consistency of state…
currantw Jun 3, 2025
052f3e1
feat: Multiplexed sessions - Cleanup documentation for `Batch.commit`…
currantw Jun 3, 2025
2b9f212
feat: Multiplexed sessions - Add logic for retrying commits if precom…
currantw Jun 3, 2025
a77cc2b
feat: Multiplexed sessions - Remove `GOOGLE_CLOUD_SPANNER_FORCE_DISAB…
currantw Jun 4, 2025
5615f2c
feat: Multiplexed sessions - Cleanup `TestDatabaseSessionManager` so …
currantw Jun 4, 2025
00059f9
feat: Multiplexed sessions - Add type hints for `SessionOptions` and …
currantw Jun 4, 2025
083d6bc
feat: Multiplexed sessions - Fix `test_observability_options`
currantw Jun 4, 2025
6e33b1d
feat: Multiplexed sessions - Update `_builders` to use mock scoped cr…
currantw Jun 4, 2025
65042ab
feat: Multiplexed sessions - Add helpers for mock scoped credentials …
currantw Jun 4, 2025
9df088d
feat: Multiplexed sessions - Fix failing `test_batch_insert_then_read`.
currantw Jun 4, 2025
607df64
feat: Multiplexed sessions - Fix failing `test_transaction_read_and_i…
currantw Jun 4, 2025
0b6f5df
feat: Multiplexed sessions - Add test helper for multiplexed env vars.
currantw Jun 4, 2025
36c9775
feat: Multiplexed sessions - Add unit tests for begin transaction bas…
currantw Jun 4, 2025
418edda
feat: Multiplexed sessions - Attempt to fix `test_transaction_read_an…
currantw Jun 4, 2025
da226c1
feat: Multiplexed sessions - Add test for log when new session create…
currantw Jun 4, 2025
c6c130e
feat: Multiplexed sessions - Add additional multiplexed unit tests fo…
currantw Jun 4, 2025
1018f4c
feat: Multiplexed sessions - Cleanup `Transaction` by extracting some…
currantw Jun 4, 2025
b761e85
feat: Multiplexed sessions - Add additional `Transaction` tests for n…
currantw Jun 5, 2025
665547b
feat: Multiplexed sessions - Fix linter
currantw Jun 5, 2025
0bd5fd1
feat: Multiplexed sessions - Remove unnecessary TODO
currantw Jun 5, 2025
4cb1f05
feat: Multiplexed sessions - Remove unnecessary constants.
currantw Jun 5, 2025
17f3c5f
feat: Multiplexed sessions - Remove support for disabling the use of …
currantw Jun 5, 2025
9c3db74
feat: Multiplexed sessions - Make deprecation comments a bit more clear.
currantw Jun 9, 2025
17a2f47
feat: Multiplexed sessions - Add some more type hints.
currantw Jun 9, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat: Multiplexed sessions - Update session to remove class attribute…
…s, add TODOs, and make `Session._transaction` default to None. Plus add some `Optional` typing hints.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
  • Loading branch information
currantw committed Jun 4, 2025
commit 2065e52efe28de1485f353b19db8ae2c615a16ee
8 changes: 4 additions & 4 deletions google/cloud/spanner_v1/batch.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
"""Context manager for Cloud Spanner batched writes."""
import functools
from datetime import datetime
from typing import List
from typing import List, Optional

from google.cloud.spanner_v1 import CommitRequest, CommitResponse
from google.cloud.spanner_v1 import Mutation
Expand Down Expand Up @@ -53,11 +53,11 @@ def __init__(self, session):
super(_BatchBase, self).__init__(session)

self._mutations: List[Mutation] = []
self.transaction_tag: str = None
self.transaction_tag: Optional[str] = None

self.committed: datetime = None
self.committed: Optional[datetime] = None
"""Timestamp at which the batch was successfully committed."""
self.commit_stats: CommitResponse.CommitStats = None
self.commit_stats: Optional[CommitResponse.CommitStats] = None

# TODO multiplexed - cleanup
def _check_state(self):
Expand Down
72 changes: 44 additions & 28 deletions google/cloud/spanner_v1/session.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
from functools import total_ordering
import time
from datetime import datetime
from typing import MutableMapping, Optional

from google.api_core.exceptions import Aborted
from google.api_core.exceptions import GoogleAPICallError
Expand Down Expand Up @@ -69,17 +70,20 @@ class Session(object):
:param is_multiplexed: (Optional) whether this session is a multiplexed session.
"""

_session_id = None
_transaction = None

def __init__(self, database, labels=None, database_role=None, is_multiplexed=False):
self._database = database
self._session_id: Optional[str] = None

# TODO multiplexed - remove
self._transaction: Optional[Transaction] = None

if labels is None:
labels = {}
self._labels = labels
self._database_role = database_role
self._is_multiplexed = is_multiplexed
self._last_use_time = datetime.utcnow()

self._labels: MutableMapping[str, str] = labels
self._database_role: Optional[str] = database_role
self._is_multiplexed: bool = is_multiplexed
self._last_use_time: datetime = datetime.utcnow()

def __lt__(self, other):
return self._session_id < other._session_id
Expand All @@ -100,7 +104,7 @@ def is_multiplexed(self):

@property
def last_use_time(self):
""" "Approximate last use time of this session
"""Approximate last use time of this session

:rtype: datetime
:returns: the approximate last use time of this session"""
Expand Down Expand Up @@ -157,27 +161,28 @@ def create(self):

if self._session_id is not None:
raise ValueError("Session ID already set by back-end")
api = self._database.spanner_api
metadata = _metadata_with_prefix(self._database.name)
if self._database._route_to_leader_enabled:

database = self._database
api = database.spanner_api

metadata = _metadata_with_prefix(database.name)
if database._route_to_leader_enabled:
metadata.append(
_metadata_with_leader_aware_routing(
self._database._route_to_leader_enabled
)
_metadata_with_leader_aware_routing(database._route_to_leader_enabled)
)

request = CreateSessionRequest(database=self._database.name)
if self._database.database_role is not None:
request.session.creator_role = self._database.database_role
create_session_request = CreateSessionRequest(database=database.name)
if database.database_role is not None:
create_session_request.session.creator_role = database.database_role

if self._labels:
request.session.labels = self._labels
create_session_request.session.labels = self._labels

# Set the multiplexed field for multiplexed sessions
if self._is_multiplexed:
request.session.multiplexed = True
create_session_request.session.multiplexed = True

observability_options = getattr(self._database, "observability_options", None)
observability_options = getattr(database, "observability_options", None)
span_name = (
"CloudSpanner.CreateMultiplexedSession"
if self._is_multiplexed
Expand All @@ -191,9 +196,9 @@ def create(self):
metadata=metadata,
) as span, MetricsCapture():
session_pb = api.create_session(
request=request,
metadata=self._database.metadata_with_request_id(
self._database._next_nth_request,
request=create_session_request,
metadata=database.metadata_with_request_id(
database._next_nth_request,
1,
metadata,
span,
Expand Down Expand Up @@ -462,6 +467,7 @@ def batch(self):

return Batch(self)

# TODO multiplexed - remove
def transaction(self):
"""Create a transaction to perform a set of reads with shared staleness.

Expand All @@ -474,7 +480,7 @@ def transaction(self):

if self._transaction is not None:
self._transaction.rolled_back = True
del self._transaction
self._transaction = None

txn = self._transaction = Transaction(self)
return txn
Expand Down Expand Up @@ -531,6 +537,7 @@ def run_in_transaction(self, func, *args, **kw):
observability_options=observability_options,
) as span, MetricsCapture():
while True:
# TODO multiplexed - remove
if self._transaction is None:
txn = self.transaction()
txn.transaction_tag = transaction_tag
Expand All @@ -552,8 +559,11 @@ def run_in_transaction(self, func, *args, **kw):

return_value = func(txn, *args, **kw)

# TODO multiplexed: store previous transaction ID.
except Aborted as exc:
del self._transaction
# TODO multiplexed - remove
self._transaction = None

if span:
delay_seconds = _get_retry_delay(
exc.errors[0],
Expand All @@ -573,7 +583,9 @@ def run_in_transaction(self, func, *args, **kw):
)
continue
except GoogleAPICallError:
del self._transaction
# TODO multiplexed - remove
self._transaction = None

add_span_event(
span,
"User operation failed due to GoogleAPICallError, not retrying",
Expand All @@ -596,7 +608,9 @@ def run_in_transaction(self, func, *args, **kw):
max_commit_delay=max_commit_delay,
)
except Aborted as exc:
del self._transaction
# TODO multiplexed - remove
self._transaction = None

if span:
delay_seconds = _get_retry_delay(
exc.errors[0],
Expand All @@ -615,7 +629,9 @@ def run_in_transaction(self, func, *args, **kw):
exc, deadline, attempts, default_retry_delay=default_retry_delay
)
except GoogleAPICallError:
del self._transaction
# TODO multiplexed - remove
self._transaction = None

add_span_event(
span,
"Transaction.commit failed due to GoogleAPICallError, not retrying",
Expand Down
6 changes: 3 additions & 3 deletions google/cloud/spanner_v1/snapshot.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

import functools
import threading
from typing import List, Union
from typing import List, Union, Optional

from google.protobuf.struct_pb2 import Struct
from google.cloud.spanner_v1 import (
Expand Down Expand Up @@ -225,12 +225,12 @@ def __init__(self, session):
self._read_request_count: int = 0

# Identifier for the transaction.
self._transaction_id: bytes = None
self._transaction_id: Optional[bytes] = None

# Precommit tokens are returned for transactions with
# multiplexed sessions. The precommit token with the
# highest sequence number is included in the commit request.
self._precommit_token: MultiplexedSessionPrecommitToken = None
self._precommit_token: Optional[MultiplexedSessionPrecommitToken] = None

# Operations within a transaction can be performed using multiple
# threads, so we need to use a lock when updating the transaction.
Expand Down
6 changes: 3 additions & 3 deletions google/cloud/spanner_v1/transaction.py
Original file line number Diff line number Diff line change
Expand Up @@ -259,7 +259,7 @@ def wrapped_method(*args, **kwargs):
attempt.increment()
rollback_method = functools.partial(
api.rollback,
session=self._session.name,
session=session.name,
transaction_id=self._transaction_id,
metadata=database.metadata_with_request_id(
nth_request,
Expand All @@ -278,7 +278,7 @@ def wrapped_method(*args, **kwargs):
self.rolled_back = True

# TODO multiplexed - remove
del self._session._transaction
self._session._transaction = None

def commit(
self, return_commit_stats=False, request_options=None, max_commit_delay=None
Expand Down Expand Up @@ -396,7 +396,7 @@ def before_next_retry(nth_retry, delay_in_seconds):
self.commit_stats = response_pb.commit_stats

# TODO multiplexed - remove
del self._session._transaction
self._session._transaction = None

return self.committed

Expand Down