Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
a43b2eb
Split 'Makefile' into datastore and bigtable portions.
tseaver Jun 24, 2016
0f4a064
Generate Bigtable V2 protos.
tseaver Jun 24, 2016
9d7d8af
Exclude new '_generated_v2' files from coverage.
tseaver Jun 24, 2016
9a2d313
Drop 'Table.rename'.
tseaver Jun 26, 2016
f5b1719
Drop overlooked docs / system test usage of 'Table.rename'.
tseaver Jun 26, 2016
e570b03
Merge pull request #1903 from tseaver/bigtable-v2-regenerate_protobufs
tseaver Jun 26, 2016
16b04c9
Parse JSON acceptance tests.
tseaver Jun 25, 2016
ebf27b5
Add 'ReadRowsResponseV2' wrapper.
tseaver Jun 26, 2016
35103f9
Merge pull request #1908 from tseaver/bigtable-v2-drop_table_rename
tseaver Jun 26, 2016
47b9246
Alias Bigtable V1 imports / factories / entry point constants.
tseaver Jun 26, 2016
377f0da
Merge pull request #1907 from tseaver/bigtable-v2-read_rows_response_…
tseaver Jun 27, 2016
cdbc47c
Merge pull request #1912 from tseaver/bigtable-v2-alias_imports_endpo…
tseaver Jun 27, 2016
3c533e0
Factor out generating GCRule-related protobufs.
tseaver Jun 26, 2016
7e9c61d
Update bigtable.column_family to use V2 protos.
tseaver Jun 26, 2016
ae3d818
Merge pull request #1913 from tseaver/bigtable-v2-column_family_crud
tseaver Jun 27, 2016
5d179bc
Clarify: testcases leaving incomplete final rows do not raise.
tseaver Jun 27, 2016
6b2f07f
Distinguish response-level errors from invalid chunks.
tseaver Jun 27, 2016
fcc3751
Merge pull request #1915 from tseaver/bigtable-v2-readrowsresponse-er…
tseaver Jun 27, 2016
8916b6c
Factor out generating RowFilter-related protobufs.
tseaver Jun 27, 2016
1b0fb53
Convert to non-instance-admin Bigtable V2 protos.
tseaver Jun 27, 2016
3b48e0f
Merge pull request #1919 from tseaver/bigtable-v2-convert_imports
tseaver Jun 27, 2016
4ed0b8f
Drop 'Cluster.table' and 'Cluster.list_tables'.
tseaver Jun 27, 2016
7cbb215
In V2, tables hold a ref. to their instance, not a cluster.
tseaver Jun 27, 2016
498271e
Add V2 Instance type.
tseaver Jun 27, 2016
e144b09
Convert instance admin to V2 protobufs.
tseaver Jun 28, 2016
9880a42
Adjust docs for V2 instance admin patterns.
tseaver Jun 28, 2016
90cd2e7
Merge pull request #1923 from tseaver/bigtable-v2-instance_admin
tseaver Jun 28, 2016
aa95442
Use 'grpcio-tools' in a virtualenv to generate latest Bigtable V2 pro…
tseaver Jun 28, 2016
d97d030
Remove duplicated line.
tseaver Jun 28, 2016
4a223b7
Use 'grpcio-tools' in a virtualenv to generate from latest Bigtable V…
tseaver Jun 28, 2016
568a463
Use 'grpcio-tools' in a virtualenv to generate from latest datastore …
tseaver Jun 28, 2016
24989d6
Accomodate final V2 renames: 'name' -> 'parent'.
tseaver Jun 28, 2016
6d3ff2e
Merge pull request #1925 from tseaver/bigtable-v2-generate_v2_protos_…
tseaver Jun 28, 2016
a488a8a
More 'cluster' -> 'instance' renames.
tseaver Jun 28, 2016
7214045
Use (same) actual V2 admin endpoint host for instance / table admin.
tseaver Jun 28, 2016
c6b747e
Use class-level constants.
tseaver Jun 28, 2016
40f609e
Add 'Client.list_instances' method.
tseaver Jun 28, 2016
c87ca51
Clean out remaining refs to V1-generated code.
tseaver Jun 28, 2016
0ac2def
Merge pull request #1926 from tseaver/bigtable-v2-cleanups
tseaver Jun 28, 2016
3467d99
Work around borked gRPC generation for g.longrunning.operations.
tseaver Jun 28, 2016
784fe01
Revise Bigtable system tests to fit instance-based patterns.
tseaver Jun 28, 2016
024c7f3
Plumb cluster 'location'/'serve_nodes' through instance creation.
tseaver Jun 28, 2016
5e85b5d
CreateInstance operation name includes location.
tseaver Jun 28, 2016
d2c9daf
'Row/ColumnFamily._table' has '_instance'.
tseaver Jun 28, 2016
a01d02e
V2 UpdateTable no longer returns operation.
tseaver Jun 28, 2016
9bdeef7
'PartialRowData' no longer keeps 'committed' state.
tseaver Jun 28, 2016
893d451
Typo fix.
tseaver Jun 28, 2016
d8a01c2
Switch to passing location ID, rather than full path, to instance.
tseaver Jun 28, 2016
6f4aced
Lint fixes.
tseaver Jun 28, 2016
0c625c5
Merge pull request #1930 from tseaver/bigtable-v2-system_tests
tseaver Jun 29, 2016
82adc5e
Adjust to V2 'instance'-based patterns.
tseaver Jun 28, 2016
40b2016
Rename testcases for clarity.
tseaver Jun 28, 2016
9dfa823
'Table.read_rows': no responses -> None.
tseaver Jun 28, 2016
e92da5c
V2 wraps row for 'ReadModifyWriteRow' request in new response message
tseaver Jun 28, 2016
0e80bdd
happybase.Table.scan: proocess all rows from a single request.
tseaver Jun 28, 2016
d18a316
Valet service.
tseaver Jun 29, 2016
d085294
Merge pull request #1931 from tseaver/bigtable-v2-happybase
tseaver Jun 29, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Convert to non-instance-admin Bigtable V2 protos.
Folds new ReadRowsResponse logic from #1907, #1915 into table row
handling.
  • Loading branch information
tseaver committed Jun 27, 2016
commit 1b0fb536715a05aab0100b81d85e083e36df057d
40 changes: 20 additions & 20 deletions gcloud/bigtable/row.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,10 @@
from gcloud._helpers import _datetime_from_microseconds
from gcloud._helpers import _microseconds_from_datetime
from gcloud._helpers import _to_bytes
from gcloud.bigtable._generated import (
bigtable_data_pb2 as data_v1_pb2)
from gcloud.bigtable._generated import (
bigtable_service_messages_pb2 as messages_v1_pb2)
from gcloud.bigtable._generated_v2 import (
data_pb2 as data_v2_pb2)
from gcloud.bigtable._generated_v2 import (
bigtable_pb2 as messages_v2_pb2)


_PACK_I64 = struct.Struct('>q').pack
Expand Down Expand Up @@ -134,13 +134,13 @@ def _set_cell(self, column_family_id, column, value, timestamp=None,
# Truncate to millisecond granularity.
timestamp_micros -= (timestamp_micros % 1000)

mutation_val = data_v1_pb2.Mutation.SetCell(
mutation_val = data_v2_pb2.Mutation.SetCell(
family_name=column_family_id,
column_qualifier=column,
timestamp_micros=timestamp_micros,
value=value,
)
mutation_pb = data_v1_pb2.Mutation(set_cell=mutation_val)
mutation_pb = data_v2_pb2.Mutation(set_cell=mutation_val)
self._get_mutations(state).append(mutation_pb)

def _delete(self, state=None):
Expand All @@ -156,8 +156,8 @@ def _delete(self, state=None):
:param state: (Optional) The state that is passed along to
:meth:`_get_mutations`.
"""
mutation_val = data_v1_pb2.Mutation.DeleteFromRow()
mutation_pb = data_v1_pb2.Mutation(delete_from_row=mutation_val)
mutation_val = data_v2_pb2.Mutation.DeleteFromRow()
mutation_pb = data_v2_pb2.Mutation(delete_from_row=mutation_val)
self._get_mutations(state).append(mutation_pb)

def _delete_cells(self, column_family_id, columns, time_range=None,
Expand Down Expand Up @@ -188,10 +188,10 @@ def _delete_cells(self, column_family_id, columns, time_range=None,
"""
mutations_list = self._get_mutations(state)
if columns is self.ALL_COLUMNS:
mutation_val = data_v1_pb2.Mutation.DeleteFromFamily(
mutation_val = data_v2_pb2.Mutation.DeleteFromFamily(
family_name=column_family_id,
)
mutation_pb = data_v1_pb2.Mutation(delete_from_family=mutation_val)
mutation_pb = data_v2_pb2.Mutation(delete_from_family=mutation_val)
mutations_list.append(mutation_pb)
else:
delete_kwargs = {}
Expand All @@ -207,9 +207,9 @@ def _delete_cells(self, column_family_id, columns, time_range=None,
family_name=column_family_id,
column_qualifier=column,
)
mutation_val = data_v1_pb2.Mutation.DeleteFromColumn(
mutation_val = data_v2_pb2.Mutation.DeleteFromColumn(
**delete_kwargs)
mutation_pb = data_v1_pb2.Mutation(
mutation_pb = data_v2_pb2.Mutation(
delete_from_column=mutation_val)
to_append.append(mutation_pb)

Expand Down Expand Up @@ -389,7 +389,7 @@ def commit(self):
if num_mutations > MAX_MUTATIONS:
raise ValueError('%d total mutations exceed the maximum allowable '
'%d.' % (num_mutations, MAX_MUTATIONS))
request_pb = messages_v1_pb2.MutateRowRequest(
request_pb = messages_v2_pb2.MutateRowRequest(
table_name=self._table.name,
row_key=self._row_key,
mutations=mutations_list,
Expand Down Expand Up @@ -504,14 +504,14 @@ def commit(self):
'mutations and %d false mutations.' % (
MAX_MUTATIONS, num_true_mutations, num_false_mutations))

request_pb = messages_v1_pb2.CheckAndMutateRowRequest(
request_pb = messages_v2_pb2.CheckAndMutateRowRequest(
table_name=self._table.name,
row_key=self._row_key,
predicate_filter=self._filter.to_pb(),
true_mutations=true_mutations,
false_mutations=false_mutations,
)
# We expect a `.messages_v1_pb2.CheckAndMutateRowResponse`
# We expect a `.messages_v2_pb2.CheckAndMutateRowResponse`
client = self._table._cluster._client
resp = client._data_stub.CheckAndMutateRow(
request_pb, client.timeout_seconds)
Expand Down Expand Up @@ -701,7 +701,7 @@ def append_cell_value(self, column_family_id, column, value):
"""
column = _to_bytes(column)
value = _to_bytes(value)
rule_pb = data_v1_pb2.ReadModifyWriteRule(
rule_pb = data_v2_pb2.ReadModifyWriteRule(
family_name=column_family_id,
column_qualifier=column,
append_value=value)
Expand Down Expand Up @@ -738,7 +738,7 @@ def increment_cell_value(self, column_family_id, column, int_value):
will fail.
"""
column = _to_bytes(column)
rule_pb = data_v1_pb2.ReadModifyWriteRule(
rule_pb = data_v2_pb2.ReadModifyWriteRule(
family_name=column_family_id,
column_qualifier=column,
increment_amount=int_value)
Expand Down Expand Up @@ -794,12 +794,12 @@ def commit(self):
if num_mutations > MAX_MUTATIONS:
raise ValueError('%d total append mutations exceed the maximum '
'allowable %d.' % (num_mutations, MAX_MUTATIONS))
request_pb = messages_v1_pb2.ReadModifyWriteRowRequest(
request_pb = messages_v2_pb2.ReadModifyWriteRowRequest(
table_name=self._table.name,
row_key=self._row_key,
rules=self._rule_pb_list,
)
# We expect a `.data_v1_pb2.Row`
# We expect a `.data_v2_pb2.Row`
client = self._table._cluster._client
row_response = client._data_stub.ReadModifyWriteRow(
request_pb, client.timeout_seconds)
Expand All @@ -814,7 +814,7 @@ def commit(self):
def _parse_rmw_row_response(row_response):
"""Parses the response to a ``ReadModifyWriteRow`` request.

:type row_response: :class:`.data_v1_pb2.Row`
:type row_response: :class:`.data_v2_pb2.Row`
:param row_response: The response row (with only modified cells) from a
``ReadModifyWriteRow`` request.

Expand Down
Loading