gRPC Calls to Bigtable & Datastore fail

[mathiasverhoeven]

Using:

gcloud==0.18.1
grpcio==1.0.0

All the RPC calls we do on Bigtable via the Python client seem to fail, e.g. listing the instances of a project returns:

In [1]: from gcloud import bigtable

In [2]: client = bigtable.Client(project=u'project-id', admin=True)

In [3]: client.start()

In [4]: client.is_started()
Out[4]: True

In [4]: client.list_instances()
AbortionError: AbortionError(code=StatusCode.INTERNAL, details="{"created":"@1472199496.200250000","description":"RST_STREAM","file":"src/core/ext/transport/chttp2/transport/frame_rst_stream.c","file_line":107,"http2_error":1}")

We've tried downgrading the gcloud or the grpcio library but without success. Also, the Bigtable Golang client seems to function correctly.

Edit: The issue is also present when querying Datastore over GRPC.

[sduskis]

Other folks are seeing this problem as well. We are actively investigating this issue.

[vlandham]

Also seeing this. Started yesterday/last night as far as I can tell.

We attempted to move back to

grpcio (1.0.0rc2)

with no improvement.

Thanks!

[sduskis]

We think we found the culprit. We are rolling out a server side fix now.

[tseaver]

@sduskis Great, thank you for the quick resolution!

[sduskis]

@tseaver: I think that there's follow up work. gRPC should handle RST_STREAM more gracefully.

[lukesneeringer]

I am assuming this must be out of date. Please re-open if it is still an active issue.