Retry did not occurred

[nitishxp]

Hi,

I got the following error in one of my Cloud function

, in reraise raise self.last_attempt.result()
File "/opt/python3.8/lib/python3.8/concurrent/futures/_base.py", line 437, in result return self.__get_result()
File "/opt/python3.8/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result raise self._exception
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/tenacity/__init__.py", line 426, in __call__ result = fn(*args, **kwargs)
File "/workspace/cf/functions.py", line 602, in execute_query return self.gbq.query(query).result()
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/cloud/bigquery/job/query.py", line 1172, in result super(QueryJob, self).result(retry=retry, timeout=timeout)
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/cloud/bigquery/job/base.py", line 679, in result return super(_AsyncJob, self).result(timeout=timeout, **kwargs)
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/api_core/future/polling.py", line 134, in result raise
  self._exception google.api_core.exceptions.InternalServerError: 500 Job failed because streaming data from one of the tables referenced in the query is temporarily unavailable. Retrying the job may solve the problem

[parthea]

@tseaver Please could you take a look?

[tseaver]

@nitishxp Although google-api-core is involved in your traceback (which I reformatted for clarity), this issue seems to be specific to google-cloud-bigquery. I am moving the issue there, where you can get better support for your question.

[tseaver]

@plamut, @tswast PTAL.

[plamut]

@nitishxp By default, InternalServerError is is listed among the retriable errors. Ditto if the exception instance contains a reason sent by the backend, "internalError" is one of them.

Does the error occur often, or was it just a one-time event_

Is your code perhaps using a custom Retry object? (a sample would be great) And which package versions do you have installed? Is there anything else in the logs that might lead to the eventual error posted here?

Any additional info would be useful, thanks!

[nitishxp]

@plamut It was a one time exception that occurred in Cloud Function.

Yes I am also using tenacity retry module.

Below are installed packages

google-cloud-bigquery==2.15.0
google-auth==1.30.0
google-cloud-pubsub==2.2.0
google-cloud-scheduler==2.1.0
google-cloud-storage==1.38.0
google-cloud-kms==2.2.0
google-cloud-secret-manager==2.1.0
google-api-python-client==2.7.0

[tswast]

From your stacktrace "500 Job failed because streaming data from one of the tables referenced in the query is temporarily unavailable. Retrying the job may solve the problem"

Retrying the API request won't help. You actually have to re-issue the query.

Closing as a duplicate of #539

[nitishxp]

@tswast will the sdk take care of it in the future releases?

[tswast]

@nitishxp It's a really hard problem to solve. I don't expect the "query" method to handle this anytime soon.

It may be a bit easier to implement in wrapper libraries with a simpler interface like the DB-API, SQLAlchemy, and/or pandas-gbq.