OperationFuture: use exponential backoff in polling schedule

[garrettjonesgoogle]

Also consider an initial wait period that is relative to the size of the input (would be API-specific).

[pbogle]

Exponential backoff is important for the speech API. Our long-running operations can be very long running (potentially > 1 hour) so it is very inefficient to poll once per second. We have quotas for abuse preventation and polling thousands of times will cause quota issues for users.

[garrettjonesgoogle]

Marking as blocking beta for Speech.

[garrettjonesgoogle]

Oh, we don't have the specific API labels in gax-java. Oh well.

[jskeet]

.NET has the same issue - our LRO polling is constant-delay, which feels appropriate to me. The default delay is once every 10 seconds, across the board - it's possible that we should work out a way of letting each service specify a default.

Exponential backoffs typically have upper bounds, and I'd expect to reach that upper bound pretty quickly for speech - polling once per minute would mean only polling 60 times for an hour-long operation, which doesn't seem unreasonable to me... whereas a truly-exponential backoff could easily end up keeping the user waiting for half an hour longer than necessary.

Basically, I think we could do with more discussion of this, ideally thinking about other LROs beyond speech, too.

[shinfan]

It is reasonable to poll once per minute if the operation is about 1 hour. But it would be wasteful if the operation is longer (e.g. 10 hours). It looks reasonable to use constant delay as default for LRO (10 seconds interval sounds reasonable to me). To support cases such as Speech which requires exponential delay, we can add some gapic config fields to allow these APIs specify the default (exponential delay, upper bound).

[jskeet]

It's still not clear to me that Speech really requires an exponential delay - and any exponential delay is likely to have an upper bound anyway. Exponential delays are much more complicated to explain to users, and I really don't see the benefit.

The cost of polling once a minute (the rate I suggest for speech, with other APIs being different) is very small compared with what I'd expect the compute cost of the actual LRO to be.

[lukesneeringer]

Worth noting that the current linear delay can have a custom minimum, multiplier, and maximum per-service (e.g. it is set in the gapic configuration YAML for each service separately); it would be pretty easy to set a one or two minute maximum for Speech. (The default is 30 seconds, incidentally, which is not horrifically unreasonable for a one hour LRO.)

[pbogle]

Fixed interval polling is probably OK as long as you don't poll more than once a minute.

[garrettjonesgoogle]

Given that fixed interval polling is ok, I'm removing off the blocking label.