Focus point distance filter

[orangejulius]

This PR builds on the work in #1215 to improve the performance of autocomplete queries with the focus.point parameter, while ensuring the results are as similar as possible.

It works by adding an Elasticsearch filter to autocomplete queries. Results must now pass one of the following tests to be scored and shown:

1. It must be in any layer except address or street (calculated in a performance-friendly way by enumerating all other layers)
2. or, it must be within a certain distance from the focus point

The allowed distance grows with the text length, and after some experimentation I settled on a hand tuned table of distances. As it stands the filter is active up to text lengths of 8, although the distance limit is a generous 2000km by then.

Currently, addresses and streets represent 565 million of the 605 million records in a full planet index, or about 93%. So in some cases this can filter out lots of results.

There are zero changes at all to either the acceptance tests or autocomplete acceptance tests, which hopefully confirms this change is pretty conservative. I feel like that's a great way for it to start out, and then we can tweak it more over time.

Potential future work

There are several things this PR doesn't try to tackle at all but that we should do in the future, using the groundwork laid here.

I actually really like the filter pattern used, with several clauses in the should block and minimum_should_match set to one. This basically means we could easily add more conditions and as long as any one of them is true, the record will not be filtered out

Popularity

Once we have more popularity values from work like pelias/openstreetmap#493 we could experiment with filtering out far away results below a popularity threshold. This would allow us to filter out some of the ~30 million venues, and possibly some of the 6.5M localities, currently in the index, providing another nice boost to performance.

Zoom

The example of searching for addresses in Australian coastal cities with a viewport centered on the middle of Australia was raised in pelias/pelias#658. This PR doesn't make that case worse, but it doesn't help either.

However, if we did introduce a zoom parameter, we could consider having it modify the distance limits so that the entire viewport was likely to be within that limit.

[orangejulius]

We have been running this on some of our geocode.earth traffic for the last 24 hours. The results are in and they are...interesting:

First, there is a nice decrease in both p99 latency and the number of Elasticsearch hits (as expected), but there is also a notable increase in the p50 latency. I wonder if the addition of the geo_distance filter is making the "base" cost for the queries a bit higher, even if it ensures the slower queries are a lot less slow.

[orangejulius]

Hello again!
We've performed some new analysis of this change. At first (as shown above) it looked like this change really was either a wash when it came to performance or was actually detrimental. However, we failed to take into account that this change only affects focus point queries with text input lengths less than or equal to 8 characters. The initial analysis looked at all autocomplete queries.

Filtering out all autocomplete queries without a focus point, and for text lengths 9 or greater, a very different picture emerges:

There's a dramatic decrease in 99th percentile latency, a notable but allowable increase in p50 latency, and a possibly insignificant increase in average latency.

So it looks like this change does do a great job of cutting off the long tail of slow autocomplete queries, without increasing the total workload on Elasticsearch, and without unduly slowing down already fast autocomplete queries.

With that, I think it's good to go, although I'm sure we will tweak it later.