feat: Enable ca-west-1 and il-central-1

[astuyve]

What does this PR do?

This will cause our CI to publish 8 layers up to 90 versions in two new AWS regions.

Motivation

Fixes #420

Testing Guidelines

Additional Notes

Types of Changes

• Bug fix
• New feature
• Breaking change
• Misc (docs, refactoring, dependency upgrade, etc.)

Check all that apply

• This PR's description is comprehensive
• This PR contains breaking changes that are documented in the description
• This PR introduces new APIs or parameters that are documented and unlikely to change in the foreseeable future
• This PR impacts documentation, and it has been updated (or a ticket has been logged)
• This PR's changes are covered by the automated tests
• This PR collects user input/sensitive content into Datadog
• This PR passes the integration tests (ask a Datadog member to run the tests)

[joeyzhao2018]

I actually think it's safer to just manually do the catch up. And then we enable the new regions... How confident we are about CI doing catch up deployments?

[astuyve]

How confident we are about CI doing catch up deployments?

Not confident.

But that's okay, this is how we'll find out.

This release includes no code changes and only handles these two regions so the downside risk is low. The risk to other regions is low.

If needed we can re-execute the build and re-run the publish scripts. Let's consider the failure modes.

1. (best case) - temporary credentials time out and we need to re-run the individual step for each new region and each runtime/architecture combo. This touches no other regions and allows us to re-execute. This should be pretty painless.
2. Hypothetically let's say the artifact times out of s3 (it's a 24hr window). Then we'd have to re-build it. That's okay too because the jobs for other regions are re-entrant. We do not automatically version ++ for production releases, we target an exact version and only publish up to that point, so those other matrix jobs will succeed and be no-ops
3. Let's say this all falls apart and we mismatch versions across regions. We can still version + 2 manually for all regions with no downside risk, as this is effectively equivalent to a regular release. Versions behind would just get +2 versions (which is our normal catch-up behavior), and the versions which had an extra release will simply +1.

Ultimately we need to get to a place where this happens automatically otherwise we have downstream risks like APIs timing out and mispublishing incompatible arm/x86 versions. I don't believe we have a choice other than to try this.

[joeyzhao2018]

ok, let's give it a shot. I think I used a wrong word "safer". I was thinking more like "easier"...

[astuyve]

I think your wording, reasoning, and caution are correct and well grounded. I also think we're obliged to make this work in CI anyway, so I've de-risked this as much as possible. I will of course watch the deployment very closely.