[C++] Consolidate thirdparty bundle URLs, version bumping logic, etc

[asfimport]

This will prevent issues like ARROW-6406

After ARROW-8266 ensures every _ep has at least two URLs this problem becomes harder since we have a few classes of URL which don't necessarily overlap: preferred (highest priority during build configuration), canonical (whence new tarballs should be downloaded when versions get bumped), and backup. It's difficult to represent this in a txt file.

A script should be available for automatically bumping versions (a generalization of upload-boost.sh):

• update versions.txt, including checksums

• run download_dependencies.sh to get fresh archives

• run (or inline) trim-boost.sh to trim the boost archive

• update ursalabs-managed archives: https://github.com/ursa-labs/thirdparty/releases/tag/latest

  Checksums:

  If a checksum is not provided to ExternalProject_Add it may re-download a tarball even if that's not necessary (any time the generated build must be modified, IIUC). Ideally we should provide a checksum for all _eps (and not just Thrift) to cut down on unnecessary network access when building bundled.

  ARROW-8222 introduces a subtlety: we now default to a trimmed boost archive which contains only the 10% which we need, only falling back to a full boost archive when that fails to download. This is faster but not equivalent to the full boost archive so we can't provide a single checksum which matches both. This will probably just mean that those cases are extra slow

Reporter: Wes McKinney / @wesm

Related issues:

• [C++] Third party download URLs are duplicated (is duplicated by)
• [C++] Maintain bundled dependency tarballs in an Apache-managed location (relates to)
• [C++] Bump versions of bundled dependencies (relates to)

_{Note: This issue was originally created as ARROW-6407. Please see the migration documentation for further details.}

[asfimport]

Ben Kietzman / @bkietz:
Even better: this could be a GitHub action triggered by updates to versions.txt https://github.com/actions/upload-release-asset

[asfimport]

Antoine Pitrou / @pitrou:
cc @nealrichardson

[asfimport]

Benson Muite / @bkmgit:
Since it is good to have multiple download locations my initial preference is to setup an OSDN mirror of https://github.com/ursa-labs/thirdparty Other mirroring options are GitLab and Gitee

Downloads with different hashes seem unlikely to be supported in Cmake, so giving the user an option to download full Boost but leaving the trimmed Boost as default would allow keeping hashes which allow for better security.

[asfimport]

Ian Cook / @ianmcook:
There has been some recent discussion in of moving the bundled third-party dependencies mirror to an ASF-managed repository such as https://apache.jfrog.io/. I know that @bkmgit  and  @kou  are interested in seeing this happen. We should include that in the effort proposed here. [ Update: there is a separate Jira for this at ARROW-14336 ]

[asfimport]

Benson Muite / @bkmgit:
Thanks @ianmcook probably good to have two locations to download from as this is one of the reasons for failed tests.

[asfimport]

Todd Farmer / @toddfarmer:
This issue was last updated over 90 days ago, which may be an indication it is no longer being actively worked. To better reflect the current state, the issue is being unassigned. Please feel free to re-take assignment of the issue if it is being actively worked, or if you plan to start that work soon.

[github-actions]

This issue has been marked as stale because it has had no activity in the past 365 days. Please remove the stale label or comment below, or this issue will be closed in 14 days. If this improvement is still desired but has no current owner, please add the 'Status: needs champion' label.