exporter: fix snapshot GC race during image unpack

[tonistiigi]

Use ApplyLayers instead of per layer ApplyLayer loop to allow recursive parent rebuild when GC collects a parent snapshot between Stat and Prepare calls.

Pre-lease the top chain ID snapshot before calling ApplyLayers so that GC cannot collect it during the Stat shortcut path, which does not add snapshots to the lease.

Fixes #6521

@rfay @stasadev @dmcg

Afaics, both containerd rootfs.ApplyLayer and rootfs.ApplyLayers are unsafe. The former can cause this "parent snapshot does not exist" that was reported here, and the second can return a chainid that may already be deleted by the time the function returns.

[stasadev]

Thank you very much @tonistiigi !

I confirm that this fix works. I ran the repro https://github.com/rfay/buildx-6521-repro for 30 minutes without any errors (it usually failed after ~3-5 minutes before).

I tested in on Arch-based Linux by patching the docker package with:

diff --git a/PKGBUILD b/PKGBUILD
index 1f05678..945c246 100644
--- a/PKGBUILD
+++ b/PKGBUILD
@@ -20,11 +20,13 @@ _TINI_COMMIT=de40ad007797e0dcd8b7126f27bb87401d224240
 source=("git+https://github.com/docker/cli.git#tag=v$pkgver"
         "git+https://github.com/moby/moby.git#tag=docker-v$pkgver"
         "git+https://github.com/krallin/tini.git#commit=$_TINI_COMMIT"
-        "$pkgname.sysusers")
+        "$pkgname.sysusers"
+        "https://patch-diff.githubusercontent.com/raw/moby/buildkit/pull/6558.patch")
 sha256sums=('b3e8d552827376a3ac74ac48d6a31cba7d61d30330f562f0db833e62c739bcc2'
             '654584d3b1b3a890a9af4aee9e23182d10f7c0b15bcff92d2d211a9892ed9e63'
             '28a6641d508f60d47315efb3c85d97360188750a45bd6d3c8737d3f1a2b44121'
-            '541826011a9836d05a2f42293d5f1beadf2ca8d89fb604487d61a013505678eb')
+            '541826011a9836d05a2f42293d5f1beadf2ca8d89fb604487d61a013505678eb'
+            'SKIP')
 
 # create a fake go path directory and pushd into it
 # $1 real directory
@@ -40,6 +42,10 @@ _fake_gopath_popd() {
   popd >/dev/null
 }
 
+prepare() {
+  patch -Np1 -d moby/vendor/github.com/moby/buildkit < 6558.patch
+}
+
 build() {
   ### check my mistakes on commit version
   echo 'Checking commit mismatch'

[stasadev]

I ran it for another 3 hours and everything was fine.