fix: do not add space to location claim for legacy provisioned spaces by alanshaw · Pull Request #178 · storacha/upload-service

alanshaw · 2025-03-11T12:36:46Z

Problem

⚠️🚦 Serving content from the gateway 🚦⚠️

I've realized that we're using the presence of space in location claims to activate access control authorization checks.

When we deploy the consolidated upload-service repos ALL location claims will have a space - we lose the code path where we can add the property or not, since it is in the storage node.

😱 😱 😱

Proposal

There are two ways to store content. If your space is provisioned by did:web:up.storacha.network then it goes to a storage node, if it has been provisioned by did:web:web3.storage then the upload service will store the data.

If the upload service stores the data we use the “old flow”, i.e. web3.storage/blob/allocate and web3.storage/blob/accept. We store the data in carpark as before and we generate the location commitment as before (without a space). The location commitment is published to the content-claims service as before, so that it can appear in (legacy) results from the indexing service. It means we have to keep the content claims service running a bit longer (writes) 😢.

The blob/index/add invocation will invoke the new indexer even for legacy space data. This allows us to still move away from writing to the legacy dynamodb block index.

I think this is largely in line with what we were originally thinking…and allows us to keep space in location commitment as indication of required content serve delegation.

This PR

In this PR I've added back the web3.storage/blob/* handlers, with a few modifications to allow them to be called directly from the space/blob/add handler and the ucan/conclude for http/put handler if we detect that the space is provisioned with did:web:web3.storage.

alanshaw · 2025-03-12T12:15:54Z

packages/upload-api/src/blob/add.js

-  )
-  if (provisioned.error) {
-    return provisioned
-  }


Note: this is actually a test for whether the space has been provisioned. It moved into the main handler so we can invoke the legacy blob/add handler if the space is provisioned with the legacy service DID.

Question: Does this mean we will only verify whether the space has been provisioned when attempting to space/blob/add something? Then we assume the space is always provisioned for the allocate invocation?

blob/allocate can only be invoked by the upload service, it is done as part of the space/blob/add invocation after we check if the space is provisioned.

So this chunk of code was previously in a function called allocate, which is called from the space/blob/add handler. It is NOT in the blob/allocate handler. It couldn't be because the audience of blob/allocate is a storage node and they should have no authority over determining whether a space is provisioned or not.

So, to summarize, there's no material change here - I just moved where the code was so that we can switch on the provisioned service DID and call the legacy space/blob/add if it's a legacy space.

sounds good! thanks for clarifying that.

alanshaw · 2025-03-14T14:21:05Z

packages/cli/tsconfig.json

    "noUnusedLocals": true,
    "noUnusedParameters": false,
    // advanced
-    "importsNotUsedAsValues": "remove",


This is the default value anyway, and I was seeing errors related to this...

Data will never be updated until keccake256@gmail.com or keccake256

alanshaw · 2025-03-14T14:23:16Z

packages/w3up-client/test/legacy-compat.node.test.js

+        legacy: { client: true, space: true },
+      }),
+  }),
+}


These new tests ensure that we can upload with legacy space + legacy client, legacy space + new client and new space + legacy client.

…acha/filecoin-client@1.0.4, @storacha/upload-api@1.3.8, @storacha/cli@1.1.14, @storacha/blob-index@1.0.3, @storacha/access@1.0.5, @storacha/ui-react@2.5.12, @storacha/filecoin-api@1.1.4, @storacha/client@1.1.9, @storacha/upload-client@1.0.7, @storacha/ui-example-react-multi-file-upload@0.0.4, @storacha/ui-example-react-uploads-list@0.0.4, @storacha/ui-example-react-file-upload@0.0.4, @storacha/ui-example-react-components@1.2.4, @storacha/ui-example-react-sign-up-in@0.0.4, @storacha/ui-example-test-playwright@1.0.10 (#185) ## @storacha/ui-react@2.5.12 (2025-03-14) ### 🧱 Updated Dependencies - Updated @storacha/ui-core to 2.4.17 ## @storacha/ui-core@2.4.17 (2025-03-14) ### 🧱 Updated Dependencies - Updated @storacha/access to 1.0.5 - Updated @storacha/client to 1.1.9 ## @storacha/filecoin-client@1.0.4 (2025-03-14) ### 🧱 Updated Dependencies - Updated @storacha/capabilities to 1.2.4 ## @storacha/access@1.0.5 (2025-03-14) ### 🧱 Updated Dependencies - Updated @storacha/capabilities to 1.2.4 ## @storacha/upload-client@1.0.7 (2025-03-14) ### 🩹 Fixes - fix: do not add space to location claim for legacy provisioned spaces ([#178](#178)) ### 🧱 Updated Dependencies - Updated @storacha/filecoin-client to 1.0.4 - Updated @storacha/capabilities to 1.2.4 - Updated @storacha/blob-index to 1.0.3 ### ❤️ Thank You - ash ## @storacha/capabilities@1.2.4 (2025-03-14) ### 🩹 Fixes - fix: do not add space to location claim for legacy provisioned spaces ([#178](#178)) ### ❤️ Thank You - ash ## @storacha/filecoin-api@1.1.4 (2025-03-14) ### 🧱 Updated Dependencies - Updated @storacha/filecoin-client to 1.0.4 - Updated @storacha/capabilities to 1.2.4 ## @storacha/client@1.1.9 (2025-03-14) ### 🧱 Updated Dependencies - Updated @storacha/filecoin-client to 1.0.4 - Updated @storacha/access to 1.0.5 - Updated @storacha/upload-client to 1.0.7 - Updated @storacha/capabilities to 1.2.4 - Updated @storacha/blob-index to 1.0.3 - Updated @storacha/upload-api to 1.3.8 ## @storacha/blob-index@1.0.3 (2025-03-14) ### 🧱 Updated Dependencies - Updated @storacha/capabilities to 1.2.4 ## @storacha/upload-api@1.3.8 (2025-03-14) ### 🩹 Fixes - fix: do not add space to location claim for legacy provisioned spaces ([#178](#178)) ### 🧱 Updated Dependencies - Updated @storacha/access to 1.0.5 - Updated @storacha/capabilities to 1.2.4 - Updated @storacha/filecoin-api to 1.1.4 - Updated @storacha/blob-index to 1.0.3 ### ❤️ Thank You - ash ## @storacha/cli@1.1.14 (2025-03-14) ### 🧱 Updated Dependencies - Updated @storacha/access to 1.0.5 - Updated @storacha/capabilities to 1.2.4 - Updated @storacha/client to 1.1.9 - Updated @storacha/upload-api to 1.3.8 Co-authored-by: Rachabot <rachabot@storacha.network>

### Context - Error while creating a new space in production: > create:1 Access to fetch at 'https://w3s.link/' from origin 'https://console.storacha.network' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: Redirect is not allowed for a preflight request. ### Issue - I was using the `w3s.link` as the gateway host to be authorized to serve the space content, but the cors policy defined in the `w3s.link` doesn't allow redirects for a preflight request. - It doesn't happen in staging because in staging I am using the actual gateway, and not the proxy: `freeway-staging.dag.haus`. ### Fix - Authorize the actual production gateway instead of the proxy: `https://freeway.dag.haus`.

🤖 I have created a release *beep* *boop* --- ## [1.17.3](storacha/console@w3console-v1.17.2...w3console-v1.17.3) (2025-01-09) ### Bug Fixes * cors preflight issue in production ([#178](storacha/console#178)) ([1b1b79a](storacha/console@1b1b79a)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

🤖 I have created a release *beep* *boop* --- ## [1.18.0](console-v1.17.6...console-v1.18.0) (2025-04-16) ### Features * Actually put error in console, as promised ([#131](#131)) ([2f438d5](2f438d5)) * add a PNG version of our logo ([#153](#153)) ([52a7854](52a7854)) * add old web3 storage migrator ([#129](#129)) ([8cbd2bb](8cbd2bb)) * add Sentry error tracking ([#165](#165)) ([5188e07](5188e07)) * change email ([b624f1d](b624f1d)) * **copy:** Add missing comma ([#115](#115)) ([6e231eb](6e231eb)) * Improved layout styling, and other tales ([#114](#114)) ([5484d36](5484d36)) * more styles ([a9f0172](a9f0172)) * point staging env referral link at staging marketing site ([0073963](0073963)) * prototype of referral system in console ([#142](#142)) ([d2b284b](d2b284b)) * referrals tweaks ([#155](#155)) ([32cc78f](32cc78f)) * **repo:** merge main from https://github.com/storacha/console ([152954e](152954e)) * share spaces via email ([#123](#123)) ([b0dd61a](b0dd61a)) * styles for upload page ([9cdac7c](9cdac7c)) * trigger deploy ([a5ae681](a5ae681)) * update TOS links to point to the Storacha docs ([#148](#148)) ([7d5ad13](7d5ad13)) * Use Blob protocol ([#132](#132)) ([0fd47ba](0fd47ba)) * Use latest `w3ui` to support reloading spaces ([#138](#138)) ([5a886bf](5a886bf)) * wording ([032218e](032218e)) ### Fixes * a little package.json cleanup ([13859fc](13859fc)) * add missing receiptsEndpoint to W3UIProvider ([#144](#144)) ([e8ca3d3](e8ca3d3)) * add more deps to the catalog and pin eslint typescript plugin ([e00af97](e00af97)) * add sentry auth token to enable sourcemap upload ([#168](#168)) ([bec52c9](bec52c9)) * apostrophe ([ee51ff9](ee51ff9)) * capture some errors we were logging to console ([#188](#188)) ([673043a](673043a)) * catalog more dependencies ([cc52f81](cc52f81)) * check ref ([b1c985f](b1c985f)) * **console:** pnpm run dev + nx configs(sentry,tailwind) ([ad5db7e](ad5db7e)) * cors preflight issue in production ([#178](#178)) ([583abb3](583abb3)) * create space ([#173](#173)) ([7ac4478](7ac4478)) * create space using the latest w3up-client version ([#171](#171)) ([5c67bdb](5c67bdb)) * **createSpace:** using the latest w3ui with w3up-client ([#176](#176)) ([fdd5895](fdd5895)) * deploy-storacha.yml needs to use the new main branch ([#122](#122)) ([3240385](3240385)) * deploy.yml needs to use the old-main branch ([#120](#120)) ([6252d89](6252d89)) * fix linting ([0c5f06b](0c5f06b)) * gateway authorization based on env ([#174](#174)) ([f446ab9](f446ab9)) * Humanode config tweaks ([b3debc4](b3debc4)) * import space preview ([#127](#127)) ([d935f5f](d935f5f)) * init page loading right after button click + stop on errors ([220df2d](220df2d)) * is this enough? ([ac552ac](ac552ac)) * lock down next and react versions ([4d316af](4d316af)) * make it possible to skip the plan gate after Stripe checkout ([#162](#162)) ([e50d81f](e50d81f)) * **prod:** gateway host ([#180](#180)) ([14c8c38](14c8c38)) * quotes ([9fb1aba](9fb1aba)) * remove console github configs ([b5a2307](b5a2307)) * remove console github configs ([#227](#227)) ([71db228](71db228)) * remove referrals feature flag ([#161](#161)) ([9c774dd](9c774dd)) * remove sentry test page ([b442306](b442306)) * remove sentry test page ([#231](#231)) ([f4071dc](f4071dc)) * Revert "feat: Use Blob protocol" ([#134](#134)) ([9611be0](9611be0)) * Revert "fix: Revert "feat: Use Blob protocol"" (😩) ([#136](#136)) ([2ec041b](2ec041b)) * set ipfs gateway url based on env var ([#177](#177)) ([49727f7](49727f7)) * share space via ucan file ([#126](#126)) ([2b7b623](2b7b623)) * **space-creator:** check provision result before proceeding ([#119](#119)) ([92e087a](92e087a)) * **staging:** gateway host ([794a856](794a856)) * trigger a console release ([b9862a1](b9862a1)) * trigger a console release ([#228](#228)) ([d9677d8](d9677d8)) * tweak plan gate query param ([#163](#163)) ([2346d15](2346d15)) * typo on space page ([#187](#187)) ([12e2b02](12e2b02)) * update packages to use catalog dependencies ([9191594](9191594)) * use correct branch name ([374fb0c](374fb0c)) * use main branch ([ea0a170](ea0a170)) * use the correct referral URL ([#149](#149)) ([728ae7e](728ae7e)) * use the same Sentry project for different envs ([#167](#167)) ([1bf2d13](1bf2d13)) * Use Ucanto 10 across the board ([#140](#140)) ([b38477a](b38477a)) * using the correct gateway host ([#183](#183)) ([83e0582](83e0582)) ### Other Changes * add storacha deploy action ([617b096](617b096)) * import console ([#139](#139)) ([59ebc4e](59ebc4e)) * **main:** release client 1.0.6 ([27cb383](27cb383)) * **main:** release client 1.0.6 ([#104](#104)) ([07f27a2](07f27a2)) * **main:** release upload-api 1.1.8 ([aec53e7](aec53e7)) * **main:** release upload-api 1.1.8 ([#103](#103)) ([e71494a](e71494a)) * **main:** release w3console 1.10.0 ([#125](#125)) ([d7484f7](d7484f7)) * **main:** release w3console 1.11.0 ([#128](#128)) ([4a9cd39](4a9cd39)) * **main:** release w3console 1.12.0 ([#130](#130)) ([c1b894d](c1b894d)) * **main:** release w3console 1.13.0 ([#133](#133)) ([a5cdbf5](a5cdbf5)) * **main:** release w3console 1.13.1 ([#135](#135)) ([0cf946d](0cf946d)) * **main:** release w3console 1.14.0 ([#137](#137)) ([928cd1e](928cd1e)) * **main:** release w3console 1.15.0 ([#139](#139)) ([18c33d3](18c33d3)) * **main:** release w3console 1.15.1 ([#141](#141)) ([d82d22e](d82d22e)) * **main:** release w3console 1.15.2 ([#145](#145)) ([810224b](810224b)) * **main:** release w3console 1.16.0 ([#146](#146)) ([2a7dcaa](2a7dcaa)) * **main:** release w3console 1.16.1 ([#160](#160)) ([145b288](145b288)) * **main:** release w3console 1.16.2 ([#164](#164)) ([c3ea3be](c3ea3be)) * **main:** release w3console 1.17.0 ([#166](#166)) ([0ea9a86](0ea9a86)) * **main:** release w3console 1.17.1 ([#169](#169)) ([7a77c70](7a77c70)) * **main:** release w3console 1.17.2 ([#172](#172)) ([a118e11](a118e11)) * **main:** release w3console 1.17.3 ([#179](#179)) ([7c1948e](7c1948e)) * **main:** release w3console 1.17.4 ([#181](#181)) ([217cd1e](217cd1e)) * **main:** release w3console 1.17.5 ([#182](#182)) ([9db003e](9db003e)) * **main:** release w3console 1.17.6 ([#185](#185)) ([152e230](152e230)) * nx sync ([5933677](5933677)) * Set `packageManager` ([#117](#117)) ([a8c7c93](a8c7c93)) * trigger deployment ([8051d81](8051d81)) * update copy ([#159](#159)) ([60250f1](60250f1)) * update deploy name ([184fe57](184fe57)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

alanshaw commented Mar 12, 2025

View reviewed changes

alanshaw requested review from a team and Peeja and removed request for Peeja March 13, 2025 22:32

alanshaw added 6 commits March 14, 2025 10:29

fix: do not add space to location claim for legacy provisioned spaces

bad9e57

fix: build

264a2fc

fix: add back legacy w3s handlers and dependencies

a68e575

chore: appease linter

861c25c

chore: appease linter

f3b93d8

fix: remove config with default param

c6b9652

alanshaw force-pushed the fix/do-not-add-space-in-location-claim-for-legacy-provisioned-spaces branch from a8c1ea2 to c6b9652 Compare March 14, 2025 14:17

alanshaw commented Mar 14, 2025

View reviewed changes

fforbeck approved these changes Mar 14, 2025

View reviewed changes

alanshaw merged commit 7d413a3 into main Mar 14, 2025
1 check passed

alanshaw deleted the fix/do-not-add-space-in-location-claim-for-legacy-provisioned-spaces branch March 14, 2025 20:00

github-actions bot mentioned this pull request Apr 16, 2025

chore(main): release console 1.18.0 #229

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: do not add space to location claim for legacy provisioned spaces#178

fix: do not add space to location claim for legacy provisioned spaces#178
alanshaw merged 6 commits intomainfrom
fix/do-not-add-space-in-location-claim-for-legacy-provisioned-spaces

alanshaw commented Mar 11, 2025 •

edited

Loading

Uh oh!

alanshaw Mar 12, 2025

Uh oh!

fforbeck Mar 14, 2025

Uh oh!

alanshaw Mar 14, 2025

Uh oh!

alanshaw Mar 14, 2025

Uh oh!

fforbeck Mar 14, 2025

Uh oh!

alanshaw Mar 14, 2025

Uh oh!

Fool256 Aug 4, 2025

Uh oh!

alanshaw Mar 14, 2025 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

alanshaw commented Mar 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Proposal

This PR

Uh oh!

alanshaw Mar 12, 2025

Choose a reason for hiding this comment

Uh oh!

fforbeck Mar 14, 2025

Choose a reason for hiding this comment

Uh oh!

alanshaw Mar 14, 2025

Choose a reason for hiding this comment

Uh oh!

alanshaw Mar 14, 2025

Choose a reason for hiding this comment

Uh oh!

fforbeck Mar 14, 2025

Choose a reason for hiding this comment

Uh oh!

alanshaw Mar 14, 2025

Choose a reason for hiding this comment

Uh oh!

Fool256 Aug 4, 2025

Choose a reason for hiding this comment

Uh oh!

alanshaw Mar 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

alanshaw commented Mar 11, 2025 •

edited

Loading

alanshaw Mar 14, 2025 •

edited

Loading