feat(utils): export resetFetchContext and clearFetchCache#1421
Merged
feat(utils): export resetFetchContext and clearFetchCache#1421
Conversation
…/fetch context The h1/h2 context created by @adobe/fetch has internal connection pooling and response caching. Exposing reset and clearCache allows consumers (especially tests using nock with fake timers) to properly isolate fetch state between test runs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
This PR will trigger a minor release when merged. |
solaris007
pushed a commit
that referenced
this pull request
Mar 10, 2026
## [@adobe/spacecat-shared-utils-v1.101.0](https://github.com/adobe/spacecat-shared/compare/@adobe/spacecat-shared-utils-v1.100.1...@adobe/spacecat-shared-utils-v1.101.0) (2026-03-10) ### Features * **utils:** export resetFetchContext and clearFetchCache ([#1421](#1421)) ([0e05bd8](0e05bd8))
Member
|
🎉 This PR is included in version @adobe/spacecat-shared-utils-v1.101.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
6 tasks
ekremney
added a commit
that referenced
this pull request
Mar 10, 2026
#1423) ## Summary Resolves "fetch failed" errors across Lambda functions by enabling HTTP connection reuse for the two highest-volume outbound callers: PostgrestClient (every DB query) and VaultClient (secret reads, auth, token renewal). ### The problem **PostgrestClient** — `@supabase/postgrest-js` defaults to `@supabase/node-fetch`, which explicitly sets the `Connection: close` header on every request. This **overrides** the agent's `keepAlive` setting and forces the TCP connection to close after each response. Every DB query = new TCP handshake + close. With thousands of queries/sec across all Lambdas, this creates massive ephemeral port pressure on the NAT gateway. > Note: Since Node 19+, `http.globalAgent.keepAlive` defaults to `true`, and `@adobe/fetch` h1 mode also uses keepAlive by default. But none of that matters when the fetch implementation itself sets `Connection: close` — which is exactly what `@supabase/node-fetch` does. **VaultClient** — Used bare `globalThis.fetch` (Node's built-in undici-based fetch). While undici supports connection pooling, using `@adobe/fetch` gives us a shared, configurable context with the same pooling semantics used across the rest of the codebase. Combined with the NACL port range issue ([P0 fix: spacecat-infrastructure#383](adobe/spacecat-infrastructure#383)), this meant Lambdas were: - Opening thousands of short-lived connections per second - Exhausting ephemeral ports on the NAT gateway - Getting `fetch failed` when the OS ran out of source ports or connections timed out in the queue ### How passing `@adobe/fetch` fixes it By passing `@adobe/fetch`'s `fetch` to `new PostgrestClient(..., { fetch })`, we **bypass `@supabase/node-fetch` entirely**. `@adobe/fetch` does NOT set `Connection: close`, so the agent's `keepAlive` actually takes effect and connections are reused. ``` BEFORE (per DB query — @supabase/node-fetch with Connection: close): DNS → TCP handshake → HTTP request → response → connection closed → port released AFTER (pooled — @adobe/fetch with keepAlive): First query: DNS → TCP handshake → HTTP request → response → connection stays open Next queries: HTTP request → response (reuses existing connection) ``` For a Lambda handling 100 requests, this goes from ~100 TCP connections to PostgREST down to ~1-2 persistent connections. Multiply across all Lambda instances and you dramatically reduce NAT gateway connection pressure. ### Key design decisions (per review feedback) **Dedicated fetch contexts** — Instead of using the shared `@adobe/fetch` default context, each package creates its own purpose-built context: - **data-access**: `keepAliveNoCache()` — forces h1 with `keepAlive: true` (for connection reuse to the plain HTTP ALB) and disables HTTP response caching (prevents stale reads after writes) - **vault-secrets**: `noCache()` — uses default protocol negotiation (h2 for HTTPS Vault targets) with caching disabled **Why `keepAliveNoCache` for PostgREST**: The default `h2()` context sets `keepAlive: false` for h1 on Node 19+ (to ensure cross-version consistency). PostgREST is behind a plain HTTP ALB → always h1 → `keepAlive: false` would defeat connection reuse. `keepAliveNoCache()` explicitly enables `keepAlive: true`. **Why `noCache` for everything**: `@adobe/fetch` has a built-in 100MB/500-entry LRU response cache enabled by default. PostgREST GET responses (status 200) are cacheable by default, meaning reads after writes could return stale data within a Lambda invocation. **Headers compatibility wrapper** (`createFetchCompat`): `@adobe/fetch`'s `Headers` class uses `instanceof` to detect Headers objects, which fails for native WHATWG `Headers`. Since `@supabase/postgrest-js` creates native `Headers`, all headers (including `Content-Type: application/json`) were silently dropped. Fix: convert native Headers to plain objects via `Object.fromEntries()`. **Direct `@adobe/fetch` dependency for vault-secrets**: Instead of pulling in `@adobe/spacecat-shared-utils` (which transitively brings cheerio, aws-xray-sdk, zod, etc.), vault-secrets imports `@adobe/fetch` directly — keeping the dependency footprint minimal for a security-critical package. ### Changes - **data-access**: use `keepAliveNoCache()` from `@adobe/fetch` for PostgREST — connection reuse + no response caching - **data-access**: add `createFetchCompat()` wrapper to convert native WHATWG Headers to plain objects - **data-access**: add `@adobe/fetch` as direct dependency - **vault-secrets**: use `noCache()` from `@adobe/fetch` — h2 for HTTPS + no response caching - **vault-secrets**: depend on `@adobe/fetch` directly instead of `@adobe/spacecat-shared-utils` (lighter deps) - **vault-secrets**: add `test/setup-env.js` for `HELIX_FETCH_FORCE_HTTP1` (nock compatibility) - **both**: `HELIX_FETCH_FORCE_HTTP1` fallback to `h1NoCache()` in tests for nock compatibility Depends on [spacecat-shared-utils@1.101.0](#1421) (merged). ## Test plan - [x] data-access: 1533 unit tests passing, 100% coverage - [x] data-access: 709 PostgREST IT tests passing - [x] data-access: 359 legacy IT tests passing - [x] vault-secrets: 75 tests passing, 100% coverage - [ ] CI passes - [ ] Deploy to dev and monitor PostgREST error rates and NAT gateway connection metrics 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
resetFetchContextandclearFetchCachefrom the@adobe/fetchh1/h2 context via@adobe/spacecat-shared-utilsreset) and HTTP response caching (clearCache). Exposing these allows consumers — especially tests using nock withsinon.useFakeTimers— to properly isolate fetch state between test runsTest plan
🤖 Generated with Claude Code