[http-client-csharp] Add Tier 1 in-memory response cache to playground-server - #10718
Merged
Conversation
…d-server Agent-Logs-Url: https://github.com/microsoft/typespec/sessions/693fd6ec-31da-4c6f-a4be-4324b946a6f4 Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Enhance performance of C# emitter in TypeSpec playground
[http-client-csharp] Add Tier 1 in-memory response cache to playground-server
May 18, 2026
Agent-Logs-Url: https://github.com/microsoft/typespec/sessions/656e84a4-17f1-4c10-87de-4f80c2df8412 Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Contributor
|
@copilot resolve merge conflicts |
…p-emitter-performance # Conflicts: # packages/http-client-csharp/playground-server/Program.cs Co-authored-by: JoshLove-msft <54595583+JoshLove-msft@users.noreply.github.com>
JoshLove-msft
marked this pull request as ready for review
June 2, 2026 00:26
JoshLove-msft
requested review from
ShivangiReja,
jsquire and
m-nash
as code owners
June 2, 2026 00:26
Contributor
Kicked off a pipeline run |
Adds four GenerationCacheTests cases that drive ComputeKey with a multi-KB code model/configuration shaped like real generator input: determinism, key change on a semantic code-model edit, key change on a configuration edit, and a cache round-trip. Addresses PR review feedback requesting coverage with a realistic code model. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Follows the generator test projects' TestData convention: the realistic code model and configuration now live in TestData/GenerationCacheTests/*.json (copied to the output directory) and are loaded via a ReadTestData helper, rather than as inlined string constants in the test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…arp-emitter-performance
jsquire
approved these changes
Jun 2, 2026
…ackage versions - Split GenerationCache.cs into CachedGenerationResponse.cs, IGenerationCache.cs and MemoryGenerationCache.cs (one public type per file). - Make the memory cache SizeLimit configurable via app settings (GenerationCache:SizeLimitBytes) with the 256 MB default, surfaced in a new appsettings.json. - Drop hardcoded package versions from playground-server.Tests.csproj and consume the central versions from generator/Packages.Data.props instead (adding a Microsoft.Extensions.Caching.Memory entry there). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
JoshLove-msft
enabled auto-merge
June 2, 2026 18:55
which@7.0.0 raised its Node engine floor to ^22.22.2 || ^24.15.0, which broke CI: the emitter pipeline's primary Node (24.x) resolves to 24.14.1 on the hosted agents, failing the engine-strict pnpm install. which@6.0.1 (engines: ^20.17.0 || >=22.9.0) is API-compatible and satisfied by every Node version in the build matrix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
JoshLove-msft
requested review from
bterlson,
catalinaperalta,
iscai-msft,
markcowl,
timotheeguerin and
witemple-msft
as code owners
June 2, 2026 19:29
This reverts commit 79fc189.
The pinned major-version specs resolved to a stale cached Node on the agents (24.14.1), which failed engine-strict checks for dependencies requiring >=24.15.0. checkLatest fetches the newest matching release. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
timotheeguerin
approved these changes
Jun 2, 2026
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…arp-emitter-performance
jorgerangel-msft
approved these changes
Jun 3, 2026
JoshLove-msft
approved these changes
Jun 3, 2026
jsquire
approved these changes
Jun 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The playground's C#
/generateendpoint spawns a freshdotnetsubprocess on every request, with no caching anywhere in the pipeline. Identical replays (undo/redo, share-link reloads, demo specs hit by many users) pay the full cost. This implements Item 3, Tier 1 of the perf plan: a container-local in-memory response cache.Changes
playground-server/GenerationCache.cs(new) —IGenerationCache+MemoryGenerationCachebacked byIMemoryCache.ComputeKey(generatorName, codeModel, configuration, generatorVersion)returns a SHA-256 hex over a length-prefixed concatenation (avoids cross-boundary collisions like"Foo"+"Bar"vs"FooBar"+""). EntrySize= body byte length; backing cacheSizeLimit = 256 MB. Entries are bounded bySizeLimit-driven compaction only — there is no TTL, since a new generator binary deploy implicitly invalidates the cache via the version-keyed entries.playground-server/Program.cs— registersAddMemoryCache+IGenerationCachesingleton. Reads the generator DLL'sFileVersionInfo.FileVersiononce at startup and folds it into the key, so a deploy of a new binary implicitly invalidates every entry./generateshort-circuits on hit (returns cached bytes, skips the subprocess) and serializes-once-then-caches on miss. Responses carryX-Cache: HIT|MISS. Cache hits are recorded in telemetry: a hit emits thePlaygroundGenerateevent with outcomecache_hit, and every request carries acacheStatus(hit|miss) property through the existing telemetry pipeline.playground-server.Tests/(new NUnit project, 14 tests) — key determinism & format; sensitivity to each of the four components (including version → invalidation); length-prefix unambiguity; null-argument guards; get/miss/set/overwrite round-trip;SizeLimitcompaction.Cache flow
Out of scope
Tier 2 file-based cache and Items 1/2/4/5 of the issue. Generator internals are untouched; production
tspcodegen is unaffected.