Skip to content

Commit aa8a243

Browse files
SyMindSyMind
authored
feat: use Rspack persistent cache by default (#81399)
Rspack now enables persistent caching by default. ## Performance Comparison ### Pages Router I benchmarked performance using this repo: https://github.com/SyMind/chakra-ui-docs/tree/next-rspack to test the Next.js pages router. I tested the performance with the following steps: 1. Execute `pnpm run dev` 2. Wait for the server to be ready (indicated by the 'Ready' message) 3. Run curl on the root endpoint (/) Each build was run 5 times, and the shortest time to reach "Compiled successfully" was recorded. Test environment: Apple M1 Pro CPU | Tool | Build without cache | Build with cache | Dev without cache | Dev with cache | |-------------|---------------------|------------------|---------------------------------|----------------| | Rspack | 3.8s | 2.6s | 1.7s | 3ms | | Webpack | 14.0s | 4.0s | 7.8s | 3.2s | ### App Router I benchmarked performance using this repo: https://github.com/SyMind/shadcn-ui/tree/next-rspack to test the Next.js app router. I tested the performance with the following steps: 1. Execute `pnpm run dev` or `pnpm run build` 2. Wait for the server to be ready (indicated by the 'Ready' message) 3. Run curl on the root endpoint (/) Each build was run 5 times, and the shortest time to reach "Compiled successfully" was recorded. Test environment: Apple M1 Pro CPU | Bundler | Build (No Cache) | Build (Cache) | Dev (No Cache) | Dev (Cache) | |------------|----------------------|-------------------|--------------------|-----------------| | Rspack | 12.3s | 5.9s | 7.1s | 1941ms | | Webpack | 27.0s | 13.0s | 11s | 9.6s | ## About Rspack Persistent Cache Strategy > packages/next/src/server/dev/hot-reloader-rspack.ts Rspack's persistent caching differs from Webpack in how it manages module graphs. While Webpack incrementally updates modules, Rspack operates on complete module graph snapshots for cache restoration. Problem: - Next.js dev server starts with no page modules in the initial entry points - When Rspack restores from persistent cache, it finds no modules and purges the entire module graph - Later page requests find no cached module information, preventing cache reuse Solution: - Track successfully built page entries after each compilation - Restore these entries on dev server restart to maintain module graph continuity - This ensures previously compiled pages can leverage persistent cache for faster builds ## Note I have updated the test case configuration in `test/integration/telemetry/next.config.use-cache` to disable persistent cache. This is because, whether using webpack or Rspack, when persistent caching is enabled, modules are no longer recompiled by loaders, which prevents the Telemetry plugin from collecting information. Please note that this issue also exists with webpack. You can reproduce it locally by running `pnpm run test test/integration/telemetry/test/config.test.js` twice.
1 parent 754db28 commit aa8a243

File tree

11 files changed

+452
-93
lines changed

11 files changed

+452
-93
lines changed

‎packages/next/src/build/webpack-config.ts‎

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2361,7 +2361,7 @@ export default async function getBaseWebpackConfig(
23612361
serverReferenceHashSalt: encryptionKey,
23622362
})
23632363

2364-
const cache: any = {
2364+
const cache: webpack.Configuration['cache'] = {
23652365
type: 'filesystem',
23662366
// Disable memory cache in development in favor of our own MemoryWithGcCachePlugin.
23672367
maxMemoryGenerations: dev ? 0 : Infinity, // Infinity is default value for production in webpack currently.
@@ -2407,6 +2407,30 @@ export default async function getBaseWebpackConfig(
24072407

24082408
webpack5Config.cache = cache
24092409

2410+
if (isRspack) {
2411+
const buildDependencies: string[] = []
2412+
if (config.configFile) {
2413+
buildDependencies.push(config.configFile)
2414+
}
2415+
if (babelConfigFile) {
2416+
buildDependencies.push(babelConfigFile)
2417+
}
2418+
if (jsConfigPath) {
2419+
buildDependencies.push(jsConfigPath)
2420+
}
2421+
2422+
// @ts-ignore
2423+
webpack5Config.experiments.cache = {
2424+
type: 'persistent',
2425+
buildDependencies,
2426+
storage: {
2427+
type: 'filesystem',
2428+
directory: cache.cacheDirectory,
2429+
},
2430+
version: `${__dirname}|${process.env.__NEXT_VERSION}|${configVars}`,
2431+
}
2432+
}
2433+
24102434
if (process.env.NEXT_WEBPACK_LOGGING) {
24112435
const infra = process.env.NEXT_WEBPACK_LOGGING.includes('infrastructure')
24122436
const profileClient =

‎packages/next/src/server/dev/hot-reloader-rspack.ts‎

Lines changed: 243 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,243 @@
1+
import path from 'path'
2+
import fs from 'fs/promises'
3+
import { createHash } from 'crypto'
4+
import HotReloaderWebpack from './hot-reloader-webpack'
5+
import { BUILT, EntryTypes, getEntries } from './on-demand-entry-handler'
6+
import type { __ApiPreviewProps } from '../api-utils'
7+
import type { RouteDefinition } from '../route-definitions/route-definition'
8+
import type { MultiCompiler } from 'webpack'
9+
import { COMPILER_NAMES } from '../../shared/lib/constants'
10+
11+
/**
12+
* Rspack Persistent Cache Strategy for Next.js Development
13+
*
14+
* Rspack's persistent caching differs from Webpack in how it manages module graphs.
15+
* While Webpack incrementally updates modules, Rspack operates on complete module
16+
* graph snapshots for cache restoration.
17+
*
18+
* Problem:
19+
* - Next.js dev server starts with no page modules in the initial entry points
20+
* - When Rspack restores from persistent cache, it finds no modules and purges
21+
* the entire module graph
22+
* - Later page requests find no cached module information, preventing cache reuse
23+
*
24+
* Solution:
25+
* - Track successfully built page entries after each compilation
26+
* - Restore these entries on dev server restart to maintain module graph continuity
27+
* - This ensures previously compiled pages can leverage persistent cache for faster builds
28+
*/
29+
export default class HotReloaderRspack extends HotReloaderWebpack {
30+
private builtEntriesCachePath?: string
31+
32+
private isClientCacheEnabled = false
33+
private isServerCacheEnabled = false
34+
private isEdgeServerCacheEnabled = false
35+
36+
public async afterCompile(multiCompiler: MultiCompiler): Promise<void> {
37+
// Always initialize the fallback error watcher for Rspack.
38+
// Rspack may restore/retain the previous build's error state, so without this
39+
// a page that previously failed to build might not be rebuilt on the next request.
40+
await super.buildFallbackError()
41+
42+
const rspackStartSpan = this.hotReloaderSpan.traceChild(
43+
'rspack-after-compile'
44+
)
45+
await rspackStartSpan.traceAsyncFn(async () => {
46+
const hash = createHash('sha1')
47+
multiCompiler.compilers.forEach((compiler) => {
48+
const cache = compiler.options.cache
49+
if (typeof cache === 'object' && 'version' in cache && cache.version) {
50+
hash.update(cache.version)
51+
if (compiler.name === COMPILER_NAMES.client) {
52+
this.isClientCacheEnabled = true
53+
} else if (compiler.name === COMPILER_NAMES.server) {
54+
this.isServerCacheEnabled = true
55+
} else if (compiler.name === COMPILER_NAMES.edgeServer) {
56+
this.isEdgeServerCacheEnabled = true
57+
}
58+
} else {
59+
hash.update('-')
60+
}
61+
return undefined
62+
})
63+
this.builtEntriesCachePath = path.join(
64+
this.distDir,
65+
'cache',
66+
'rspack',
67+
hash.digest('hex').substring(0, 16),
68+
'built-entries.json'
69+
)
70+
71+
const hasBuiltEntriesCache = await fs
72+
.access(this.builtEntriesCachePath)
73+
.then(
74+
() => true,
75+
() => false
76+
)
77+
if (hasBuiltEntriesCache) {
78+
try {
79+
const builtEntries: ReturnType<typeof getEntries> = JSON.parse(
80+
(await fs.readFile(this.builtEntriesCachePath, 'utf-8')) || '{}'
81+
)
82+
83+
await Promise.all(
84+
Object.keys(builtEntries).map(async (entryKey) => {
85+
const entryData = builtEntries[entryKey]
86+
87+
const isEntry = entryData.type === EntryTypes.ENTRY
88+
const isChildEntry = entryData.type === EntryTypes.CHILD_ENTRY
89+
90+
// Check if the page was removed or disposed and remove it
91+
if (isEntry) {
92+
const pageExists =
93+
!entryData.dispose &&
94+
(await fs.access(entryData.absolutePagePath).then(
95+
() => true,
96+
() => false
97+
))
98+
if (!pageExists) {
99+
delete builtEntries[entryKey]
100+
return
101+
} else if (
102+
!('hash' in builtEntries[entryKey]) ||
103+
builtEntries[entryKey].hash !==
104+
(await calculateFileHash(entryData.absolutePagePath))
105+
) {
106+
delete builtEntries[entryKey]
107+
return
108+
}
109+
}
110+
111+
// For child entries, if it has an entry file and it's gone, remove it
112+
if (isChildEntry) {
113+
if (entryData.absoluteEntryFilePath) {
114+
const pageExists =
115+
!entryData.dispose &&
116+
(await fs.access(entryData.absoluteEntryFilePath).then(
117+
() => true,
118+
() => false
119+
))
120+
if (!pageExists) {
121+
delete builtEntries[entryKey]
122+
return
123+
} else {
124+
if (
125+
!('hash' in builtEntries[entryKey]) ||
126+
builtEntries[entryKey].hash !==
127+
(await calculateFileHash(
128+
entryData.absoluteEntryFilePath
129+
))
130+
) {
131+
delete builtEntries[entryKey]
132+
return
133+
}
134+
}
135+
}
136+
}
137+
})
138+
)
139+
Object.assign(getEntries(multiCompiler.outputPath), builtEntries)
140+
} catch (error) {
141+
console.error('Rspack failed to read built entries cache: ', error)
142+
}
143+
}
144+
})
145+
}
146+
147+
public async ensurePage({
148+
page,
149+
clientOnly,
150+
appPaths,
151+
definition,
152+
isApp,
153+
url,
154+
}: {
155+
page: string
156+
clientOnly: boolean
157+
appPaths?: ReadonlyArray<string> | null
158+
isApp?: boolean
159+
definition?: RouteDefinition
160+
url?: string
161+
}): Promise<void> {
162+
await super.ensurePage({
163+
page,
164+
clientOnly,
165+
appPaths,
166+
definition,
167+
isApp,
168+
url,
169+
})
170+
const entries = getEntries(this.multiCompiler!.outputPath)
171+
const builtEntries: { [entryName: string]: any } = {}
172+
await Promise.all(
173+
Object.keys(entries).map(async (entryName) => {
174+
const entry = entries[entryName]
175+
if (entry.status !== BUILT) return
176+
const result =
177+
/^(client|server|edge-server)@(app|pages|root)@(.*)/g.exec(entryName)
178+
const [, key /* pageType */, ,] = result! // this match should always happen
179+
if (key === 'client' && !this.isClientCacheEnabled) return
180+
if (key === 'server' && !this.isServerCacheEnabled) return
181+
if (key === 'edge-server' && !this.isEdgeServerCacheEnabled) return
182+
183+
// TODO: Rspack does not store middleware entries in persistent cache, causing
184+
// test/integration/middleware-src/test/index.test.ts to fail. This is a temporary
185+
// workaround to skip middleware entry caching until Rspack properly supports it.
186+
if (page === '/middleware') {
187+
return
188+
}
189+
190+
let hash: string | undefined
191+
if (entry.type === EntryTypes.ENTRY) {
192+
hash = await calculateFileHash(entry.absolutePagePath)
193+
} else if (entry.absoluteEntryFilePath) {
194+
hash = await calculateFileHash(entry.absoluteEntryFilePath)
195+
}
196+
if (!hash) {
197+
return
198+
}
199+
200+
builtEntries[entryName] = entry
201+
builtEntries[entryName].hash = hash
202+
})
203+
)
204+
205+
const hasBuitEntriesCache = await fs
206+
.access(this.builtEntriesCachePath!)
207+
.then(
208+
() => true,
209+
() => false
210+
)
211+
try {
212+
if (!hasBuitEntriesCache) {
213+
await fs.mkdir(path.dirname(this.builtEntriesCachePath!), {
214+
recursive: true,
215+
})
216+
}
217+
await fs.writeFile(
218+
this.builtEntriesCachePath!,
219+
JSON.stringify(builtEntries, null, 2)
220+
)
221+
} catch (error) {
222+
console.error('Rspack failed to write built entries cache: ', error)
223+
}
224+
}
225+
}
226+
227+
async function calculateFileHash(
228+
filePath: string,
229+
algorithm: string = 'sha256'
230+
): Promise<string | undefined> {
231+
if (
232+
!(await fs.access(filePath).then(
233+
() => true,
234+
() => false
235+
))
236+
) {
237+
return
238+
}
239+
const fileBuffer = await fs.readFile(filePath)
240+
const hash = createHash(algorithm)
241+
hash.update(fileBuffer)
242+
return hash.digest('hex')
243+
}

‎packages/next/src/server/dev/hot-reloader-webpack.ts‎

Lines changed: 16 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -226,18 +226,18 @@ function erroredPages(compilation: webpack.Compilation) {
226226
export default class HotReloaderWebpack implements NextJsHotReloaderInterface {
227227
private hasAppRouterEntrypoints: boolean
228228
private hasPagesRouterEntrypoints: boolean
229-
private dir: string
230-
private buildId: string
229+
protected dir: string
230+
protected buildId: string
231231
private encryptionKey: string
232-
private middlewares: ((
232+
protected middlewares: ((
233233
req: IncomingMessage,
234234
res: ServerResponse,
235235
next: () => void
236236
) => Promise<void>)[]
237-
private pagesDir?: string
238-
private distDir: string
237+
protected pagesDir?: string
238+
protected distDir: string
239239
private webpackHotMiddleware?: WebpackHotMiddleware
240-
private config: NextConfigComplete
240+
protected config: NextConfigComplete
241241
private clientStats: webpack.Stats | null
242242
private clientError: Error | null = null
243243
private serverError: Error | null = null
@@ -246,13 +246,13 @@ export default class HotReloaderWebpack implements NextJsHotReloaderInterface {
246246
private serverChunkNames?: Set<string>
247247
private prevChunkNames?: Set<any>
248248
private onDemandEntries?: ReturnType<typeof onDemandEntryHandler>
249-
private previewProps: __ApiPreviewProps
249+
protected previewProps: __ApiPreviewProps
250250
private watcher: any
251251
private rewrites: CustomRoutes['rewrites']
252252
private fallbackWatcher: any
253-
private hotReloaderSpan: Span
253+
protected hotReloaderSpan: Span
254254
private pagesMapping: { [key: string]: string } = {}
255-
private appDir?: string
255+
protected appDir?: string
256256
private telemetry: Telemetry
257257
private resetFetch: () => void
258258
private lockfile: Lockfile | undefined
@@ -1243,6 +1243,8 @@ export default class HotReloaderWebpack implements NextJsHotReloaderInterface {
12431243
this.activeWebpackConfigs
12441244
) as unknown as webpack.MultiCompiler
12451245

1246+
await this.afterCompile(this.multiCompiler)
1247+
12461248
// Copy over the filesystem so that it is shared between all compilers.
12471249
const inputFileSystem = this.multiCompiler.compilers[0].inputFileSystem
12481250
for (const compiler of this.multiCompiler.compilers) {
@@ -1643,7 +1645,7 @@ export default class HotReloaderWebpack implements NextJsHotReloaderInterface {
16431645
}),
16441646
})
16451647

1646-
this.middlewares = [
1648+
this.middlewares.push(
16471649
getOverlayMiddleware({
16481650
rootDirectory: this.dir,
16491651
isSrcDir: this.isSrcDir,
@@ -1694,9 +1696,8 @@ export default class HotReloaderWebpack implements NextJsHotReloaderInterface {
16941696
getDevServerUrl: () => process.env.__NEXT_PRIVATE_ORIGIN,
16951697
}),
16961698
]
1697-
: []),
1698-
]
1699-
1699+
: [])
1700+
)
17001701
setStackFrameResolver(async (request) => {
17011702
return getOriginalStackFrames({
17021703
isServer: request.isServer,
@@ -1711,6 +1712,8 @@ export default class HotReloaderWebpack implements NextJsHotReloaderInterface {
17111712
})
17121713
}
17131714

1715+
protected async afterCompile(_multiCompiler: webpack.MultiCompiler) {}
1716+
17141717
public invalidate(
17151718
{ reloadAfterInvalidation }: { reloadAfterInvalidation: boolean } = {
17161719
reloadAfterInvalidation: false,

‎packages/next/src/server/dev/on-demand-entry-handler.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,6 @@ import type {
88
} from '../../shared/lib/app-router-types'
99
import type { CompilerNameValues } from '../../shared/lib/constants'
1010
import type { RouteDefinition } from '../route-definitions/route-definition'
11-
import type HotReloaderWebpack from './hot-reloader-webpack'
1211

1312
import createDebug from 'next/dist/compiled/debug'
1413
import { EventEmitter } from 'events'
@@ -39,6 +38,7 @@ import { PAGE_SEGMENT_KEY } from '../../shared/lib/segment'
3938
import {
4039
HMR_MESSAGE_SENT_TO_BROWSER,
4140
HMR_MESSAGE_SENT_TO_SERVER,
41+
type NextJsHotReloaderInterface,
4242
} from './hot-reloader-types'
4343
import { isAppPageRouteDefinition } from '../route-definitions/app-page-route-definition'
4444
import { scheduleOnNextTick } from '../../lib/scheduler'
@@ -548,7 +548,7 @@ export function onDemandEntryHandler({
548548
rootDir,
549549
appDir,
550550
}: {
551-
hotReloader: HotReloaderWebpack
551+
hotReloader: NextJsHotReloaderInterface
552552
maxInactiveAge: number
553553
multiCompiler: webpack.MultiCompiler
554554
nextConfig: NextConfigComplete

0 commit comments

Comments
 (0)