test(integration): add scale correctness suite (#1192) by kriszyp · Pull Request #1229 · HarperFast/harper

kriszyp · 2026-06-10T15:22:28Z

Summary

Adds integrationTests/server/scale.test.ts with five correctness tests at scale (up to 100 K records), exercising large indexed tables on every PR run — the gap identified in Promote scale tests to PR CI: 100K–1M record subset + 1.44M multi-index table correctness #1192.
Adds integrationTests/fixtures/scale-test/ fixture with Product (category indexed) and Attribute (productId + key indexed) tables.
Uses setupHarperWithFixture + sendOperation / SQL COUNT patterns consistent with existing server tests; REST pagination uses Harper's limit(N) query syntax.

Tests

100K insert: paginated read + indexed search — SQL count, two non-overlapping REST pages, exact category-A count (20 K), and search_by_conditions result purity.
10K correctness — total-count growth and pagination non-overlap. (Full 1M tracked in nightly YCSB.)
Multi-condition indexed search — 5 K records; SQL count + search_by_conditions verify category='A' filter purity.
Bulk delete correctness — 1 K insert then bulk-delete; point-check + total-count delta confirm record-level correctness.
EAV at scale — 1 K entity rows + 5 K attribute rows; search_by_conditions on indexed productId returns exactly 5 per entity.

Notes

value is a reserved field name in Harper's condition schema; the Attribute.attrValue rename avoids the clash.
REST pagination: ?limit=N&offset=M is treated as search conditions; correct syntax is ?limit(N,end).
All tests pass locally (5/5); CI gates before marking ready.

Test plan

CI integration-test job runs integrationTests/server/scale.test.ts and passes (5/5).
No regressions in adjacent server tests.

Closes #1192

Generated with Claude Code

Five tests verify record-level correctness at scale (up to 100K records) against a fixture-deployed Product/Attribute schema: 100K paginated read + indexed search, 10K count + pagination non-overlap, multi-condition indexed search, bulk insert→delete correctness, and 1K EAV entity→attribute join via indexed productId. Uses Harper's `limit(N)` query syntax for REST pagination; full 1M / YCSB coverage remains in nightly CI. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

gemini-code-assist

Code Review

This pull request introduces scale correctness integration tests, including a configuration file, a GraphQL schema, and a test suite (scale.test.ts) to verify record-level correctness at scale (up to 100K records). The review feedback correctly identifies several instances where JSON responses from external API calls are parsed and cast directly without verifying if they are arrays. It is recommended to validate these responses using Array.isArray before iterating or mapping over them to prevent potential runtime errors.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

gemini-code-assist · 2026-06-10T15:23:51Z

+			headers: { Authorization: auth },
+		});
+		strictEqual(pageResp.status, 200, `REST page GET failed: ${pageResp.status}`);
+		const page = (await pageResp.json()) as unknown[];


The response from the external API is parsed and mapped directly. Under the repository guidelines, responses from external APIs must be validated with Array.isArray before iterating or mapping to prevent runtime errors if the API returns an unexpected error object.

Suggested change

const page = (await pageResp.json()) as unknown[];

const page = await pageResp.json();

ok(Array.isArray(page), 'Expected page to be an array');

References

When handling responses from peer nodes or external APIs, explicitly verify that expected array fields are actually arrays (e.g., using Array.isArray) before iterating over them, to prevent malformed responses from causing runtime errors.

gemini-code-assist · 2026-06-10T15:23:51Z

+			headers: { Authorization: auth },
+		});
+		strictEqual(page2Resp.status, 200, `REST page2 GET failed: ${page2Resp.status}`);
+		const page2 = (await page2Resp.json()) as Array<{ id: string }>;


The response from the external API is parsed and iterated directly. Under the repository guidelines, responses from external APIs must be validated with Array.isArray before iterating or mapping to prevent runtime errors if the API returns an unexpected error object.

Suggested change

const page2 = (await page2Resp.json()) as Array<{ id: string }>;

const page2 = await page2Resp.json();

ok(Array.isArray(page2), 'Expected page2 to be an array');

References

When handling responses from peer nodes or external APIs, explicitly verify that expected array fields are actually arrays (e.g., using Array.isArray) before iterating over them, to prevent malformed responses from causing runtime errors.

gemini-code-assist · 2026-06-10T15:23:51Z

+		const ids1 = new Set(((await p1Resp.json()) as Array<{ id: string }>).map((r) => r.id));
+		const ids2 = ((await p2Resp.json()) as Array<{ id: string }>).map((r) => r.id);


The responses from p1Resp and p2Resp are parsed and mapped directly. Under the repository guidelines, responses from external APIs must be validated with Array.isArray before iterating or mapping to prevent runtime errors if the API returns an unexpected error object.

Suggested change

const ids1 = new Set(((await p1Resp.json()) as Array<{ id: string }>).map((r) => r.id));

const ids2 = ((await p2Resp.json()) as Array<{ id: string }>).map((r) => r.id);

const p1Data = await p1Resp.json();

ok(Array.isArray(p1Data), 'Expected page 1 response to be an array');

const ids1 = new Set((p1Data as Array<{ id: string }>).map((r) => r.id));

const p2Data = await p2Resp.json();

ok(Array.isArray(p2Data), 'Expected page 2 response to be an array');

const ids2 = (p2Data as Array<{ id: string }>).map((r) => r.id);

References

When handling responses from peer nodes or external APIs, explicitly verify that expected array fields are actually arrays (e.g., using Array.isArray) before iterating over them, to prevent malformed responses from causing runtime errors.

claude · 2026-06-10T15:26:48Z

+// Suite
+// ──────────────────────────────────────────────────────────────────────────────
+
+suite('Scale correctness', (ctx: ContextWithHarper) => {


Blocker: suite callback argument is node:test's TestContext, not ContextWithHarper.

In node:test, the function passed to suite(name, fn) receives a node:test TestContext as its sole argument — not the Harper context. Every other integration test in this repo declares ctx as an outer let/const binding and relies on before() to populate it via setupHarperWithFixture(ctx, …). By naming the suite callback parameter ctx: ContextWithHarper, this code shadows that outer binding, so ctx.harper inside every test() callback resolves to the node:test TestContext object — not the running Harper instance. All five tests will fail at runtime with property-access errors.

The fix is to match the repo's established pattern:

Suggested change

suite('Scale correctness', (ctx: ContextWithHarper) => {

suite('Scale correctness', () => {

const ctx = {} as ContextWithHarper;

Then before/after/test bodies close over the outer ctx, exactly as in every other test file in this repo.

claude · 2026-06-10T15:27:06Z

+		// Verify via total growth instead.
+		const totalAfter = await countRows(ctx, 'Product');
+		// Total should have grown by exactly TOTAL (IDs are unique, no overlap with Test 1).
+		const totalTest1 = 100_000;


Blocker: Test 2 hardcodes Test 1's exact record count.

const totalTest1 = 100_000; makes Test 2 structurally dependent on Test 1 running first and inserting exactly 100 K records in the same table. If Test 1 is skipped, re-ordered, or its TOTAL changes, this assertion silently produces a wrong expected value — or fails with a misleading message that hides the real issue.

Use the actual pre-insert count to compute the delta instead:

Suggested change

const totalTest1 = 100_000;

// Record the count before this test's inserts so the delta is self-contained.

const countBefore = await countRows(ctx, 'Product');

for (let start = 0; start < TOTAL; start += BATCH) {

Then assert strictEqual(totalAfter, countBefore + TOTAL, …) using the locally-measured baseline. This removes the hard coupling to Test 1's implementation.

claude · 2026-06-10T15:27:19Z

+			headers: { Authorization: auth },
+		});
+		strictEqual(pageResp.status, 200, `REST page GET failed: ${pageResp.status}`);
+		const page = (await pageResp.json()) as unknown[];


Unguarded array assumption on REST response.

pageResp.json() is cast directly to unknown[] and then .length is accessed without verifying the response is actually an array. If the server returns an error envelope { error: "..." } the cast silently succeeds and page.length will be undefined, turning the subsequent strictEqual into a confusing "Expected 100, got undefined" rather than a clear failure.

Add a guard before using the value:

Suggested change

const page = (await pageResp.json()) as unknown[];

const page = (await pageResp.json()) as unknown[];

ok(Array.isArray(page), `Expected array response from /Product, got: ${JSON.stringify(page)}`);

Same pattern applies to page2 (line 113), p1Resp/p2Resp (lines 181–182).

claude · 2026-06-10T15:27:40Z

+			`category='A' AND inStock=true AND id >= 'prod-${ID_OFFSET}' AND id < 'prod-${ID_OFFSET + TOTAL}'`
+		);
+		// Allow a small tolerance for lexicographic edge effects on string-range predicates.
+		// The exact count via search_by_conditions is the gold standard; SQL gives us a lower bound.


Near-no-op assertion undermines Test 3's value.

The comment on line 212 concedes that the SQL range predicate has "lexicographic edge effects" and then falls back to ok(sqlCount > 0, …). This is effectively a liveness check, not a correctness check — sqlCount = 1 would pass. The main purpose of Test 3 is to verify multi-condition indexed-search correctness, but the SQL side of the verification is so weakened that it adds almost no signal.

Options:

Drop the SQL COUNT entirely and rely solely on the search_by_conditions purity loop below (which is the real assertion).

Use a numeric ID scheme for Test 3's ID_OFFSET so lexicographic order matches numeric order, enabling a precise SQL count assertion.

claude · 2026-06-10T15:27:52Z

Reviewed; no blockers found.

(Prior finding #1 — suite callback param — was a false positive: crl-verification.test.ts and custom-resources.test.ts use the same (ctx: ContextWithHarper) pattern; setupHarperWithFixture mutates the TestContext in-place. Findings #2 and #3 are fixed in 55e9293.)

- Snapshot Product count before Test 2's inserts instead of hardcoding Test 1's 100K, removing the hidden inter-test dependency. - Guard all REST pagination responses with Array.isArray before reading .length / mapping, per repo external-response handling guidelines. - Drop Test 3's weak SQL string-range count (lexicographic ordering made it a near-no-op liveness check); rely on the search_by_conditions purity loop as the real multi-condition correctness assertion. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Ethan-Arrowood

LGTM — solid correctness coverage for the scale gap from #1192. The one red check is just a stale build-artifact/infra failure (not the tests), so a re-run should green it up. Good to go.

_{sent with Claude Opus 4.8}

kriszyp requested review from Ethan-Arrowood and heskew June 10, 2026 15:23

gemini-code-assist Bot reviewed Jun 10, 2026

View reviewed changes

claude Bot reviewed Jun 10, 2026

View reviewed changes

kriszyp added this to the v5.1 milestone Jun 14, 2026

kriszyp marked this pull request as ready for review June 14, 2026 22:34

Ethan-Arrowood approved these changes Jun 22, 2026

View reviewed changes

kriszyp merged commit 2a64b96 into main Jun 25, 2026
252 of 258 checks passed

kriszyp deleted the kris/integration-test-scale branch June 25, 2026 17:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

test(integration): add scale correctness suite (#1192) - #1229

test(integration): add scale correctness suite (#1192)#1229
kriszyp merged 2 commits into
mainfrom
kris/integration-test-scale

kriszyp commented Jun 10, 2026 •

edited

Loading

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot Jun 10, 2026

Uh oh!

gemini-code-assist Bot Jun 10, 2026

Uh oh!

gemini-code-assist Bot Jun 10, 2026

Uh oh!

claude Bot Jun 10, 2026

Uh oh!

claude Bot Jun 10, 2026

Uh oh!

claude Bot Jun 10, 2026

Uh oh!

claude Bot Jun 10, 2026

Uh oh!

claude Bot commented Jun 10, 2026 •

edited

Loading

Uh oh!

Ethan-Arrowood left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	const page = (await pageResp.json()) as unknown[];
	const page = await pageResp.json();
	ok(Array.isArray(page), 'Expected page to be an array');

	const page2 = (await page2Resp.json()) as Array<{ id: string }>;
	const page2 = await page2Resp.json();
	ok(Array.isArray(page2), 'Expected page2 to be an array');

		const ids1 = new Set(((await p1Resp.json()) as Array<{ id: string }>).map((r) => r.id));
		const ids2 = ((await p2Resp.json()) as Array<{ id: string }>).map((r) => r.id);

-		const ids1 = new Set(((await p1Resp.json()) as Array<{ id: string }>).map((r) => r.id));
-		const ids2 = ((await p2Resp.json()) as Array<{ id: string }>).map((r) => r.id);
+		const p1Data = await p1Resp.json();
+		ok(Array.isArray(p1Data), 'Expected page 1 response to be an array');
+		const ids1 = new Set((p1Data as Array<{ id: string }>).map((r) => r.id));
+		const p2Data = await p2Resp.json();
+		ok(Array.isArray(p2Data), 'Expected page 2 response to be an array');
+		const ids2 = (p2Data as Array<{ id: string }>).map((r) => r.id);

	suite('Scale correctness', (ctx: ContextWithHarper) => {
	suite('Scale correctness', () => {
	const ctx = {} as ContextWithHarper;

-		const totalTest1 = 100_000;
+		// Record the count before this test's inserts so the delta is self-contained.
+		const countBefore = await countRows(ctx, 'Product');
+		for (let start = 0; start < TOTAL; start += BATCH) {

	const page = (await pageResp.json()) as unknown[];
	const page = (await pageResp.json()) as unknown[];
	ok(Array.isArray(page), `Expected array response from /Product, got: ${JSON.stringify(page)}`);

Uh oh!

Conversation

kriszyp commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Tests

Notes

Test plan

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Jun 10, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jun 10, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jun 10, 2026

Choose a reason for hiding this comment

Uh oh!

claude Bot Jun 10, 2026

Choose a reason for hiding this comment

Uh oh!

claude Bot Jun 10, 2026

Choose a reason for hiding this comment

Uh oh!

claude Bot Jun 10, 2026

Choose a reason for hiding this comment

Uh oh!

claude Bot Jun 10, 2026

Choose a reason for hiding this comment

Uh oh!

claude Bot commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Ethan-Arrowood left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

kriszyp commented Jun 10, 2026 •

edited

Loading

claude Bot commented Jun 10, 2026 •

edited

Loading