Skip to content

BE-587, BE-588: HashQL: Introduce CodeGenerationContext/CodeExecutionContext split and eval-stage Interner#8839

Open
indietyp wants to merge 7 commits into
bm/be-581-hashql-cached-constructor-produces-bare-fnptr-instead-offrom
bm/be-587-hashql-eval-interner-and-context-restructuring
Open

BE-587, BE-588: HashQL: Introduce CodeGenerationContext/CodeExecutionContext split and eval-stage Interner#8839
indietyp wants to merge 7 commits into
bm/be-581-hashql-cached-constructor-produces-bare-fnptr-instead-offrom
bm/be-587-hashql-eval-interner-and-context-restructuring

Conversation

@indietyp

@indietyp indietyp commented Jun 8, 2026

Copy link
Copy Markdown
Member

🌟 What is the purpose of this PR?

This PR refactors the HashQL evaluation pipeline to introduce a cleaner separation between the code generation phase and the code execution phase. The previously unified EvalContext is split into CodeGenerationContext (used during MIR compilation and Postgres query preparation) and CodeExecutionContext (used during orchestrated query execution). This makes the data flow and lifecycle of each phase explicit and prevents accidental misuse of context across phase boundaries.

Additionally, a dedicated eval::intern::Interner is introduced for the evaluation stage, which must be constructed from the MIR interner via From to preserve interned pointer identity across the MIR/eval boundary. Input construction for orchestrator tests is also migrated away from using the JSON decoder to directly building typed Value trees using the heap allocator, removing a dependency on the type environment at input-construction time.

The compile_query implementation in the graph API is stubbed out with unimplemented! pending the HashQL API wire-up tracked in BE-537.

🔗 Related links

🔍 What does this change?

  • Renames EvalContext to CodeGenerationContext and introduces a new CodeExecutionContext struct that is produced from CodeGenerationContext via From, carrying only the fields needed at execution time.
  • Adds eval::intern::Interner as a dedicated interner for the evaluation stage, constructed from hashql_mir::intern::Interner to preserve pointer identity.
  • Updates Orchestrator and PostgresCompiler to use the appropriate context type for their respective phases.
  • Replaces JSON-decoder-based input construction in orchestrator tests with direct Value tree construction using heap-allocated Rc and StructBuilder, removing the dependency on the post-lowering type environment for building inputs.
  • Stubs out EntityQuery::compile_query with unimplemented! and suppresses dead_code warnings pending the full HashQL API integration.
  • Updates the EvalDiagnosticCategory enum to replace the graph-feature-gated Graph variant with an Orchestrator variant backed by OrchestratorDiagnosticCategory.
  • Migrates orchestrator test helpers to pass Lowered by value (consuming it) and convert the MIR interner into the eval interner at the appropriate pipeline stage.

🛡 What tests cover this?

  • Existing orchestrator integration tests and eval_postgres compile tests exercise the refactored context types and input construction.
  • Decoder unit tests updated to use Interner::testing for the new eval-stage interner.

❓ How to test this?

  1. Run cargo test across the hashql-eval and hash-graph-api crates.
  2. Run the orchestrator integration tests to confirm query execution still produces correct results against a live Postgres instance.

indietyp added 3 commits June 8, 2026 12:32
chore: add new dependency

chore: format

feat: error module

feat: introduce hashql_eval interner

chore: checkpoint

feat: checkpoint

feat: checkpoint

chore: remove old value module

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

chore: checkpoint

feat: move entity query into its own modul

fix: query request

feat: checkpoint (it compiles!)

feat: checkpoint

feat: checkpoint

feat: checkpoint

fix: issue around cached thunking

feat: covariance for opaque inners

fix: cfgattr serde

chore: remove graph module

fix: merge fuckup
Copilot AI review requested due to automatic review settings June 8, 2026 11:41
@cursor

cursor Bot commented Jun 8, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Entity/subgraph requests with a raw query field now fail with 501 instead of compiling HashQL; eval/orchestrator and Postgres compile paths are refactored but covered by existing tests.

Overview
Splits HashQL eval into code generation vs execution contexts: EvalContext becomes CodeGenerationContext (Postgres compilation, live-out, diagnostics) and a slimmer CodeExecutionContext used by Orchestrator after From conversion.

Adds hashql_eval::intern::Interner, created from the MIR interner so symbol interning stays consistent across the MIR→eval boundary; orchestrator codec/partial paths and tests use it instead of hashql_mir::intern::Interner.

Graph API: removes the full in-request HashQL compile pipeline for EntityQuery::Query; those requests now return HTTP 501 with a BE-537 link until HashQL is wired in the API. Filter-based queries are unchanged.

Orchestrator tests build interpreter Inputs as heap-backed opaque/struct Value trees (no JSON Decoder / type env at input construction). Execution helpers consume Lowered by value, convert MIR→eval interner before compile, then run via run_in with the heap allocator.

EvalDiagnosticCategory drops the graph compiler variant in favor of Orchestrator for eval diagnostics.

Reviewed by Cursor Bugbot for commit 3d05f60. Bugbot is set up for automated code reviews on this repo. Configure here.

@vercel

vercel Bot commented Jun 8, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hash Ready Ready Preview, Comment Jun 9, 2026 7:40am
2 Skipped Deployments
Project Deployment Actions Updated (UTC)
hashdotdesign-tokens Ignored Ignored Preview Jun 9, 2026 7:40am
petrinaut Skipped Skipped Jun 9, 2026 7:40am

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 8af8844. Configure here.

Comment thread libs/@local/hashql/eval/tests/orchestrator/inputs.rs
@codecov

codecov Bot commented Jun 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 60.52632% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.55%. Comparing base (b939729) to head (3d05f60).

Files with missing lines Patch % Lines
libs/@local/hashql/eval/src/context.rs 52.94% 8 Missing ⚠️
.../@local/graph/api/src/rest/entity_query_request.rs 0.00% 4 Missing ⚠️
libs/@local/hashql/core/src/graph/linked.rs 0.00% 2 Missing ⚠️
libs/@local/hashql/eval/src/error.rs 0.00% 1 Missing ⚠️
Additional details and impacted files
@@                                          Coverage Diff                                           @@
##           bm/be-581-hashql-cached-constructor-produces-bare-fnptr-instead-of    #8839      +/-   ##
======================================================================================================
+ Coverage                                                               59.53%   59.55%   +0.02%     
======================================================================================================
  Files                                                                    1347     1348       +1     
  Lines                                                                  130960   130870      -90     
  Branches                                                                 5884     5884              
======================================================================================================
- Hits                                                                    77961    77942      -19     
+ Misses                                                                  52092    52021      -71     
  Partials                                                                  907      907              
Flag Coverage Δ
apps.hash-ai-worker-ts 1.39% <ø> (ø)
apps.hash-api 0.00% <ø> (ø)
local.hash-backend-utils 2.81% <ø> (ø)
local.hash-graph-sdk 9.63% <ø> (ø)
local.hash-isomorphic-utils 0.00% <ø> (ø)
rust.hash-graph-api 2.56% <0.00%> (+0.04%) ⬆️
rust.hashql-compiletest 28.41% <ø> (ø)
rust.hashql-eval 74.94% <71.87%> (-0.27%) ⬇️
rust.hashql-hir 89.06% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@indietyp indietyp changed the title BE-587: HashQL: Introduce CodeGenerationContext/CodeExecutionContext split and eval-stage Interner BE-587, BE-588: HashQL: Introduce CodeGenerationContext/CodeExecutionContext split and eval-stage Interner Jun 8, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors the HashQL evaluation pipeline to explicitly separate code-generation vs code-execution concerns by splitting the former EvalContext into CodeGenerationContext and CodeExecutionContext, and introduces an eval-stage intern::Interner to preserve interned pointer identity across the MIR → eval boundary. Test harnesses and orchestrator utilities are updated to construct typed Value inputs directly (without the JSON decoder/type-env dependency) and to pass the appropriate context/interner types through the compilation/execution stages.

Changes:

  • Split evaluation context into generation/execution contexts and updated Postgres compilation + orchestrator execution call sites accordingly.
  • Added eval-stage intern::Interner (convertible from MIR interner) and migrated decoder/tests to use it.
  • Reworked orchestrator test input construction to build heap-allocated typed Value trees directly; graph API compile_query was stubbed pending BE-537.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
libs/@local/hashql/eval/tests/orchestrator/output.rs Generalized success rendering to support non-Global value allocators.
libs/@local/hashql/eval/tests/orchestrator/main.rs Updated test driver to consume lowered artifacts and pass symbol intern sets into input builder.
libs/@local/hashql/eval/tests/orchestrator/inputs.rs Replaced JSON-decoder-based input construction with direct typed Value construction using heap allocation + struct builders.
libs/@local/hashql/eval/tests/orchestrator/execution.rs Updated lowering/execution glue to use MIR interner → eval interner conversion and the new context split; switched to Orchestrator::run_in.
libs/@local/hashql/eval/src/postgres/projections.rs Added #[inline] on a conversion impl.
libs/@local/hashql/eval/src/postgres/parameters.rs Added #[inline] on a conversion impl.
libs/@local/hashql/eval/src/postgres/mod.rs Switched compiler context type to CodeGenerationContext and updated docs accordingly.
libs/@local/hashql/eval/src/postgres/filter/tests.rs Migrated tests to CodeGenerationContext and eval-stage test interner helper.
libs/@local/hashql/eval/src/postgres/filter/mod.rs Updated filter compiler to reference CodeGenerationContext.
libs/@local/hashql/eval/src/orchestrator/partial.rs Switched to eval-stage Interner in hydration pipeline.
libs/@local/hashql/eval/src/orchestrator/mod.rs Updated orchestrator to depend on CodeExecutionContext; added allocator-parametric run_in usage in call sites.
libs/@local/hashql/eval/src/orchestrator/events.rs Added #[inline] on Default impl.
libs/@local/hashql/eval/src/orchestrator/codec/decode/tests.rs Updated decoder tests to use eval-stage Interner::testing.
libs/@local/hashql/eval/src/orchestrator/codec/decode/mod.rs Switched decoder interner reference to the eval-stage interner.
libs/@local/hashql/eval/src/lib.rs Exported new intern module.
libs/@local/hashql/eval/src/intern.rs Introduced eval-stage Interner convertible from MIR interner (preserves interned identity).
libs/@local/hashql/eval/src/error.rs Replaced graph-gated diagnostic category variant with an orchestrator-backed variant.
libs/@local/hashql/eval/src/context.rs Implemented CodeGenerationContext / CodeExecutionContext split and From conversion.
libs/@local/hashql/core/src/graph/linked.rs Minor Debug impl formatting cleanup + added #[inline] on accessors.
libs/@local/hashql/compiletest/src/suite/eval_postgres.rs Updated compiletest suite to use CodeGenerationContext and eval interner conversion.
libs/@local/graph/api/src/rest/entity_query_request.rs Added lint expectation and stubbed compile_query while HashQL API wiring is pending BE-537.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread libs/@local/hashql/eval/tests/orchestrator/inputs.rs
Comment thread libs/@local/graph/api/src/rest/entity_query_request.rs Outdated
@codspeed-hq

codspeed-hq Bot commented Jun 8, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 80 untouched benchmarks


Comparing bm/be-587-hashql-eval-interner-and-context-restructuring (3d05f60) with bm/be-581-hashql-cached-constructor-produces-bare-fnptr-instead-of (b939729)

Open in CodSpeed

@indietyp indietyp force-pushed the bm/be-581-hashql-cached-constructor-produces-bare-fnptr-instead-of branch from 74f8c73 to a203514 Compare June 8, 2026 11:55
@indietyp indietyp force-pushed the bm/be-587-hashql-eval-interner-and-context-restructuring branch from 8af8844 to 82bb954 Compare June 8, 2026 11:55
Copilot AI review requested due to automatic review settings June 8, 2026 12:22
@indietyp indietyp force-pushed the bm/be-587-hashql-eval-interner-and-context-restructuring branch from 82bb954 to fa1c26d Compare June 8, 2026 12:22
@indietyp indietyp force-pushed the bm/be-581-hashql-cached-constructor-produces-bare-fnptr-instead-of branch from a203514 to eb135ab Compare June 8, 2026 12:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.

Comment thread libs/@local/hashql/eval/tests/orchestrator/inputs.rs Outdated
Comment thread libs/@local/hashql/eval/tests/orchestrator/execution.rs
Comment thread libs/@local/hashql/eval/tests/orchestrator/main.rs
Comment thread libs/@local/graph/api/src/rest/entity_query_request.rs
@indietyp indietyp force-pushed the bm/be-587-hashql-eval-interner-and-context-restructuring branch from fa1c26d to e8db60a Compare June 8, 2026 13:05
@indietyp indietyp force-pushed the bm/be-581-hashql-cached-constructor-produces-bare-fnptr-instead-of branch from eb135ab to 631682a Compare June 8, 2026 13:05
Copilot AI review requested due to automatic review settings June 9, 2026 07:17
@indietyp indietyp force-pushed the bm/be-581-hashql-cached-constructor-produces-bare-fnptr-instead-of branch from 631682a to b939729 Compare June 9, 2026 07:17
@indietyp indietyp force-pushed the bm/be-587-hashql-eval-interner-and-context-restructuring branch from e8db60a to 4a8d4fd Compare June 9, 2026 07:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.

Comment thread libs/@local/graph/api/src/rest/entity_query_request.rs
Comment thread libs/@local/graph/api/src/rest/entity_query_request.rs
Comment thread libs/@local/hashql/eval/src/postgres/filter/tests.rs Outdated
Comment thread libs/@local/hashql/eval/tests/orchestrator/inputs.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/libs Relates to first-party libraries/crates/packages (area) area/tests New or updated tests type/eng > backend Owned by the @backend team

Development

Successfully merging this pull request may close these issues.

2 participants