Skip to content

Conversation

@indietyp
Copy link
Member

🌟 What is the purpose of this PR?

Implement statement coloring (cost analysis) for the HashQL MIR execution analysis module. This enables the compiler to determine which statements can be executed on different backends (Postgres, Embedding DB, Interpreter) by assigning costs to each statement.

🔗 Related links

🚫 Blocked by

N/A

🔍 What does this change?

New execution analysis module (pass/analysis/execution/):

  • cost.rs: Cost type with overflow-safe arithmetic, StatementCostVec and TraversalCostVec for tracking costs per statement/traversal
  • target.rs: ExecutionTarget trait and implementations for Postgres, Interpreter, Embedding
  • statement_placement/: Framework for computing which statements can run on each backend
    • common.rs: Shared cost visitor logic
    • postgres/: Postgres backend - supports binary/unary ops, aggregates (tuple/struct), input, entity projections (JSONB/columns)
    • embedding/: Embedding DB backend - only supports entity.encodings.vectors projections
    • interpret/: Interpreter backend - universal fallback, supports all statements
    • lookup/: Entity projection path lookup trie mapping field paths to Access (Postgres column, JSONB, Embedding)

Pretty printer enhancements:

  • Added TextFormatAnnotations trait for annotating MIR output with cost information
  • Refactored to use line buffering for proper annotation alignment
  • Cost annotations appear as // cost: N comments on statements

Symbol table additions:

  • Added sym::lexical::foo, bar, unknown for testing
  • Changed PathNode.name from Symbol<'static> to &'static Symbol<'static> to prevent symbol duplication across crates

Type visitor improvements:

  • Extended TypeVisitor with additional traversal methods

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph

🛡 What tests cover this?

  • Unit tests in cost.rs (Cost arithmetic, StatementCostVec indexing)
  • Snapshot tests for statement placement across all three backends (16 new snapshots)
  • Unit tests for entity projection lookup in lookup/tests.rs
  • Unit tests for ExecutionTarget in target.rs

❓ How to test this?

cargo nextest run --package hashql-mir
cargo insta test --package hashql-mir

@vercel
Copy link

vercel bot commented Jan 29, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hash Ready Ready Preview, Comment Feb 2, 2026 4:42pm
petrinaut Error Error Feb 2, 2026 4:42pm
2 Skipped Deployments
Project Deployment Actions Updated (UTC)
hashdotdesign Ignored Ignored Preview Feb 2, 2026 4:42pm
hashdotdesign-tokens Ignored Ignored Preview Feb 2, 2026 4:42pm

@cursor
Copy link

cursor bot commented Jan 29, 2026

PR Summary

Medium Risk
Introduces new execution-cost analysis and backend placement logic plus refactors core/type visiting and MIR pretty-printing, so regressions could affect optimizer decisions and test tooling despite being largely internal and well-tested.

Overview
Adds a new MIR execution analysis module that assigns per-statement and per-traversal costs and computes backend placement for Postgres, Embedding, and Interpreter, including entity-field projection lookup via a schema trie.

Refactors MIR pretty-printing to use TextFormatOptions + TextFormatAnnotations (supporting inline cost comments) and updates existing tests/formatters accordingly; extends the body! test macro to support named-field projections and [Opaque ...] types, and wires new symbol-path constants (e.g., sym::path::Entity) used by the placement logic.

Includes supporting core changes: type::visit::Visitor now returns a Try-based result with helper recursion guard, DenseBitSet gains a set helper, and several MIR/interpret helpers become const (with updated miri test selection and minor docs fixes).

Written by Cursor Bugbot for commit cc929a8. This will update automatically on new commits. Configure here.

@github-actions github-actions bot added the area/infra Relates to version control, CI, CD or IaC (area) label Jan 29, 2026
@vercel vercel bot temporarily deployed to Preview – petrinaut January 29, 2026 20:46 Inactive
@indietyp indietyp force-pushed the bm/be-317-hashql-projectionextraction-for-entity-access branch from 6ffafd7 to b4bf91a Compare January 29, 2026 20:48
@indietyp indietyp force-pushed the bm/be-302-hashql-statement-coloring-based-on-capabilities branch from 738c295 to d838213 Compare January 29, 2026 20:48
@indietyp indietyp force-pushed the bm/be-317-hashql-projectionextraction-for-entity-access branch from b4bf91a to 8f967ee Compare January 30, 2026 10:36
@indietyp indietyp force-pushed the bm/be-302-hashql-statement-coloring-based-on-capabilities branch from d838213 to e81fd90 Compare January 30, 2026 10:36
@github-actions
Copy link
Contributor

Benchmark results

@rust/hash-graph-benches – Integrations

policy_resolution_large

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 2002 $$26.4 \mathrm{ms} \pm 117 \mathrm{μs}\left({\color{gray}-2.585 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$3.24 \mathrm{ms} \pm 20.0 \mathrm{μs}\left({\color{gray}0.813 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 1001 $$12.3 \mathrm{ms} \pm 117 \mathrm{μs}\left({\color{gray}-0.026 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 3314 $$42.3 \mathrm{ms} \pm 330 \mathrm{μs}\left({\color{gray}0.042 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$14.3 \mathrm{ms} \pm 101 \mathrm{μs}\left({\color{gray}3.27 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 1526 $$24.0 \mathrm{ms} \pm 146 \mathrm{μs}\left({\color{gray}0.958 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 2078 $$43.0 \mathrm{ms} \pm 281 \mathrm{μs}\left({\color{gray}-3.779 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$20.0 \mathrm{ms} \pm 112 \mathrm{μs}\left({\color{lightgreen}-12.882 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 1033 $$31.1 \mathrm{ms} \pm 217 \mathrm{μs}\left({\color{gray}2.91 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_medium

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 102 $$3.56 \mathrm{ms} \pm 17.2 \mathrm{μs}\left({\color{gray}-1.832 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.81 \mathrm{ms} \pm 13.5 \mathrm{μs}\left({\color{gray}-0.586 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 51 $$3.14 \mathrm{ms} \pm 12.8 \mathrm{μs}\left({\color{gray}-1.807 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 269 $$4.96 \mathrm{ms} \pm 27.0 \mathrm{μs}\left({\color{gray}-0.605 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$3.36 \mathrm{ms} \pm 17.4 \mathrm{μs}\left({\color{gray}-0.098 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 107 $$3.91 \mathrm{ms} \pm 20.0 \mathrm{μs}\left({\color{gray}-0.531 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 133 $$3.96 \mathrm{ms} \pm 25.5 \mathrm{μs}\left({\color{lightgreen}-6.599 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$3.20 \mathrm{ms} \pm 16.1 \mathrm{μs}\left({\color{gray}-2.710 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 63 $$3.82 \mathrm{ms} \pm 19.7 \mathrm{μs}\left({\color{gray}-2.265 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_none

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 2 $$2.38 \mathrm{ms} \pm 11.8 \mathrm{μs}\left({\color{gray}1.61 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.33 \mathrm{ms} \pm 10.3 \mathrm{μs}\left({\color{gray}1.01 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 1 $$2.41 \mathrm{ms} \pm 12.7 \mathrm{μs}\left({\color{gray}1.44 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 8 $$2.63 \mathrm{ms} \pm 13.3 \mathrm{μs}\left({\color{gray}0.971 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$2.50 \mathrm{ms} \pm 13.6 \mathrm{μs}\left({\color{gray}1.14 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 3 $$2.71 \mathrm{ms} \pm 15.3 \mathrm{μs}\left({\color{gray}1.14 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_small

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 52 $$2.78 \mathrm{ms} \pm 12.7 \mathrm{μs}\left({\color{gray}0.865 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.37 \mathrm{ms} \pm 12.7 \mathrm{μs}\left({\color{gray}-1.293 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 25 $$2.56 \mathrm{ms} \pm 9.23 \mathrm{μs}\left({\color{gray}-0.901 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 94 $$3.09 \mathrm{ms} \pm 17.3 \mathrm{μs}\left({\color{gray}-0.470 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$2.62 \mathrm{ms} \pm 11.1 \mathrm{μs}\left({\color{gray}-0.245 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 26 $$2.86 \mathrm{ms} \pm 11.7 \mathrm{μs}\left({\color{gray}0.291 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 66 $$3.01 \mathrm{ms} \pm 16.2 \mathrm{μs}\left({\color{gray}1.60 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$2.59 \mathrm{ms} \pm 10.3 \mathrm{μs}\left({\color{gray}-0.707 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 29 $$2.84 \mathrm{ms} \pm 12.8 \mathrm{μs}\left({\color{gray}-0.218 \mathrm{\%}}\right) $$ Flame Graph

read_scaling_complete

Function Value Mean Flame graphs
entity_by_id;one_depth 1 entities $$38.6 \mathrm{ms} \pm 155 \mathrm{μs}\left({\color{gray}-0.728 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 10 entities $$75.8 \mathrm{ms} \pm 343 \mathrm{μs}\left({\color{gray}-1.739 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 25 entities $$43.1 \mathrm{ms} \pm 192 \mathrm{μs}\left({\color{gray}-0.004 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 5 entities $$45.6 \mathrm{ms} \pm 248 \mathrm{μs}\left({\color{gray}0.815 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 50 entities $$52.0 \mathrm{ms} \pm 246 \mathrm{μs}\left({\color{gray}-2.634 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 1 entities $$40.8 \mathrm{ms} \pm 172 \mathrm{μs}\left({\color{gray}0.330 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 10 entities $$419 \mathrm{ms} \pm 952 \mathrm{μs}\left({\color{gray}0.984 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 25 entities $$93.6 \mathrm{ms} \pm 434 \mathrm{μs}\left({\color{gray}-1.832 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 5 entities $$83.3 \mathrm{ms} \pm 354 \mathrm{μs}\left({\color{gray}-0.599 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 50 entities $$310 \mathrm{ms} \pm 786 \mathrm{μs}\left({\color{red}10.9 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 1 entities $$14.2 \mathrm{ms} \pm 54.2 \mathrm{μs}\left({\color{gray}-1.700 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 10 entities $$14.8 \mathrm{ms} \pm 63.0 \mathrm{μs}\left({\color{gray}-0.057 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 25 entities $$14.9 \mathrm{ms} \pm 71.1 \mathrm{μs}\left({\color{gray}-1.109 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 5 entities $$14.5 \mathrm{ms} \pm 59.1 \mathrm{μs}\left({\color{gray}-1.096 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 50 entities $$16.9 \mathrm{ms} \pm 113 \mathrm{μs}\left({\color{gray}-3.420 \mathrm{\%}}\right) $$ Flame Graph

read_scaling_linkless

Function Value Mean Flame graphs
entity_by_id 1 entities $$14.6 \mathrm{ms} \pm 74.9 \mathrm{μs}\left({\color{gray}-0.709 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$14.2 \mathrm{ms} \pm 62.1 \mathrm{μs}\left({\color{gray}-1.370 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 100 entities $$14.3 \mathrm{ms} \pm 50.3 \mathrm{μs}\left({\color{gray}-1.821 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1000 entities $$14.9 \mathrm{ms} \pm 72.9 \mathrm{μs}\left({\color{gray}-0.842 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10000 entities $$21.8 \mathrm{ms} \pm 143 \mathrm{μs}\left({\color{gray}-1.647 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity

Function Value Mean Flame graphs
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1 $$31.3 \mathrm{ms} \pm 306 \mathrm{μs}\left({\color{gray}4.42 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1 $$30.8 \mathrm{ms} \pm 327 \mathrm{μs}\left({\color{gray}1.71 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1 $$30.1 \mathrm{ms} \pm 305 \mathrm{μs}\left({\color{gray}4.80 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1 $$31.0 \mathrm{ms} \pm 306 \mathrm{μs}\left({\color{red}5.31 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2 $$31.1 \mathrm{ms} \pm 286 \mathrm{μs}\left({\color{gray}1.30 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1 $$29.8 \mathrm{ms} \pm 271 \mathrm{μs}\left({\color{gray}-2.491 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1 $$30.0 \mathrm{ms} \pm 341 \mathrm{μs}\left({\color{gray}-0.675 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1 $$30.9 \mathrm{ms} \pm 344 \mathrm{μs}\left({\color{gray}2.93 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1 $$30.1 \mathrm{ms} \pm 289 \mathrm{μs}\left({\color{gray}2.13 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity_type

Function Value Mean Flame graphs
get_entity_type_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba $$8.12 \mathrm{ms} \pm 33.0 \mathrm{μs}\left({\color{gray}2.91 \mathrm{\%}}\right) $$ Flame Graph

representative_read_multiple_entities

Function Value Mean Flame graphs
entity_by_property traversal_paths=0 0 $$46.8 \mathrm{ms} \pm 275 \mathrm{μs}\left({\color{gray}-1.167 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=255 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true $$92.9 \mathrm{ms} \pm 396 \mathrm{μs}\left({\color{gray}-0.175 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false $$52.4 \mathrm{ms} \pm 249 \mathrm{μs}\left({\color{gray}1.05 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true $$60.8 \mathrm{ms} \pm 325 \mathrm{μs}\left({\color{gray}0.833 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true $$68.0 \mathrm{ms} \pm 359 \mathrm{μs}\left({\color{gray}-0.115 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true $$74.0 \mathrm{ms} \pm 376 \mathrm{μs}\left({\color{gray}-1.112 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=0 0 $$49.4 \mathrm{ms} \pm 202 \mathrm{μs}\left({\color{gray}-1.089 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=255 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true $$76.6 \mathrm{ms} \pm 311 \mathrm{μs}\left({\color{gray}-0.426 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false $$56.7 \mathrm{ms} \pm 296 \mathrm{μs}\left({\color{gray}-0.209 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true $$64.7 \mathrm{ms} \pm 246 \mathrm{μs}\left({\color{gray}0.139 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true $$67.1 \mathrm{ms} \pm 327 \mathrm{μs}\left({\color{gray}0.509 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true $$67.0 \mathrm{ms} \pm 359 \mathrm{μs}\left({\color{gray}0.914 \mathrm{\%}}\right) $$

scenarios

Function Value Mean Flame graphs
full_test query-limited $$135 \mathrm{ms} \pm 543 \mathrm{μs}\left({\color{gray}2.34 \mathrm{\%}}\right) $$ Flame Graph
full_test query-unlimited $$134 \mathrm{ms} \pm 468 \mathrm{μs}\left({\color{gray}1.99 \mathrm{\%}}\right) $$ Flame Graph
linked_queries query-limited $$84.7 \mathrm{ms} \pm 2.88 \mathrm{ms}\left({\color{lightgreen}-18.172 \mathrm{\%}}\right) $$ Flame Graph
linked_queries query-unlimited $$570 \mathrm{ms} \pm 724 \mathrm{μs}\left({\color{gray}-2.716 \mathrm{\%}}\right) $$ Flame Graph

@indietyp indietyp force-pushed the bm/be-317-hashql-projectionextraction-for-entity-access branch from 8f967ee to dbbeb22 Compare January 30, 2026 23:24
@indietyp indietyp force-pushed the bm/be-302-hashql-statement-coloring-based-on-capabilities branch from e81fd90 to 0af7773 Compare January 30, 2026 23:24
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) area/tests New or updated tests type/eng > backend Owned by the @backend team

Development

Successfully merging this pull request may close these issues.

2 participants