Skip to content

feat(ampsync): add configurable index creation on table setup#1985

Open
matiasedgeandnode wants to merge 1 commit intomainfrom
ma/ampsync-indexes
Open

feat(ampsync): add configurable index creation on table setup#1985
matiasedgeandnode wants to merge 1 commit intomainfrom
ma/ampsync-indexes

Conversation

@matiasedgeandnode
Copy link

@matiasedgeandnode matiasedgeandnode commented Mar 17, 2026

Summary

  • Adds --config / AMPSYNC_CONFIG option to specify a TOML file with index definitions
  • Indexes are created alongside the table on first startup (no-op if table exists)
  • Supports btree/hash/gin/gist/brin methods, unique indexes, column sort order (ASC/DESC), covering indexes (INCLUDE), and partial indexes (WHERE)
  • All identifiers validated via sqlparser + pg_escape to prevent SQL injection; WHERE clauses parsed for syntactic validity

Config example

[[index]]
table = "usdc_transfers"
name = "idx_buyer_timestamp"
columns = ["buyer_address", "timestamp desc"]
include = ["value_usdc"]

Test plan

  • just check-crate ampsync passes
  • just clippy-crate ampsync passes (zero warnings)
  • cargo nextest run -p ampsync — unit tests pass (config parsing, SQL generation, validation, identifier rejection)
  • Integration: run ampsync with --config ampsync.toml against a fresh DB, verify indexes are created
  • Integration: restart ampsync against existing table, verify it skips creation

@matiasedgeandnode matiasedgeandnode force-pushed the ma/ampsync-indexes branch 3 times, most recently from 910a945 to 1719948 Compare March 17, 2026 16:35
@matiasedgeandnode matiasedgeandnode marked this pull request as ready for review March 17, 2026 16:35
/// See the "Index Configuration" section in the crate README for format details.
///
/// Can also be set via AMPSYNC_CONFIG environment variable
#[arg(long, env = "AMPSYNC_CONFIG")]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest using a different name here, index-config or something like that. Otherwise it sounds like the configuration file to run ampsync, where i would expect to be able to set things like auth, table, etc (basically all other config).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants