Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Summary by CodeRabbitRelease Notes
WalkthroughIntroduces comprehensive CDC (Change Data Capture) support across the framework. Adds CDC data structures, integration into infrastructure mapping and planning, validation, and display logic. Includes Python and TypeScript SDKs, examples, and documentation for CDC source configuration and streaming transforms. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant CLI as Framework CLI
participant Infra as InfrastructureMap
participant Validator as Plan Validator
participant Display as Display Engine
User->>CLI: Define CdcSource & CdcTable
CLI->>Infra: Load partial CDC definitions
activate Infra
Infra->>Infra: Resolve runtime env (connection)
Infra->>Infra: Convert to full CDC types
deactivate Infra
CLI->>Validator: Validate plan with CDC sources
activate Validator
Validator->>Validator: Check non-empty names/connections
Validator->>Validator: Check table configs (primary_key, source_table)
deactivate Validator
CLI->>Display: Render plan output
activate Display
Display->>Display: Show CDC sources in infrastructure map
Display->>Display: Display ProcessChange::CdcSource variants
deactivate Display
Display-->>User: Plan with CDC resources
Estimated code review effort🎯 4 (Complex) | ⏱️ ~75 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 3 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
| metadata=table.metadata, | ||
| ) | ||
| for table in source.tables.values() | ||
| ] |
There was a problem hiding this comment.
Variable shadowing loses all table definitions
High Severity
The loop variable tables at line 1186 shadows the outer tables dictionary initialized at line 989. In Python, this reassignment overwrites the dictionary containing all TableConfig entries. When the InfrastructureMap is constructed at line 1211, the tables variable holds only the list from the last CDC source iteration instead of the dictionary of OLAP tables. This causes all OLAP table definitions to be lost from the infrastructure map when any CDC source exists.
| const eventColumns = buildCdcEventColumns(this.columnArray); | ||
| const streamConfig: StreamConfig<CdcEvent<T>> = { | ||
| ...(typeof config.stream === "object" ? config.stream : {}), | ||
| lifeCycle: config.lifeCycle, |
There was a problem hiding this comment.
Stream config lifeCycle overwritten by top-level value
Medium Severity
When constructing the streamConfig, the lifeCycle property is unconditionally set to config.lifeCycle, overwriting any lifeCycle that may have been provided in the user's custom stream config object (config.stream). This is inconsistent with the table config handling at line 159, which correctly uses nullish coalescing (config.table.lifeCycle ?? config.lifeCycle) to preserve nested config values. A user providing { stream: { lifeCycle: LifeCycle.DeletionProtected } } would have their stream lifecycle unexpectedly replaced.
| from __future__ import annotations | ||
|
|
||
| from datetime import datetime | ||
| from typing import Optional, Generic, Literal, TypeVar, Union, Any |
There was a problem hiding this comment.
Unused imports in Python CDC source module
Low Severity
TypeVar and Union are imported from typing but never used in the module. The generic type variable T is imported from .types instead, and Union is not needed since the code uses Python 3.10+ union syntax (bool | StreamConfig).
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 21
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
apps/framework-cli-e2e/test/templates.test.ts (1)
215-228: 🧹 Nitpick | 🔵 TrivialExtract the CDC connection string and confirm linting.
The literal is duplicated across both env blocks; a shared constant reduces drift. Please also confirm TS lint ran per guidelines.♻️ Proposed refactor
+const TEST_CDC_CONNECTION = + "postgres://test_user:test_pass@localhost:5432/test_db"; @@ - TEST_CDC_CONNECTION: - "postgres://test_user:test_pass@localhost:5432/test_db", + TEST_CDC_CONNECTION, @@ - TEST_CDC_CONNECTION: - "postgres://test_user:test_pass@localhost:5432/test_db", + TEST_CDC_CONNECTION,
🤖 Fix all issues with AI agents
In `@apps/framework-cli-e2e/test/lifecycle.test.ts`:
- Around line 254-263: Guard access to plan.changes.processes_changes to avoid a
TypeError when changes is undefined: update the hasCdcChange computation (the
expression that currently calls plan.changes.processes_changes.some(...)) to use
optional chaining and a fallback (e.g.,
plan.changes?.processes_changes?.some(...) ?? false) and keep the existing
expect(hasCdcChange).to.be.true assertion; this references runMoosePlanJson and
the plan.changes/processes_changes symbols to locate the change.
In `@apps/framework-cli/src/framework/core/infrastructure/cdc_source.rs`:
- Around line 9-148: Add rustdoc comments to the public structs and methods:
document CdcTable, CdcSource and their methods to_proto(), from_proto(),
expanded_display(), short_display() describing purpose, fields, and expected
behavior; then add unit tests that construct representative CdcTable and
CdcSource instances (including metadata and lifecycle variants), assert
round-trip fidelity via to_proto() -> from_proto(), and verify
expanded_display()/short_display() outputs and that metadata/source fields map
correctly between proto and struct. Ensure tests cover edge cases like empty
description/source None and each LifeCycle variant so conversions in
CdcTable::to_proto/from_proto and CdcSource::to_proto/from_proto are validated.
In `@apps/framework-cli/src/framework/core/infrastructure/mod.rs`:
- Line 24: Add Rustdoc comments to the public module cdc_source and its public
types and methods: document the CdcTable and CdcSource structs and each public
method (to_proto, from_proto, expanded_display, short_display). For each item
provide a brief one-line description of its purpose, document parameters and
return values for conversion/formatting methods (e.g., to_proto/from_proto), and
include any relevant usage notes or examples for expanded_display/short_display
output expectations; ensure comments use /// style above the items in the
cdc_source module so they appear in generated docs.
In `@apps/framework-cli/src/framework/core/partial_infrastructure_map.rs`:
- Around line 1442-1452: The error uses DmV2LoadingError::RuntimeEnvResolution
with a table_name which will mislabel CDC sources; update convert_cdc_sources to
produce a CDC-specific error variant or use a neutral field name: change the
error construction inside convert_cdc_sources (where
resolve_runtime_env(&source.connection) is mapped_err) to either call a new
DmV2LoadingError variant like RuntimeEnvResolutionForCdc { source_name:
name.clone(), field: "connection".to_string(), error: error.to_string() } or
adjust the existing RuntimeEnvResolution to use a neutral identifier (e.g.,
resource_name) and pass name.clone() so the message correctly refers to the CDC
source; ensure the enum definition for DmV2LoadingError is updated to include
the new variant or neutral field and update any display/formatting logic to use
the new field name.
- Around line 475-498: Add rustdoc comments for the public structs
PartialCdcTable and PartialCdcSource: for each struct add a brief ///
description above the struct explaining its purpose and how it is used in the
partial infrastructure map, and document each public field with /// comments
describing the field meaning (e.g., name, source_table, primary_key, stream,
table, snapshot, version, metadata for PartialCdcTable and name, kind,
connection, tables, metadata, life_cycle for PartialCdcSource) so both structs
and all their public fields have clear rustdoc documentation.
In `@apps/framework-cli/src/framework/core/plan_validator.rs`:
- Around line 78-111: The loop over plan.target_infra_map.cdc_sources currently
allows a CDC source with an empty source.tables list; add a check in the for
source in plan.target_infra_map.cdc_sources.values() block (before iterating
source.tables) that returns Err(ValidationError::CdcValidation(format!("CDC
source '{}' must define at least one table", source.name))) when
source.tables.is_empty(), so sources with zero tables fail validation.
In `@apps/framework-cli/src/infrastructure/processes/mod.rs`:
- Around line 206-222: The logs added in ProcessChange::CdcSource (messages
containing "no apply step in v1" for Change::Added/Removed/Updated) indicate CDC
sources are reviewable by plan but not appliable in v1; update the CDC
documentation page (connect-cdc.mdx) to clearly state that moose plan will
surface CDC source changes but moose apply cannot create/remove/update CDC
sources in v1, include an explicit note or warning and an example showing
plan-only behavior and what users should expect when they see the "no apply step
in v1" message.
In `@apps/framework-docs-v2/content/moosestack/streaming/connect-cdc.mdx`:
- Around line 123-133: The function to_orders_ingest is missing a type
annotation for its event parameter and should explicitly type its return; update
its signature to accept event: CdcEvent[OrdersIngestRow] (or the appropriate
generic) and return Optional[OrdersIngestRow]; also add the necessary imports
(from typing import Optional and the CdcEvent type) and adjust any import paths
so OrdersIngestRow and CdcEvent are referenced correctly.
In `@packages/protobuf/infrastructure_map.proto`:
- Around line 523-540: Add missing documentation comments for the Protocol
Buffer messages and their fields: add a brief message-level comment above
CdcSource describing its role, and field-level comments for name, kind,
connection, tables, metadata, and life_cycle; likewise add a message-level
comment for CdcTable and field-level comments for name, source_table,
primary_key, stream, table, snapshot, version, and metadata. Use the standard
proto comment style (// or /** */) and include purpose, expected
format/constraints (e.g., whether fields are optional, semantics of "kind" and
allowed values, what "connection" contains, how "primary_key" is used, and
meaning of stream/table/snapshot/version), and reference the LifeCycle enum
semantics so readers understand lifecycle handling.
In `@packages/py-moose-lib/moose_lib/dmv2/_registry.py`:
- Line 25: Add documentation pages for the new public CDC APIs get_cdc_sources()
and get_cdc_source() under apps/framework-docs-v2/content following py-moose-lib
public API guidelines: create a short doc (usage, parameters, return values,
examples) for get_cdc_sources and one for get_cdc_source, reference the
functions by name exactly, and link them from the library API index so the new
public accessors are discoverable before merging.
In `@packages/py-moose-lib/moose_lib/dmv2/cdc_source.py`:
- Around line 51-57: Replace PEP 604 union annotations with typing.Union for the
CDC source fields: change the type annotations on stream and table from "bool |
StreamConfig" and "bool | OlapConfig" to "Union[bool, StreamConfig]" and
"Union[bool, OlapConfig]" respectively, and add/ensure "from typing import
Union" is imported at the top of moose_lib/dmv2/cdc_source.py so the new
annotations resolve; keep other Optional and Literal annotations unchanged
(refer to the stream and table fields in the file to locate the changes).
- Around line 35-40: The CdcRow model defines dunder-prefixed attributes
(__cdc_op, __cdc_lsn, __cdc_ts, __cdc_is_deleted) which Pydantic ignores due to
name-mangling; change those names to non-dunder identifiers (e.g., _cdc_op or
cdc_op) in the CdcRow class definition and update any code that references
CdcRow.__cdc_op / __cdc_lsn / __cdc_ts / __cdc_is_deleted to use the new names
(CdcRow._cdc_op or CdcRow.cdc_op) so instantiation, validation and serialization
work correctly; ensure any validators, default values, imports (CdcOperation,
datetime) and type hints stay consistent with the renamed fields.
In `@packages/py-moose-lib/moose_lib/dmv2/registry.py`:
- Around line 141-148: The functions get_cdc_sources and get_cdc_source
reference the CdcSource type but it isn't imported; add CdcSource to the
module's top-level type imports alongside the other registry types, then update
the return annotations in get_cdc_sources and get_cdc_source to use the real
type (remove the quotes around "CdcSource") so the annotations read Dict[str,
CdcSource] and Optional[CdcSource], ensuring _cdc_sources stays used as the
backing dict.
In `@packages/py-moose-lib/moose_lib/internal.py`:
- Around line 1185-1208: The local variable named `tables` inside the loop over
get_cdc_sources() shadows the outer `tables` dict; rename the inner variable
(for example to `cdc_tables` or `table_list`) where it is created in the list
comprehension and update its use when constructing the CdcSourceJson (e.g., pass
`tables=cdc_tables`), leaving get_cdc_sources(), CdcTableJson, and CdcSourceJson
unchanged otherwise.
In `@packages/ts-moose-lib/src/dmv2/index.ts`:
- Around line 73-81: The new public CDC APIs exported from ts-moose-lib
(CdcSource, CdcSourceConfig, CdcTable, CdcTableConfig, CdcEvent, CdcOperation,
CdcRow and the functions getCdcSources/getCdcSource) are not documented in
apps/framework-docs-v2; add documentation pages under
apps/framework-docs-v2/content that introduce CDC, create an API reference page
for each exported symbol (include purpose, key fields/params, types, and short
usage examples showing getCdcSources/getCdcSource), and update the docs
sidebar/index so these pages are discoverable; ensure names match the exported
identifiers (CdcSource, CdcSourceConfig, CdcTable, CdcTableConfig, CdcEvent,
CdcOperation, CdcRow, getCdcSources, getCdcSource).
In `@packages/ts-moose-lib/src/dmv2/registry.ts`:
- Around line 216-231: Add documentation entries for the new public API
functions getCdcSources and getCdcSource(name: string) in the reference docs
file (apps/framework-docs-v2/content/moosestack/reference/index.mdx). For each
function (getCdcSources and getCdcSource), add a short description, signatures,
return types, and example usage consistent with the style used for other
registry getters (e.g., getTables/getStreams/getWorkflows), and include notes
about returned types (Map<string, CdcSource> and CdcSource | undefined) and when
undefined is returned; ensure these docs are linked into the reference index so
they appear in the published docs.
In `@packages/ts-moose-lib/src/dmv2/sdk/cdcSource.ts`:
- Around line 29-45: Replace the object-shaped type aliases with interfaces:
change the declarations of CdcSourceConfig and CdcTableConfig to interface
declarations preserving all properties and their types (including optional
metadata, lifeCycle, generic parameter T and nested StreamConfig/OlapConfig
usages); keep the exported names, generics, and property signatures identical
but use the interface keyword so they follow the project guideline to prefer
interfaces for object-shaped configs.
- Around line 206-231: The op property in buildCdcEventSchema currently allows
any string; tighten validation by replacing op: { type: "string" } with an enum
of the known CDC operation values (e.g., for Debezium-style ops use { type:
"string", enum: ["c","u","d","r"] } or whatever project-specific set is
correct), updating the schema returned by buildCdcEventSchema so op is
constrained to that enum and adjusting any callers/tests if they rely on broader
values.
- Around line 72-74: The public method registerTable(table: CdcTable<any>) lacks
an explicit return type; update its signature to include an explicit void return
type (e.g., registerTable(...): void) so the exported API complies with the
coding guideline, leaving the body unchanged; reference the registerTable method
and the CdcTable type to locate the change in cdcSource.ts.
In `@templates/python-tests/src/ingest/cdc.py`:
- Around line 23-29: Add explicit PEP 8–compliant type hints to the
to_orders_ingest function signature: annotate its parameters (e.g., the
env/runtime input types used to construct orders_cdc_source and any other args)
and its return type (likely the ingest/config object or CdcSource) so callers
and linters can infer types; update the to_orders_ingest declaration (function
name: to_orders_ingest) to include proper parameter types and a return
annotation (referencing CdcSource or the specific ingest/config class used)
while keeping the existing construction of orders_cdc_source and CdcSourceConfig
unchanged.
- Around line 57-68: Add type hints to the to_orders_ingest function signature:
annotate the parameter (event) and the return type as Optional[OrdersIngestRow];
if a concrete CDC event type exists in the codebase use that (e.g.,
DebeziumEvent) otherwise import typing.Any and annotate event: Any. Also ensure
typing.Optional (and Any if used) is imported at the top of the module and keep
the existing body unchanged so the function becomes def to_orders_ingest(event:
<EventType>) -> Optional[OrdersIngestRow].
📜 Review details
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (35)
apps/framework-cli-e2e/test/lifecycle.test.tsapps/framework-cli-e2e/test/templates.test.tsapps/framework-cli-e2e/test/utils/plan-utils.tsapps/framework-cli/src/cli/display/infrastructure.rsapps/framework-cli/src/cli/routines/peek.rsapps/framework-cli/src/cli/routines/seed_data.rsapps/framework-cli/src/framework/core/infra_reality_checker.rsapps/framework-cli/src/framework/core/infrastructure/cdc_source.rsapps/framework-cli/src/framework/core/infrastructure/mod.rsapps/framework-cli/src/framework/core/infrastructure_map.rsapps/framework-cli/src/framework/core/partial_infrastructure_map.rsapps/framework-cli/src/framework/core/plan_validator.rsapps/framework-cli/src/infrastructure/processes/mod.rsapps/framework-docs-v2/content/moosestack/configuration.mdxapps/framework-docs-v2/content/moosestack/deploying/configuring-moose-for-cloud.mdxapps/framework-docs-v2/content/moosestack/streaming/connect-cdc.mdxexamples/cdp-analytics/packages/moosestack-service/app/workflows/cdc-placeholder.tspackages/protobuf/infrastructure_map.protopackages/py-moose-lib/moose_lib/dmv2/__init__.pypackages/py-moose-lib/moose_lib/dmv2/_registry.pypackages/py-moose-lib/moose_lib/dmv2/cdc_source.pypackages/py-moose-lib/moose_lib/dmv2/registry.pypackages/py-moose-lib/moose_lib/internal.pypackages/ts-moose-lib/src/browserCompatible.tspackages/ts-moose-lib/src/dmv2/dataModelMetadata.tspackages/ts-moose-lib/src/dmv2/index.tspackages/ts-moose-lib/src/dmv2/internal.tspackages/ts-moose-lib/src/dmv2/registry.tspackages/ts-moose-lib/src/dmv2/sdk/cdcSource.tstemplates/python-tests/.envtemplates/python-tests/src/ingest/cdc.pytemplates/python-tests/src/main.pytemplates/typescript-tests/.envtemplates/typescript-tests/src/index.tstemplates/typescript-tests/src/ingest/cdc.ts
🧰 Additional context used
📓 Path-based instructions (11)
**/*.rs
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Run
cargo clippyto ensure Rust code passes Clippy's linting standards before each commit
**/*.rs: Usethiserrorwith#[derive(thiserror::Error)]for error handling in Rust; define errors near the fallibility unit, never useanyhow::Result
Use snake_case for functions and variables, PascalCase for types and traits, SCREAMING_SNAKE_CASE for constants in Rust
Place constants inconstants.rsat the appropriate module level in Rust
Use tuple structs with validation constructors (newtypes) for Rust type safety (e.g.,struct UserId(String))
Write inline tests with#[cfg(test)]modules in Rust
Documentation is required for all public APIs in Rust
Runcargo clippy --all-targets -- -D warningsbefore committing Rust code; no warnings allowed
Files:
apps/framework-cli/src/framework/core/plan_validator.rsapps/framework-cli/src/infrastructure/processes/mod.rsapps/framework-cli/src/framework/core/infrastructure/mod.rsapps/framework-cli/src/cli/routines/seed_data.rsapps/framework-cli/src/cli/routines/peek.rsapps/framework-cli/src/framework/core/infra_reality_checker.rsapps/framework-cli/src/framework/core/infrastructure/cdc_source.rsapps/framework-cli/src/framework/core/partial_infrastructure_map.rsapps/framework-cli/src/cli/display/infrastructure.rsapps/framework-cli/src/framework/core/infrastructure_map.rs
apps/framework-cli/**/*.rs
📄 CodeRabbit inference engine (apps/framework-cli/AGENTS.md)
apps/framework-cli/**/*.rs: Always runcargo clippy --all-targets -- -D warningsbefore commits; fix all warnings - no Clippy warnings may remain (treat warnings as errors)
Userustfmt --edition 2021for consistent formatting
Write meaningful names for functions, variables, and types
Document all public APIs and breaking changes
Usethiserrorcrate instead ofanyhow::Resultfor error handling
Define errors near their unit of fallibility (no global Error types)
Use#[derive(thiserror::Error)]with#[error()]messages for error definitions
Define newtypes as tuple structs:struct UserId(u64);
Add validation constructors for newtypes:UserId::new(id: u64) -> Result<Self, Error>
Derive standard traits for newtypes:#[derive(Debug, Clone, PartialEq)]
ImplementFrom/TryFromfor newtype conversions
Usederive_moreornutypeto reduce newtype boilerplate
Useconstfor static values (prefer overstatic)
UseUPPER_SNAKE_CASEnaming for constants
Scope constant visibility with preference:pub(crate)>pub(super)>pub
Group related constants together
Write unit tests for all public functions
Test error conditions and edge cases
Files:
apps/framework-cli/src/framework/core/plan_validator.rsapps/framework-cli/src/infrastructure/processes/mod.rsapps/framework-cli/src/framework/core/infrastructure/mod.rsapps/framework-cli/src/cli/routines/seed_data.rsapps/framework-cli/src/cli/routines/peek.rsapps/framework-cli/src/framework/core/infra_reality_checker.rsapps/framework-cli/src/framework/core/infrastructure/cdc_source.rsapps/framework-cli/src/framework/core/partial_infrastructure_map.rsapps/framework-cli/src/cli/display/infrastructure.rsapps/framework-cli/src/framework/core/infrastructure_map.rs
**/framework-cli/src/**
⚙️ CodeRabbit configuration file
**/framework-cli/src/**: When reviewing changes to Moose CLI:
- Check if any user-facing changes were made (commands, flags, configs, apis, etc)
- If yes, verify the documentation for THAT SPECIFIC feature is updated in apps/framework-docs-v2/content
- If docs for that feature doesn't exist yet, it should be added. If the change removes public apis, the documentation for those should also be removed. Changing unrelated docs doesn't satisfy this requirement.
- In addition to reviewing for docs discrepancies, you should also review the code as per usual guidelines.
Files:
apps/framework-cli/src/framework/core/plan_validator.rsapps/framework-cli/src/infrastructure/processes/mod.rsapps/framework-cli/src/framework/core/infrastructure/mod.rsapps/framework-cli/src/cli/routines/seed_data.rsapps/framework-cli/src/cli/routines/peek.rsapps/framework-cli/src/framework/core/infra_reality_checker.rsapps/framework-cli/src/framework/core/infrastructure/cdc_source.rsapps/framework-cli/src/framework/core/partial_infrastructure_map.rsapps/framework-cli/src/cli/display/infrastructure.rsapps/framework-cli/src/framework/core/infrastructure_map.rs
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Run linting checks before submitting PRs for TypeScript/JavaScript code
**/*.{ts,tsx,js,jsx}: Group imports by external dependencies, internal modules, and types; use named exports from barrel files (index.ts)
Use camelCase for variables and functions, PascalCase for types/classes/components, UPPER_SNAKE_CASE for constants in TypeScript/JavaScript
Prefix unused variables with underscore (e.g.,_unusedParam) to bypass linting errors in TypeScript/JavaScript
Format TypeScript/JavaScript code with Prettier usingexperimentalTernaries: true; auto-formats on commit via Husky + lint-staged
Files:
apps/framework-cli-e2e/test/utils/plan-utils.tspackages/ts-moose-lib/src/dmv2/dataModelMetadata.tsapps/framework-cli-e2e/test/lifecycle.test.tstemplates/typescript-tests/src/index.tspackages/ts-moose-lib/src/dmv2/registry.tspackages/ts-moose-lib/src/browserCompatible.tsapps/framework-cli-e2e/test/templates.test.tsexamples/cdp-analytics/packages/moosestack-service/app/workflows/cdc-placeholder.tspackages/ts-moose-lib/src/dmv2/index.tspackages/ts-moose-lib/src/dmv2/internal.tstemplates/typescript-tests/src/ingest/cdc.tspackages/ts-moose-lib/src/dmv2/sdk/cdcSource.ts
**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
Prefer interfaces for objects, types for unions/intersections in TypeScript; explicit return types on public APIs
Files:
apps/framework-cli-e2e/test/utils/plan-utils.tspackages/ts-moose-lib/src/dmv2/dataModelMetadata.tsapps/framework-cli-e2e/test/lifecycle.test.tstemplates/typescript-tests/src/index.tspackages/ts-moose-lib/src/dmv2/registry.tspackages/ts-moose-lib/src/browserCompatible.tsapps/framework-cli-e2e/test/templates.test.tsexamples/cdp-analytics/packages/moosestack-service/app/workflows/cdc-placeholder.tspackages/ts-moose-lib/src/dmv2/index.tspackages/ts-moose-lib/src/dmv2/internal.tstemplates/typescript-tests/src/ingest/cdc.tspackages/ts-moose-lib/src/dmv2/sdk/cdcSource.ts
apps/framework-docs-v2/content/**/*.mdx
📄 CodeRabbit inference engine (apps/framework-docs-v2/CLAUDE.md)
apps/framework-docs-v2/content/**/*.mdx: Use{{ include "shared/path.mdx" }}directives to reuse content fragments, which are processed viaprocessIncludes()during build
Validate code snippets in documentation with the@testdirective for TypeScript and Python code blocks
TypeScript code snippets in documentation should be validated for syntax with brace matching; Python snippets should be validated for indentation
Files:
apps/framework-docs-v2/content/moosestack/deploying/configuring-moose-for-cloud.mdxapps/framework-docs-v2/content/moosestack/configuration.mdxapps/framework-docs-v2/content/moosestack/streaming/connect-cdc.mdx
packages/protobuf/**/*.proto
📄 CodeRabbit inference engine (packages/protobuf/AGENTS.md)
packages/protobuf/**/*.proto: NEVER make breaking changes to existing.protofiles, including removing or renaming fields, changing field numbers or types, or changing message/service names in public APIs
Runbuf lintbefore committing Protocol Buffer files
Runbuf breakingto check for breaking changes in Protocol Buffer files
Always use explicit field numbers in Protocol Buffer definitions (never rely on auto-assignment)
Reserve removed Protocol Buffer field numbers using the syntax:reserved 2, 15, 9 to 11;
Reserve removed Protocol Buffer field names using the syntax:reserved "foo", "bar";
Useoptionalfor all Protocol Buffer fields unless required by business logic
Usesnake_casefor Protocol Buffer field names (e.g.,user_id,created_at)
UsePascalCasefor Protocol Buffer message and service names (e.g.,UserProfile,DataService)
UseUPPER_SNAKE_CASEfor Protocol Buffer enum values (e.g.,STATUS_ACTIVE,TYPE_PREMIUM)
Document all messages and fields in Protocol Buffer files with comments
Group related messages in the same Protocol Buffer file
Use imports for shared types across Protocol Buffer files
Mark deprecated Protocol Buffer fields withdeprecated = trueand add a comment explaining the replacement
Files:
packages/protobuf/infrastructure_map.proto
**/ts-moose-lib/src/**/*.ts
⚙️ CodeRabbit configuration file
**/ts-moose-lib/src/**/*.ts: When reviewing changes to typescript moose lib:
- Check if any public apis were changed (class, method, type, config, etc).
- If yes, verify the documentation for THAT SPECIFIC feature is updated in apps/framework-docs-v2/content.
- If docs for that feature doesn't exist yet, it should be added. If the change removes public apis, the documentation for those should also be removed. Changing unrelated docs doesn't satisfy this requirement.
- In addition to reviewing for docs discrepancies, you should also review the code as per usual guidelines.
Files:
packages/ts-moose-lib/src/dmv2/dataModelMetadata.tspackages/ts-moose-lib/src/dmv2/registry.tspackages/ts-moose-lib/src/browserCompatible.tspackages/ts-moose-lib/src/dmv2/index.tspackages/ts-moose-lib/src/dmv2/internal.tspackages/ts-moose-lib/src/dmv2/sdk/cdcSource.ts
templates/**
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
For template modifications in the
templates/directory, verify they can still run in isolation
Files:
templates/typescript-tests/src/index.tstemplates/typescript-tests/src/ingest/cdc.tstemplates/python-tests/src/ingest/cdc.pytemplates/python-tests/src/main.py
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Follow PEP 8 style guide in Python; use snake_case for functions/variables, PascalCase for classes, UPPER_SNAKE_CASE for constants
Format Python code with Black using line-length 88; auto-formats on commit via Husky + lint-staged
Use type hints for function signatures and public APIs in Python
Files:
packages/py-moose-lib/moose_lib/dmv2/cdc_source.pypackages/py-moose-lib/moose_lib/internal.pypackages/py-moose-lib/moose_lib/dmv2/registry.pytemplates/python-tests/src/ingest/cdc.pypackages/py-moose-lib/moose_lib/dmv2/__init__.pytemplates/python-tests/src/main.pypackages/py-moose-lib/moose_lib/dmv2/_registry.py
**/py-moose-lib/moose_lib/**/*.py
⚙️ CodeRabbit configuration file
**/py-moose-lib/moose_lib/**/*.py: When reviewing changes to python moose lib:
- Check if any public apis were changed (class, method, type, config, etc)
- If yes, verify the documentation for THAT SPECIFIC feature is updated in apps/framework-docs-v2/content
- If docs for that feature doesn't exist yet, it should be added. If the change removes public apis, the documentation for those should also be removed. Changing unrelated docs doesn't satisfy this requirement.
- In addition to reviewing for docs discrepancies, you should also review the code as per usual guidelines.
Files:
packages/py-moose-lib/moose_lib/dmv2/cdc_source.pypackages/py-moose-lib/moose_lib/internal.pypackages/py-moose-lib/moose_lib/dmv2/registry.pypackages/py-moose-lib/moose_lib/dmv2/__init__.pypackages/py-moose-lib/moose_lib/dmv2/_registry.py
🧠 Learnings (18)
📚 Learning: 2026-01-19T22:36:56.117Z
Learnt from: DatGuyJonathan
Repo: 514-labs/moosestack PR: 3282
File: apps/framework-cli-e2e/test/utils/llm-agent-utils.ts:613-742
Timestamp: 2026-01-19T22:36:56.117Z
Learning: In apps/framework-cli-e2e/test/LLM-related tests (specifically llm-docs-automation.test.ts and llm-agent-utils.ts), token limit exceedances are intentional. The tests should fail when the agent exceeds Claude's context window to reflect real-user failure scenarios. Treat this as a test-quality signal: if the agent can fit within the context window, the documentation or task complexity may be insufficient and needs improvement. This guideline applies to all TypeScript test files under the framework-cli-e2e test directory that exercise LLM context handling.
Applied to files:
apps/framework-cli-e2e/test/utils/plan-utils.tsapps/framework-cli-e2e/test/lifecycle.test.tsapps/framework-cli-e2e/test/templates.test.ts
📚 Learning: 2026-01-28T14:50:00.978Z
Learnt from: cjus
Repo: 514-labs/moosestack PR: 3407
File: apps/framework-docs-v2/content/guides/static-report-generation.mdx:1402-1406
Timestamp: 2026-01-28T14:50:00.978Z
Learning: When reviewing WebApp implementations that run in a Node.js backend, prefer Node.js-specific types (e.g., Buffer) and avoid browser-focused cross-platform compatibility considerations. This guideline applies to MDX documentation and code examples across the repository where backend/runtime code is discussed; ensure reviewers distinguish server-side context from frontend/browser constraints and validate type usage accordingly.
Applied to files:
apps/framework-docs-v2/content/moosestack/deploying/configuring-moose-for-cloud.mdxapps/framework-docs-v2/content/moosestack/configuration.mdxapps/framework-docs-v2/content/moosestack/streaming/connect-cdc.mdx
📚 Learning: 2025-12-16T23:09:39.883Z
Learnt from: CR
Repo: 514-labs/moosestack PR: 0
File: packages/protobuf/AGENTS.md:0-0
Timestamp: 2025-12-16T23:09:39.883Z
Learning: Applies to packages/protobuf/**/*.proto : NEVER make breaking changes to existing `.proto` files, including removing or renaming fields, changing field numbers or types, or changing message/service names in public APIs
Applied to files:
packages/protobuf/infrastructure_map.proto
📚 Learning: 2025-12-16T23:09:39.883Z
Learnt from: CR
Repo: 514-labs/moosestack PR: 0
File: packages/protobuf/AGENTS.md:0-0
Timestamp: 2025-12-16T23:09:39.883Z
Learning: Version Protocol Buffer API changes carefully and test generated code in target languages before merging
Applied to files:
packages/protobuf/infrastructure_map.proto
📚 Learning: 2025-12-16T23:09:39.883Z
Learnt from: CR
Repo: 514-labs/moosestack PR: 0
File: packages/protobuf/AGENTS.md:0-0
Timestamp: 2025-12-16T23:09:39.883Z
Learning: Applies to packages/protobuf/**/*.proto : Mark deprecated Protocol Buffer fields with `deprecated = true` and add a comment explaining the replacement
Applied to files:
packages/protobuf/infrastructure_map.proto
📚 Learning: 2025-12-16T23:09:39.883Z
Learnt from: CR
Repo: 514-labs/moosestack PR: 0
File: packages/protobuf/AGENTS.md:0-0
Timestamp: 2025-12-16T23:09:39.883Z
Learning: Applies to packages/protobuf/**/*.proto : Document all messages and fields in Protocol Buffer files with comments
Applied to files:
packages/protobuf/infrastructure_map.proto
📚 Learning: 2025-12-16T23:09:10.917Z
Learnt from: CR
Repo: 514-labs/moosestack PR: 0
File: apps/framework-cli/AGENTS.md:0-0
Timestamp: 2025-12-16T23:09:10.917Z
Learning: Applies to apps/framework-cli/**/*.rs : Document all public APIs and breaking changes
Applied to files:
apps/framework-cli/src/framework/core/infrastructure/mod.rsapps/framework-cli/src/cli/display/infrastructure.rs
📚 Learning: 2026-02-01T23:07:12.348Z
Learnt from: CR
Repo: 514-labs/moosestack PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-02-01T23:07:12.348Z
Learning: Add E2E tests to `python-tests` and `typescript-tests` templates when changing user-facing features
Applied to files:
apps/framework-cli-e2e/test/lifecycle.test.tstemplates/typescript-tests/src/index.tsapps/framework-cli-e2e/test/templates.test.tstemplates/typescript-tests/.env
📚 Learning: 2026-01-15T19:41:53.563Z
Learnt from: CR
Repo: 514-labs/moosestack PR: 0
File: examples/cdp-analytics/CLAUDE.md:0-0
Timestamp: 2026-01-15T19:41:53.563Z
Learning: Applies to examples/cdp-analytics/app/ingest/models.ts : Data models in ingest/models.ts should define interfaces using the Model naming convention, and create corresponding IngestPipeline instances with table, stream, and ingestApi options enabled
Applied to files:
templates/typescript-tests/src/index.tspackages/ts-moose-lib/src/dmv2/index.tstemplates/typescript-tests/src/ingest/cdc.ts
📚 Learning: 2026-01-28T14:50:11.465Z
Learnt from: cjus
Repo: 514-labs/moosestack PR: 3407
File: apps/framework-docs-v2/content/guides/static-report-generation.mdx:1402-1406
Timestamp: 2026-01-28T14:50:11.465Z
Learning: Moose WebApp handlers are Node.js-only server-side code that runs in the backend runtime, never in browsers. When reviewing WebApp implementations, Node.js-specific types like `Buffer` are appropriate and don't need cross-platform browser compatibility considerations.
Applied to files:
packages/ts-moose-lib/src/dmv2/registry.tspackages/ts-moose-lib/src/dmv2/internal.ts
📚 Learning: 2026-01-27T17:36:29.854Z
Learnt from: CR
Repo: 514-labs/moosestack PR: 0
File: examples/nextjs-moose/CLAUDE.md:0-0
Timestamp: 2026-01-27T17:36:29.854Z
Learning: Applies to examples/nextjs-moose/app/actions.ts : Server actions must import query functions from the moose package and wrap them, serving as the bridge between frontend React components and backend moose query functions
Applied to files:
packages/ts-moose-lib/src/dmv2/registry.ts
📚 Learning: 2026-01-27T17:36:29.854Z
Learnt from: CR
Repo: 514-labs/moosestack PR: 0
File: examples/nextjs-moose/CLAUDE.md:0-0
Timestamp: 2026-01-27T17:36:29.854Z
Learning: Applies to examples/nextjs-moose/components/dashboard/**/*.ts : Dashboard hooks must call server actions (not moose functions directly) to decouple React components from ClickHouse implementation details
Applied to files:
packages/ts-moose-lib/src/dmv2/registry.ts
📚 Learning: 2026-01-15T19:41:53.563Z
Learnt from: CR
Repo: 514-labs/moosestack PR: 0
File: examples/cdp-analytics/CLAUDE.md:0-0
Timestamp: 2026-01-15T19:41:53.563Z
Learning: Applies to examples/cdp-analytics/app/views/*.ts : Use SummingMergeTree engine for materialized views that require incremental updates, and wrap aggregated metrics with SUM() when querying to properly merge partial rows
Applied to files:
packages/ts-moose-lib/src/browserCompatible.tstemplates/typescript-tests/src/ingest/cdc.ts
📚 Learning: 2026-01-27T17:36:29.854Z
Learnt from: CR
Repo: 514-labs/moosestack PR: 0
File: examples/nextjs-moose/CLAUDE.md:0-0
Timestamp: 2026-01-27T17:36:29.854Z
Learning: The moose query layer represents a two-step OLAP migration pattern: Step 1 (not shown) shifts just-in-time joins to write-time via Materialized Views; Step 2 (this demo) accelerates endpoint creation with the query layer. This architecture assumes MVs/OLAP tables already exist and focuses on the read-time query selection and filtering layer
Applied to files:
examples/cdp-analytics/packages/moosestack-service/app/workflows/cdc-placeholder.tsapps/framework-docs-v2/content/moosestack/streaming/connect-cdc.mdx
📚 Learning: 2026-01-27T17:36:29.854Z
Learnt from: CR
Repo: 514-labs/moosestack PR: 0
File: examples/nextjs-moose/CLAUDE.md:0-0
Timestamp: 2026-01-27T17:36:29.854Z
Learning: Server actions serve as the primary bridge between frontend React components and backend moose query functions. The communication flow should always be: React Component → Hook → Server Action → Moose Query Function → eventsModel.toSql() → ClickHouse
Applied to files:
examples/cdp-analytics/packages/moosestack-service/app/workflows/cdc-placeholder.ts
📚 Learning: 2026-01-27T17:36:29.854Z
Learnt from: CR
Repo: 514-labs/moosestack PR: 0
File: examples/nextjs-moose/CLAUDE.md:0-0
Timestamp: 2026-01-27T17:36:29.854Z
Learning: Applies to examples/nextjs-moose/moose/src/query-examples/**/*.ts : Query example functions must use `eventsModel.toSql()` to generate SQL, ensuring consistent query construction through the defined model
Applied to files:
examples/cdp-analytics/packages/moosestack-service/app/workflows/cdc-placeholder.ts
📚 Learning: 2026-01-20T15:49:26.025Z
Learnt from: 514Ben
Repo: 514-labs/moosestack PR: 3150
File: packages/py-moose-lib/moose_lib/dmv2/materialized_view.py:35-35
Timestamp: 2026-01-20T15:49:26.025Z
Learning: In the Python moose library (packages/py-moose-lib), prefer typing.Union[X, Y] syntax over PEP 604's X | Y for type annotations. Use Union when you want to explicitly convey a union in the type system, especially if readability or tooling friendliness (e.g., older type checkers) is a consideration. Reserve X | Y for simple, modern type hints where you want concise syntax and are sure the project’s type checker version supports PEP 604 without issues.
Applied to files:
packages/py-moose-lib/moose_lib/dmv2/cdc_source.pypackages/py-moose-lib/moose_lib/internal.pypackages/py-moose-lib/moose_lib/dmv2/registry.pypackages/py-moose-lib/moose_lib/dmv2/__init__.pypackages/py-moose-lib/moose_lib/dmv2/_registry.py
📚 Learning: 2026-01-26T00:56:27.011Z
Learnt from: DatGuyJonathan
Repo: 514-labs/moosestack PR: 3400
File: apps/framework-cli/src/framework/core/infrastructure_map.rs:1238-1298
Timestamp: 2026-01-26T00:56:27.011Z
Learning: Repo 514-labs/moosestack — Workflows: The CLI’s workflow diff is intended to detect only Temporal schedule–affecting changes. In apps/framework-cli/src/framework/core/infrastructure_map.rs, workflows_config_equal should compare schedule, retries, and timeout only; it must not include tasks. Task code/config changes are picked up automatically when the orchestration worker restarts and should not trigger a WorkflowChange.
Applied to files:
apps/framework-cli/src/framework/core/partial_infrastructure_map.rsapps/framework-cli/src/cli/display/infrastructure.rsapps/framework-cli/src/framework/core/infrastructure_map.rs
🧬 Code graph analysis (7)
apps/framework-cli/src/infrastructure/processes/mod.rs (2)
packages/py-moose-lib/moose_lib/dmv2/cdc_source.py (1)
CdcSource(60-85)packages/ts-moose-lib/src/dmv2/sdk/cdcSource.ts (1)
CdcSource(47-75)
apps/framework-cli-e2e/test/lifecycle.test.ts (1)
apps/framework-cli-e2e/test/constants.ts (1)
TIMEOUTS(7-51)
packages/ts-moose-lib/src/dmv2/registry.ts (2)
packages/ts-moose-lib/src/dmv2/sdk/cdcSource.ts (1)
CdcSource(47-75)packages/ts-moose-lib/src/dmv2/internal.ts (1)
getMooseInternal(1287-1288)
packages/py-moose-lib/moose_lib/dmv2/cdc_source.py (2)
packages/py-moose-lib/moose_lib/dmv2/types.py (1)
TypedMooseResource(128-144)packages/py-moose-lib/moose_lib/blocks.py (1)
ReplacingMergeTreeEngine(50-63)
templates/python-tests/src/ingest/cdc.py (1)
packages/py-moose-lib/moose_lib/dmv2/cdc_source.py (4)
CdcSource(60-85)CdcSourceConfig(42-46)CdcTable(88-153)CdcTableConfig(49-57)
apps/framework-cli/src/framework/core/partial_infrastructure_map.rs (3)
packages/py-moose-lib/moose_lib/dmv2/cdc_source.py (2)
CdcSource(60-85)CdcTable(88-153)packages/ts-moose-lib/src/dmv2/sdk/cdcSource.ts (2)
CdcSource(47-75)CdcTable(77-204)packages/py-moose-lib/moose_lib/dmv2/life_cycle.py (1)
LifeCycle(11-34)
packages/ts-moose-lib/src/dmv2/sdk/cdcSource.ts (3)
packages/ts-moose-lib/src/dmv2/utils/stackTrace.ts (1)
getSourceLocationFromStack(92-128)packages/ts-moose-lib/src/dmv2/internal.ts (1)
getMooseInternal(1287-1288)packages/ts-moose-lib/src/dmv2/typedBase.ts (2)
TypedBase(25-103)TypiaValidators(8-15)
🪛 Ruff (0.14.14)
packages/py-moose-lib/moose_lib/dmv2/cdc_source.py
[warning] 61-61: Missing return type annotation for special method __init__
Add return type annotation: None
(ANN204)
[warning] 84-84: Remove quotes from type annotation
Remove quotes
(UP037)
[warning] 93-93: Missing return type annotation for special method __init__
Add return type annotation: None
(ANN204)
[warning] 93-93: Missing type annotation for **kwargs
(ANN003)
[warning] 98-98: Avoid specifying long messages outside the exception class
(TRY003)
packages/py-moose-lib/moose_lib/dmv2/registry.py
[error] 141-141: Undefined name CdcSource
(F821)
[error] 146-146: Undefined name CdcSource
(F821)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
- GitHub Check: Test E2E TypeScript Default Template (Node 20, pnpm-9)
- GitHub Check: Test E2E TypeScript Tests Template (Node 20, npm)
- GitHub Check: Test E2E Python Tests Template (Python 3.13)
- GitHub Check: Test E2E Python Tests Template (Python 3.12)
- GitHub Check: Test E2E Backward Compatibility - TypeScript (Node 20)
- GitHub Check: Test E2E TypeScript Tests Template (Node 22, npm)
- GitHub Check: Test E2E Cluster Support - Python (Python 3.13)
- GitHub Check: Test E2E TypeScript Tests Template (Node 24, npm)
🔇 Additional comments (49)
templates/python-tests/.env (1)
1-1: LGTM!Test-only credentials for local CDC testing. Appropriately uses dummy values (
test_user:test_pass@localhost) suitable for committed test templates.templates/typescript-tests/.env (1)
1-1: LGTM!Mirrors Python template with identical test credentials. Consistent cross-language CDC test configuration.
apps/framework-cli-e2e/test/utils/plan-utils.ts (1)
10-10: LGTM!Optional field correctly added.
Record<string, any>aligns with existing loose typing for test utilities.templates/typescript-tests/src/index.ts (1)
9-9: Export follows established pattern. Properly grouped with other ingest module re-exports.apps/framework-docs-v2/content/moosestack/configuration.mdx (1)
117-138: Documentation accurately reflects runtime environment APIs.Both
mooseRuntimeEnvandmoose_runtime_envare correctly exported and the code snippets are valid. No changes needed.packages/ts-moose-lib/src/dmv2/dataModelMetadata.ts (1)
19-19: CdcTable with 3 arguments is correct. Constructor signature (name, source, config) matchestypesToArgsLengthentry. Default handling via JSON schemas + columns is appropriate for a source type. CdcTable documentation exists inapps/framework-docs-v2/content/moosestack/streaming/connect-cdc.mdxper ts-moose-lib guidelines.apps/framework-docs-v2/content/moosestack/deploying/configuring-moose-for-cloud.mdx (1)
123-145: LGTM!CDC documentation section is well-structured and consistent with other optional service sections. Code snippets are syntactically correct.
apps/framework-docs-v2/content/moosestack/streaming/connect-cdc.mdx (1)
10-164: LGTM!Comprehensive CDC documentation with clear TypeScript and Python examples. Event shape and destination table defaults are well-documented.
templates/typescript-tests/src/ingest/cdc.ts (1)
1-65: LGTM!Clean CDC template implementation. Optional chaining on
changes?.addTransformcorrectly handles the case where stream may not be configured. Naming conventions followed.packages/py-moose-lib/moose_lib/dmv2/cdc_source.py (2)
60-86: LGTM on CdcSource implementation.Registration pattern is consistent with other resource types. Metadata handling and source file capture follow established patterns.
80-82: 🧹 Nitpick | 🔵 TrivialUse
Dictfrom typing for consistency.
dict[str, CdcTable[Any]]should useDictfrom typing per package conventions.♻️ Proposed fix
+from typing import Optional, Generic, Literal, TypeVar, Union, Any, Dict ... - self.tables: dict[str, CdcTable[Any]] = {} + self.tables: Dict[str, "CdcTable[Any]"] = {}Based on learnings about typing conventions in this package.
Likely an incorrect or invalid review comment.
packages/py-moose-lib/moose_lib/dmv2/__init__.py (1)
52-60: LGTM!CDC public API properly exported. Symbols in
__all__match imports. Documentation exists atconnect-cdc.mdx.Also applies to: 128-129, 167-174, 222-223
packages/py-moose-lib/moose_lib/internal.py (1)
538-564: LGTM!
CdcTableJsonandCdcSourceJsonmodels follow established patterns.InfrastructureMapproperly extended withcdc_sourcesfield.Also applies to: 595-595
templates/python-tests/src/main.py (1)
9-9: LGTM.apps/framework-cli/src/cli/display/infrastructure.rs (1)
760-762: LGTM — CDC process changes now display consistently.apps/framework-cli/src/cli/routines/peek.rs (1)
421-421: LGTM.apps/framework-cli/src/framework/core/infra_reality_checker.rs (8)
978-978: LGTM!Test fixture updated to include required
cdc_sourcesfield.
1051-1051: LGTM!Consistent with other test fixture updates.
1130-1130: LGTM!Test fixture updated correctly.
1200-1200: LGTM!Test fixture updated correctly.
1272-1272: LGTM!Test fixture updated correctly.
1360-1360: LGTM!Test fixture updated correctly.
1565-1565: LGTM!Test fixture updated correctly.
1632-1632: LGTM!Test fixture updated correctly.
packages/protobuf/infrastructure_map.proto (1)
34-35: LGTM!Non-breaking addition with proper field number.
examples/cdp-analytics/packages/moosestack-service/app/workflows/cdc-placeholder.ts (1)
10-10: LGTM!Documentation and example updated to reflect new CDC-as-code APIs. URL constant consistent with header comment.
Also applies to: 24-54, 62-63
packages/ts-moose-lib/src/browserCompatible.ts (1)
27-33: LGTM!Browser-compatible re-exports correctly mirror main module exports.
Also applies to: 62-63
apps/framework-cli/src/framework/core/plan_validator.rs (2)
16-18: LGTM!Error variant follows thiserror conventions with descriptive message.
241-241: LGTM!Test helper correctly initializes
cdc_sourcesfield.packages/ts-moose-lib/src/dmv2/registry.ts (1)
19-19: No issues in this import change.packages/ts-moose-lib/src/dmv2/internal.ts (5)
44-150: CDC registry wiring looks consistent.
548-566: CDC JSON interfaces look consistent with existing schema shapes.
938-949: CDC sources container added to infra map assembly.
1244-1276: CDC source serialization into infra map looks solid.
1339-1339: CDC registry reset in dev mode is correct.apps/framework-cli/src/framework/core/partial_infrastructure_map.rs (3)
46-70: No issues in the import updates.
584-731: CDC sources are correctly threaded into the infra map.
1511-1521: CDC metadata path normalization is correct.apps/framework-cli/src/framework/core/infrastructure_map.rs (3)
1584-1667:⚠️ Potential issue | 🟡 MinorAdd CDC diff/lifecycle unit tests.
diff_cdc_sourcesnow feeds the public diff output; add tests for add/update/remove plus lifecycle skips to prevent regressions.As per coding guidelines: Write unit tests for all public functions.
⛔ Skipped due to learnings
Learnt from: CR Repo: 514-labs/moosestack PR: 0 File: .github/copilot-instructions.md:0-0 Timestamp: 2025-12-16T23:08:31.521Z Learning: Write tests for new functionality
592-594: CDC docs already exist and are current. Comprehensive documentation for CDC sources is present inapps/framework-docs-v2/content/moosestack/streaming/connect-cdc.mdx, including TypeScript and Python examples forCdcSourceandCdcTable. No documentation updates required for this field.
36-38: Incomplete review. This change requires two separate verifications per guidelines:
Cargo checks: Cannot verify
cargo clippy --all-targets -- -D warningsandcargo fmt --all -- --check --edition 2021in sandbox environment (no compiler available). This must be verified locally before commit.CLI documentation: Per guidelines for
**/framework-cli/src/**, user-facing CLI changes require documentation updates. Verify if the CDC infrastructure changes introduce new user-facing CLI commands, flags, or configuration options. If yes, confirm corresponding documentation exists inapps/framework-docs-v2/content/.packages/ts-moose-lib/src/dmv2/sdk/cdcSource.ts (7)
1-10: Confirm TypeScript linting was run.Please run the repo’s lint script for these TS changes and share the output.
As per coding guidelines, Run linting checks before submitting PRs for TypeScript/JavaScript code.
111-129: Good validation on primary keys and reserved column names.Clear, early checks prevent invalid CDC definitions.
131-202: Stream/table wiring and defaults look solid.The defaults for CDC stream creation and ReplacingMergeTree configuration are sensible and consistent.
233-257: Schema extension for CDC row metadata is clean and consistent.
259-351: CDC event column modeling looks consistent with schema.
353-407: CDC row metadata columns are well defined.
11-82: Documentation forCdcSource,CdcTable, andCdcEventalready exists inapps/framework-docs-v2/content/moosestack/streaming/connect-cdc.mdxwith comprehensive TypeScript and Python examples. No additional documentation work is required.apps/framework-cli/src/cli/routines/seed_data.rs (1)
691-691: LGTM!Correctly initializes the new
cdc_sourcesfield with an emptyHashMapto maintain compatibility with the updatedInfrastructureMapstruct.
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
| const plan = await runMoosePlanJson(testProjectDir); | ||
|
|
||
| const cdcSources = plan.target_infra_map.cdc_sources ?? {}; | ||
| expect(Object.keys(cdcSources).length).to.be.greaterThan(0); | ||
| expect(Object.keys(cdcSources)).to.include("orders_cdc"); | ||
|
|
||
| const hasCdcChange = plan.changes.processes_changes.some( | ||
| (change) => change.CdcSource !== undefined, | ||
| ); | ||
| expect(hasCdcChange).to.be.true; |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
cat -n apps/framework-cli-e2e/test/lifecycle.test.ts | sed -n '250,270p'Repository: 514-labs/moosestack
Length of output: 864
🏁 Script executed:
# Find the type definition for plan object
rg -A5 "runMoosePlanJson" apps/framework-cli-e2e/test/lifecycle.test.ts | head -20Repository: 514-labs/moosestack
Length of output: 808
🏁 Script executed:
# Check if plan.changes is documented or has type guards elsewhere in tests
rg "plan\.changes" apps/framework-cli-e2e/test/ -A2 -B2Repository: 514-labs/moosestack
Length of output: 3988
Guard plan.changes before accessing processes_changes.
Line 260 accesses plan.changes.processes_changes without optional chaining. Other utility functions in the codebase (hasTableAdded, hasTableRemoved, etc.) consistently guard with plan.changes?.olap_changes, and this same test uses plan.changes?.olap_changes?.length elsewhere. Use optional chaining to avoid TypeError if changes is undefined.
Suggested fix
- const hasCdcChange = plan.changes.processes_changes.some(
+ const processesChanges = plan.changes?.processes_changes ?? [];
+ expect(
+ processesChanges,
+ "expected plan.changes.processes_changes to exist",
+ ).to.not.be.empty;
+ const hasCdcChange = processesChanges.some(
(change) => change.CdcSource !== undefined,
);🤖 Prompt for AI Agents
In `@apps/framework-cli-e2e/test/lifecycle.test.ts` around lines 254 - 263, Guard
access to plan.changes.processes_changes to avoid a TypeError when changes is
undefined: update the hasCdcChange computation (the expression that currently
calls plan.changes.processes_changes.some(...)) to use optional chaining and a
fallback (e.g., plan.changes?.processes_changes?.some(...) ?? false) and keep
the existing expect(hasCdcChange).to.be.true assertion; this references
runMoosePlanJson and the plan.changes/processes_changes symbols to locate the
change.
| #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] | ||
| pub struct CdcTable { | ||
| pub name: String, | ||
| #[serde(rename = "sourceTable")] | ||
| pub source_table: String, | ||
| #[serde(rename = "primaryKey")] | ||
| pub primary_key: Vec<String>, | ||
| pub stream: Option<String>, | ||
| pub table: Option<String>, | ||
| pub snapshot: Option<String>, | ||
| pub version: Option<String>, | ||
| pub metadata: Option<Metadata>, | ||
| } | ||
|
|
||
| impl CdcTable { | ||
| pub fn to_proto(&self) -> ProtoCdcTable { | ||
| ProtoCdcTable { | ||
| name: self.name.clone(), | ||
| source_table: self.source_table.clone(), | ||
| primary_key: self.primary_key.clone(), | ||
| stream: self.stream.clone(), | ||
| table: self.table.clone(), | ||
| snapshot: self.snapshot.clone(), | ||
| version: self.version.clone(), | ||
| metadata: MessageField::from_option(self.metadata.as_ref().map(|m| { | ||
| crate::proto::infrastructure_map::Metadata { | ||
| description: m.description.clone().unwrap_or_default(), | ||
| source: MessageField::from_option(m.source.as_ref().map(|s| { | ||
| crate::proto::infrastructure_map::SourceLocation { | ||
| file: s.file.clone(), | ||
| special_fields: Default::default(), | ||
| } | ||
| })), | ||
| special_fields: Default::default(), | ||
| } | ||
| })), | ||
| special_fields: Default::default(), | ||
| } | ||
| } | ||
|
|
||
| pub fn from_proto(proto: ProtoCdcTable) -> Self { | ||
| CdcTable { | ||
| name: proto.name, | ||
| source_table: proto.source_table, | ||
| primary_key: proto.primary_key, | ||
| stream: proto.stream, | ||
| table: proto.table, | ||
| snapshot: proto.snapshot, | ||
| version: proto.version, | ||
| metadata: proto.metadata.into_option().map(|m| Metadata { | ||
| description: if m.description.is_empty() { | ||
| None | ||
| } else { | ||
| Some(m.description) | ||
| }, | ||
| source: m | ||
| .source | ||
| .into_option() | ||
| .map(|s| super::table::SourceLocation { file: s.file }), | ||
| }), | ||
| } | ||
| } | ||
| } | ||
|
|
||
| #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] | ||
| pub struct CdcSource { | ||
| pub name: String, | ||
| pub kind: String, | ||
| pub connection: String, | ||
| pub tables: Vec<CdcTable>, | ||
| pub metadata: Option<Metadata>, | ||
| pub life_cycle: LifeCycle, | ||
| } | ||
|
|
||
| impl CdcSource { | ||
| pub fn expanded_display(&self) -> String { | ||
| format!( | ||
| "CDC Source: {} ({}) - {} tables", | ||
| self.name, | ||
| self.kind, | ||
| self.tables.len() | ||
| ) | ||
| } | ||
|
|
||
| pub fn short_display(&self) -> String { | ||
| self.expanded_display() | ||
| } | ||
|
|
||
| pub fn to_proto(&self) -> ProtoCdcSource { | ||
| ProtoCdcSource { | ||
| name: self.name.clone(), | ||
| kind: self.kind.clone(), | ||
| connection: self.connection.clone(), | ||
| tables: self.tables.iter().map(|t| t.to_proto()).collect(), | ||
| metadata: MessageField::from_option(self.metadata.as_ref().map(|m| { | ||
| crate::proto::infrastructure_map::Metadata { | ||
| description: m.description.clone().unwrap_or_default(), | ||
| source: MessageField::from_option(m.source.as_ref().map(|s| { | ||
| crate::proto::infrastructure_map::SourceLocation { | ||
| file: s.file.clone(), | ||
| special_fields: Default::default(), | ||
| } | ||
| })), | ||
| special_fields: Default::default(), | ||
| } | ||
| })), | ||
| life_cycle: match self.life_cycle { | ||
| LifeCycle::FullyManaged => ProtoLifeCycle::FULLY_MANAGED.into(), | ||
| LifeCycle::DeletionProtected => ProtoLifeCycle::DELETION_PROTECTED.into(), | ||
| LifeCycle::ExternallyManaged => ProtoLifeCycle::EXTERNALLY_MANAGED.into(), | ||
| }, | ||
| special_fields: Default::default(), | ||
| } | ||
| } | ||
|
|
||
| pub fn from_proto(proto: ProtoCdcSource) -> Self { | ||
| CdcSource { | ||
| name: proto.name, | ||
| kind: proto.kind, | ||
| connection: proto.connection, | ||
| tables: proto.tables.into_iter().map(CdcTable::from_proto).collect(), | ||
| metadata: proto.metadata.into_option().map(|m| Metadata { | ||
| description: if m.description.is_empty() { | ||
| None | ||
| } else { | ||
| Some(m.description) | ||
| }, | ||
| source: m | ||
| .source | ||
| .into_option() | ||
| .map(|s| super::table::SourceLocation { file: s.file }), | ||
| }), | ||
| life_cycle: match proto.life_cycle.enum_value_or_default() { | ||
| ProtoLifeCycle::FULLY_MANAGED => LifeCycle::FullyManaged, | ||
| ProtoLifeCycle::DELETION_PROTECTED => LifeCycle::DeletionProtected, | ||
| ProtoLifeCycle::EXTERNALLY_MANAGED => LifeCycle::ExternallyManaged, | ||
| }, | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
wc -l apps/framework-cli/src/framework/core/infrastructure/cdc_source.rsRepository: 514-labs/moosestack
Length of output: 133
🏁 Script executed:
cat -n apps/framework-cli/src/framework/core/infrastructure/cdc_source.rs | tail -50Repository: 514-labs/moosestack
Length of output: 2656
🏁 Script executed:
cat -n apps/framework-cli/src/framework/core/infrastructure/cdc_source.rs | head -50Repository: 514-labs/moosestack
Length of output: 2311
Add rustdoc and unit tests for public CDC APIs.
Public structs CdcTable and CdcSource and all their public methods lack documentation. Add rustdoc comments and write unit tests for to_proto(), from_proto(), and display methods per coding guidelines.
🤖 Prompt for AI Agents
In `@apps/framework-cli/src/framework/core/infrastructure/cdc_source.rs` around
lines 9 - 148, Add rustdoc comments to the public structs and methods: document
CdcTable, CdcSource and their methods to_proto(), from_proto(),
expanded_display(), short_display() describing purpose, fields, and expected
behavior; then add unit tests that construct representative CdcTable and
CdcSource instances (including metadata and lifecycle variants), assert
round-trip fidelity via to_proto() -> from_proto(), and verify
expanded_display()/short_display() outputs and that metadata/source fields map
correctly between proto and struct. Ensure tests cover edge cases like empty
description/source None and each LifeCycle variant so conversions in
CdcTable::to_proto/from_proto and CdcSource::to_proto/from_proto are validated.
| use std::hash::Hash; | ||
|
|
||
| pub mod api_endpoint; | ||
| pub mod cdc_source; |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check public items and documentation in the cdc_source module
rg -n "pub (struct|enum|fn|trait)" apps/framework-cli/src/framework/core/infrastructure/cdc_source.rs
echo "---"
rg -n "//!|///" apps/framework-cli/src/framework/core/infrastructure/cdc_source.rsRepository: 514-labs/moosestack
Length of output: 429
Add rustdoc comments for public items in cdc_source module.
CdcTable and CdcSource structs plus their public methods (to_proto, from_proto, expanded_display, short_display) lack documentation.
🤖 Prompt for AI Agents
In `@apps/framework-cli/src/framework/core/infrastructure/mod.rs` at line 24, Add
Rustdoc comments to the public module cdc_source and its public types and
methods: document the CdcTable and CdcSource structs and each public method
(to_proto, from_proto, expanded_display, short_display). For each item provide a
brief one-line description of its purpose, document parameters and return values
for conversion/formatting methods (e.g., to_proto/from_proto), and include any
relevant usage notes or examples for expanded_display/short_display output
expectations; ensure comments use /// style above the items in the cdc_source
module so they appear in generated docs.
| #[derive(Debug, Clone, Serialize, Deserialize)] | ||
| #[serde(rename_all = "camelCase")] | ||
| pub struct PartialCdcTable { | ||
| pub name: String, | ||
| pub source_table: String, | ||
| pub primary_key: Vec<String>, | ||
| pub stream: Option<String>, | ||
| pub table: Option<String>, | ||
| pub snapshot: Option<String>, | ||
| pub version: Option<String>, | ||
| pub metadata: Option<Metadata>, | ||
| } | ||
|
|
||
| #[derive(Debug, Clone, Serialize, Deserialize)] | ||
| #[serde(rename_all = "camelCase")] | ||
| pub struct PartialCdcSource { | ||
| pub name: String, | ||
| pub kind: String, | ||
| pub connection: String, | ||
| pub tables: Vec<PartialCdcTable>, | ||
| pub metadata: Option<Metadata>, | ||
| #[serde(alias = "life_cycle")] | ||
| pub life_cycle: Option<LifeCycle>, | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
cat -n apps/framework-cli/src/framework/core/partial_infrastructure_map.rs | sed -n '470,500p'Repository: 514-labs/moosestack
Length of output: 1172
Add rustdoc for public CDC structs.
PartialCdcTable and PartialCdcSource are public but lack documentation. Coding guidelines require rustdoc for all public APIs.
🤖 Prompt for AI Agents
In `@apps/framework-cli/src/framework/core/partial_infrastructure_map.rs` around
lines 475 - 498, Add rustdoc comments for the public structs PartialCdcTable and
PartialCdcSource: for each struct add a brief /// description above the struct
explaining its purpose and how it is used in the partial infrastructure map, and
document each public field with /// comments describing the field meaning (e.g.,
name, source_table, primary_key, stream, table, snapshot, version, metadata for
PartialCdcTable and name, kind, connection, tables, metadata, life_cycle for
PartialCdcSource) so both structs and all their public fields have clear rustdoc
documentation.
| fn convert_cdc_sources(&self) -> Result<HashMap<String, CdcSource>, DmV2LoadingError> { | ||
| let mut sources = HashMap::new(); | ||
|
|
||
| for (name, source) in &self.cdc_sources { | ||
| let connection = resolve_runtime_env(&source.connection).map_err(|error| { | ||
| DmV2LoadingError::RuntimeEnvResolution { | ||
| table_name: name.clone(), | ||
| field: "connection".to_string(), | ||
| error: error.to_string(), | ||
| } | ||
| })?; |
There was a problem hiding this comment.
Clarify error context for CDC connection resolution.
RuntimeEnvResolution uses table_name; for CDC sources the message will say “table” even though it’s a source. Consider renaming the field or adding a CDC-specific variant to avoid confusing error output.
🤖 Prompt for AI Agents
In `@apps/framework-cli/src/framework/core/partial_infrastructure_map.rs` around
lines 1442 - 1452, The error uses DmV2LoadingError::RuntimeEnvResolution with a
table_name which will mislabel CDC sources; update convert_cdc_sources to
produce a CDC-specific error variant or use a neutral field name: change the
error construction inside convert_cdc_sources (where
resolve_runtime_env(&source.connection) is mapped_err) to either call a new
DmV2LoadingError variant like RuntimeEnvResolutionForCdc { source_name:
name.clone(), field: "connection".to_string(), error: error.to_string() } or
adjust the existing RuntimeEnvResolution to use a neutral identifier (e.g.,
resource_name) and pass name.clone() so the message correctly refers to the CDC
source; ensure the enum definition for DmV2LoadingError is updated to include
the new variant or neutral field and update any display/formatting logic to use
the new field name.
| export type CdcSourceConfig = { | ||
| kind: string; | ||
| connection: string; | ||
| metadata?: { description?: string }; | ||
| lifeCycle?: LifeCycle; | ||
| }; | ||
|
|
||
| export type CdcTableConfig<T> = { | ||
| sourceTable: string; | ||
| primaryKey: Array<keyof T & string>; | ||
| stream?: boolean | Omit<StreamConfig<CdcEvent<T>>, "destination">; | ||
| table?: boolean | OlapConfig<CdcRow<T>>; | ||
| snapshot?: "initial" | "never"; | ||
| version?: string; | ||
| metadata?: { description?: string }; | ||
| lifeCycle?: LifeCycle; | ||
| }; |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Prefer interfaces for object-shaped configs.
CdcSourceConfig and CdcTableConfig are object shapes and should be interface per TS guidelines.
As per coding guidelines, Prefer interfaces for objects, types for unions/intersections in TypeScript.
🔧 Proposed refactor
-export type CdcSourceConfig = {
+export interface CdcSourceConfig {
kind: string;
connection: string;
metadata?: { description?: string };
lifeCycle?: LifeCycle;
-};
+}
-export type CdcTableConfig<T> = {
+export interface CdcTableConfig<T> {
sourceTable: string;
primaryKey: Array<keyof T & string>;
stream?: boolean | Omit<StreamConfig<CdcEvent<T>>, "destination">;
table?: boolean | OlapConfig<CdcRow<T>>;
snapshot?: "initial" | "never";
version?: string;
metadata?: { description?: string };
lifeCycle?: LifeCycle;
-};
+}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| export type CdcSourceConfig = { | |
| kind: string; | |
| connection: string; | |
| metadata?: { description?: string }; | |
| lifeCycle?: LifeCycle; | |
| }; | |
| export type CdcTableConfig<T> = { | |
| sourceTable: string; | |
| primaryKey: Array<keyof T & string>; | |
| stream?: boolean | Omit<StreamConfig<CdcEvent<T>>, "destination">; | |
| table?: boolean | OlapConfig<CdcRow<T>>; | |
| snapshot?: "initial" | "never"; | |
| version?: string; | |
| metadata?: { description?: string }; | |
| lifeCycle?: LifeCycle; | |
| }; | |
| export interface CdcSourceConfig { | |
| kind: string; | |
| connection: string; | |
| metadata?: { description?: string }; | |
| lifeCycle?: LifeCycle; | |
| } | |
| export interface CdcTableConfig<T> { | |
| sourceTable: string; | |
| primaryKey: Array<keyof T & string>; | |
| stream?: boolean | Omit<StreamConfig<CdcEvent<T>>, "destination">; | |
| table?: boolean | OlapConfig<CdcRow<T>>; | |
| snapshot?: "initial" | "never"; | |
| version?: string; | |
| metadata?: { description?: string }; | |
| lifeCycle?: LifeCycle; | |
| } |
🤖 Prompt for AI Agents
In `@packages/ts-moose-lib/src/dmv2/sdk/cdcSource.ts` around lines 29 - 45,
Replace the object-shaped type aliases with interfaces: change the declarations
of CdcSourceConfig and CdcTableConfig to interface declarations preserving all
properties and their types (including optional metadata, lifeCycle, generic
parameter T and nested StreamConfig/OlapConfig usages); keep the exported names,
generics, and property signatures identical but use the interface keyword so
they follow the project guideline to prefer interfaces for object-shaped
configs.
| registerTable(table: CdcTable<any>) { | ||
| this.tables.set(table.name, table); | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
cat -n packages/ts-moose-lib/src/dmv2/sdk/cdcSource.ts | sed -n '70,80p'Repository: 514-labs/moosestack
Length of output: 392
🏁 Script executed:
# Check the full class context to understand if registerTable is public
cat -n packages/ts-moose-lib/src/dmv2/sdk/cdcSource.ts | sed -n '1,100p'Repository: 514-labs/moosestack
Length of output: 3502
🏁 Script executed:
# Search for related documentation about CdcSource
fd -t f "cdcSource\|CdcSource" apps/framework-docs-v2/content/ 2>/dev/null | head -20Repository: 514-labs/moosestack
Length of output: 45
Add explicit return type on public method.
The registerTable method is public (exported class, no access modifier) and lacks an explicit return type. Add : void to comply with the coding guideline requiring explicit return types on public APIs.
✅ Proposed fix
- registerTable(table: CdcTable<any>) {
+ registerTable(table: CdcTable<any>): void {
this.tables.set(table.name, table);
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| registerTable(table: CdcTable<any>) { | |
| this.tables.set(table.name, table); | |
| } | |
| registerTable(table: CdcTable<any>): void { | |
| this.tables.set(table.name, table); | |
| } |
🤖 Prompt for AI Agents
In `@packages/ts-moose-lib/src/dmv2/sdk/cdcSource.ts` around lines 72 - 74, The
public method registerTable(table: CdcTable<any>) lacks an explicit return type;
update its signature to include an explicit void return type (e.g.,
registerTable(...): void) so the exported API complies with the coding
guideline, leaving the body unchanged; reference the registerTable method and
the CdcTable type to locate the change in cdcSource.ts.
| const buildCdcEventSchema = ( | ||
| base: IJsonSchemaCollection.IV3_1, | ||
| ): IJsonSchemaCollection.IV3_1 => { | ||
| const baseSchema = base.schemas[0] as any; | ||
| const nullableBase = { | ||
| anyOf: [baseSchema, { type: "null" }], | ||
| } as any; | ||
|
|
||
| return { | ||
| ...base, | ||
| schemas: [ | ||
| { | ||
| type: "object", | ||
| properties: { | ||
| op: { type: "string" }, | ||
| before: nullableBase, | ||
| after: nullableBase, | ||
| ts: { type: "string", format: "date-time" }, | ||
| lsn: { type: "string" }, | ||
| source: { type: "string" }, | ||
| }, | ||
| required: ["op", "ts", "lsn", "source"], | ||
| } as any, | ||
| ], | ||
| } as IJsonSchemaCollection.IV3_1; | ||
| }; |
There was a problem hiding this comment.
Constrain op to the CDC operation enum.
The schema currently accepts any string for op, which weakens validation. Consider restricting to the known operations.
🧰 Proposed fix
- op: { type: "string" },
+ op: { type: "string", enum: ["insert", "update", "delete"] },📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const buildCdcEventSchema = ( | |
| base: IJsonSchemaCollection.IV3_1, | |
| ): IJsonSchemaCollection.IV3_1 => { | |
| const baseSchema = base.schemas[0] as any; | |
| const nullableBase = { | |
| anyOf: [baseSchema, { type: "null" }], | |
| } as any; | |
| return { | |
| ...base, | |
| schemas: [ | |
| { | |
| type: "object", | |
| properties: { | |
| op: { type: "string" }, | |
| before: nullableBase, | |
| after: nullableBase, | |
| ts: { type: "string", format: "date-time" }, | |
| lsn: { type: "string" }, | |
| source: { type: "string" }, | |
| }, | |
| required: ["op", "ts", "lsn", "source"], | |
| } as any, | |
| ], | |
| } as IJsonSchemaCollection.IV3_1; | |
| }; | |
| const buildCdcEventSchema = ( | |
| base: IJsonSchemaCollection.IV3_1, | |
| ): IJsonSchemaCollection.IV3_1 => { | |
| const baseSchema = base.schemas[0] as any; | |
| const nullableBase = { | |
| anyOf: [baseSchema, { type: "null" }], | |
| } as any; | |
| return { | |
| ...base, | |
| schemas: [ | |
| { | |
| type: "object", | |
| properties: { | |
| op: { type: "string", enum: ["insert", "update", "delete"] }, | |
| before: nullableBase, | |
| after: nullableBase, | |
| ts: { type: "string", format: "date-time" }, | |
| lsn: { type: "string" }, | |
| source: { type: "string" }, | |
| }, | |
| required: ["op", "ts", "lsn", "source"], | |
| } as any, | |
| ], | |
| } as IJsonSchemaCollection.IV3_1; | |
| }; |
🤖 Prompt for AI Agents
In `@packages/ts-moose-lib/src/dmv2/sdk/cdcSource.ts` around lines 206 - 231, The
op property in buildCdcEventSchema currently allows any string; tighten
validation by replacing op: { type: "string" } with an enum of the known CDC
operation values (e.g., for Debezium-style ops use { type: "string", enum:
["c","u","d","r"] } or whatever project-specific set is correct), updating the
schema returned by buildCdcEventSchema so op is constrained to that enum and
adjusting any callers/tests if they rely on broader values.
| orders_cdc_source = CdcSource( | ||
| "orders_cdc", | ||
| CdcSourceConfig( | ||
| kind="postgresql", | ||
| connection=moose_runtime_env.get("TEST_CDC_CONNECTION"), | ||
| metadata={"description": "CDC source for orders (template test)"}, | ||
| ), |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
find . -name "cdc.py" -type f | grep -E "templates|test"Repository: 514-labs/moosestack
Length of output: 105
🏁 Script executed:
# First, let's find the file
git ls-files | grep -E "templates/python-tests.*cdc.py"Repository: 514-labs/moosestack
Length of output: 103
🏁 Script executed:
# Check if the file exists and read it
if [ -f "templates/python-tests/src/ingest/cdc.py" ]; then
wc -l "templates/python-tests/src/ingest/cdc.py"
echo "---"
cat -n "templates/python-tests/src/ingest/cdc.py"
else
echo "File not found"
fiRepository: 514-labs/moosestack
Length of output: 2091
🏁 Script executed:
# Search for CdcSourceConfig and CdcSource definitions
rg "class CdcSourceConfig" -A 10Repository: 514-labs/moosestack
Length of output: 893
🏁 Script executed:
# Search for moose_runtime_env implementation
rg "moose_runtime_env" -B 2 -A 2 | head -50Repository: 514-labs/moosestack
Length of output: 3885
🏁 Script executed:
# Check moose_runtime_env implementation to understand .get() behavior
rg "def get" packages/py-moose-lib/moose_lib/secrets.py -A 10Repository: 514-labs/moosestack
Length of output: 905
🏁 Script executed:
# Also check the moose_runtime_env type definition
rg "moose_runtime_env.*=" packages/py-moose-lib/moose_lib/secrets.py -A 2 -B 2Repository: 514-labs/moosestack
Length of output: 199
🏁 Script executed:
# Check if CdcSourceConfig.connection validates on None
cat packages/py-moose-lib/moose_lib/dmv2/cdc_source.pyRepository: 514-labs/moosestack
Length of output: 5508
🏁 Script executed:
# Look for test examples or documentation about how CDC connections should work
rg "CdcSourceConfig" templates/ -A 3 -B 1Repository: 514-labs/moosestack
Length of output: 763
🏁 Script executed:
# Check the full MooseRuntimeEnv class implementation
cat packages/py-moose-lib/moose_lib/secrets.pyRepository: 514-labs/moosestack
Length of output: 4307
🏁 Script executed:
# Look for how this is used in other templates to see if there's a pattern
grep -r "moose_runtime_env.get" templates/ -B 2 -A 2 | head -80Repository: 514-labs/moosestack
Length of output: 2117
Add type hints to to_orders_ingest function.
to_orders_ingest (lines 57–68) lacks parameter and return type hints per PEP 8.
Note: moose_runtime_env.get("TEST_CDC_CONNECTION") returns a marker string during template loading and requires the env var to be set at runtime; the current pattern aligns with other templates in the directory (s3_tests.py, s3_queue_tests.py).
🤖 Prompt for AI Agents
In `@templates/python-tests/src/ingest/cdc.py` around lines 23 - 29, Add explicit
PEP 8–compliant type hints to the to_orders_ingest function signature: annotate
its parameters (e.g., the env/runtime input types used to construct
orders_cdc_source and any other args) and its return type (likely the
ingest/config object or CdcSource) so callers and linters can infer types;
update the to_orders_ingest declaration (function name: to_orders_ingest) to
include proper parameter types and a return annotation (referencing CdcSource or
the specific ingest/config class used) while keeping the existing construction
of orders_cdc_source and CdcSourceConfig unchanged.
| def to_orders_ingest(event): | ||
| row = event.after or event.before | ||
| if row is None: | ||
| return None | ||
| return OrdersIngestRow( | ||
| order_id=row.id, | ||
| customer_id=row.customer_id, | ||
| total_usd=row.total_cents / 100, | ||
| status=row.status, | ||
| updated_at=row.created_at, | ||
| op=event.op, | ||
| ) |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
find . -type f -name "cdc.py" | head -20Repository: 514-labs/moosestack
Length of output: 105
🏁 Script executed:
git ls-files | grep -E "templates.*cdc\.py|python.*cdc\.py" | head -20Repository: 514-labs/moosestack
Length of output: 103
🏁 Script executed:
fd "cdc\.py" --type fRepository: 514-labs/moosestack
Length of output: 103
🏁 Script executed:
cat -n templates/python-tests/src/ingest/cdc.pyRepository: 514-labs/moosestack
Length of output: 2043
🏁 Script executed:
# Verify that OrdersIngestRow is properly defined and imported within the file
cat -n templates/python-tests/src/ingest/cdc.py | grep -A 5 "class OrdersIngestRow"Repository: 514-labs/moosestack
Length of output: 239
🏁 Script executed:
# Check if there are any existing type hints in the file to establish the pattern
rg "def \w+\(" templates/python-tests/src/ingest/cdc.py -A 2Repository: 514-labs/moosestack
Length of output: 149
Add type hints to the CDC transform.
Python guidelines require type hints on function signatures; to_orders_ingest lacks them.
Proposed fix (type hints)
+from typing import Any, Optional
+
-def to_orders_ingest(event):
+def to_orders_ingest(event: Any) -> Optional[OrdersIngestRow]:📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def to_orders_ingest(event): | |
| row = event.after or event.before | |
| if row is None: | |
| return None | |
| return OrdersIngestRow( | |
| order_id=row.id, | |
| customer_id=row.customer_id, | |
| total_usd=row.total_cents / 100, | |
| status=row.status, | |
| updated_at=row.created_at, | |
| op=event.op, | |
| ) | |
| from typing import Any, Optional | |
| def to_orders_ingest(event: Any) -> Optional[OrdersIngestRow]: | |
| row = event.after or event.before | |
| if row is None: | |
| return None | |
| return OrdersIngestRow( | |
| order_id=row.id, | |
| customer_id=row.customer_id, | |
| total_usd=row.total_cents / 100, | |
| status=row.status, | |
| updated_at=row.created_at, | |
| op=event.op, | |
| ) |
🤖 Prompt for AI Agents
In `@templates/python-tests/src/ingest/cdc.py` around lines 57 - 68, Add type
hints to the to_orders_ingest function signature: annotate the parameter (event)
and the return type as Optional[OrdersIngestRow]; if a concrete CDC event type
exists in the codebase use that (e.g., DebeziumEvent) otherwise import
typing.Any and annotate event: Any. Also ensure typing.Optional (and Any if
used) is imported at the top of the module and keep the existing body unchanged
so the function becomes def to_orders_ingest(event: <EventType>) ->
Optional[OrdersIngestRow].
|
Found 32 test failures on Blacksmith runners: Failures
|
1 participant
Summary
Testing
Note
Medium Risk
Adds a new CDC resource type that flows through planning/diffing/serialization across Rust, TS, and Python; mistakes could affect plan correctness and secret handling (connection strings), though apply behavior is currently no-op.
Overview
Adds first-class CDC-as-code support end-to-end: TS and Python SDKs introduce
CdcSource/CdcTabletyped APIs (including CDC event/row shapes) and register them into the dmv2 infra JSON.Updates the Rust CLI/planner to ingest
cdc_sourcesintoInfrastructureMap, serialize via protobuf, diff CDC sources asProcessChange::CdcSourcewith lifecycle-aware add/update/remove behavior, validate required CDC fields, and mask CDC connection strings inmoose planJSON export.Expands templates, E2E tests, and docs to cover runtime-env based CDC connections and to assert CDC sources appear in
moose planoutput (with dummyTEST_CDC_CONNECTION).Written by Cursor Bugbot for commit dcce032. This will update automatically on new commits. Configure here.