generated from cloudposse/terraform-example-module
-
-
Notifications
You must be signed in to change notification settings - Fork 37
Open
Description
Describe the Feature
Current Behavior:
The delta_target block in AWS Glue crawler module does not support the create_native_delta_table argument, which limits compatibility with engines that can query Delta transaction logs directly.
dynamic "delta_target" {
for_each = var.delta_target != null ? var.delta_target : []
content {
connection_name = delta_target.value.connection_name
delta_tables = delta_target.value.delta_tables
write_manifest = delta_target.value.write_manifest
}
}Expected Behavior
Add support for the create_native_delta_table argument per AWS Glue Crawler spec:
create_native_delta_table – (Optional) Specifies whether the crawler will create native tables, to allow integration with query engines that support querying of the Delta transaction log directly.
create_native_delta_table = delta_target.value.create_native_delta_tableUse Case
By allowing the delta target to specify argument create_native_delta_table will allow integration with query engines that support querying of the Delta transaction log directly.
Describe Ideal Solution
Update the delta target dynamic block to include the missing argument
dynamic "delta_target" {
for_each = var.delta_target != null ? var.delta_target : []
content {
connection_name = delta_target.value.connection_name
delta_tables = delta_target.value.delta_tables
create_native_delta_table = delta_target.value.create_native_delta_table
write_manifest = delta_target.value.write_manifest
}
}Alternatives Considered
N/A
Additional Context
N/A
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels