Skip to content

Consider supporting having the user choose between all-optional fields or not #45

@bartkl

Description

@bartkl

The balance between schema evolution and validation is a difficult one. Many experts say going all-optional is the way to go, others seem to still see reasons to use required fields.

I think to solve this once and for all we should simply allow the user to choose here:

  1. Prioritize validation
  • certain cardinality constraints from SHACL will be mapped onto required fields
  1. Prioritize evolution
  • everything should be optional

I will be writing more on these choices at some point, so be sure to keep an eye on the wiki.

Anyways, leaving this as a choice to the user, everyone should be happy :).

Part of the implementation is to have the CLI allow the user to actually make the choice. I'm leaning towards defaulting towards (2), since I feel that's a better attested to practice, and it's Lancaster's default behavior as well. If you know what you're doing and like to use (1), you can pass in an option like --prioritize-validation.

One thing to note in the case of (1): everyone, and I mean everyone everywhere, says at the very least that you should use default values for compatibility. I'm surprised why people would favor that over (2), since it has clearer semantics and better guarantee of flexibility in the sense you're looking for.

Metadata

Metadata

Assignees

Labels

avroAvro schemafeatureNew feature or changing an existing one

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions