could `configure-generators --spec spec.csv` cope with entire missing source tables ?

[andysouth]

configure-generators --spec spec.csv is intended to allow us to set the generator options for columns and then apply to multiple projects.

Currently some of our OMOP projects don't have all tables in the source data. This causes configure-generators to fail when using the specification csv. We can get around this by editing spec.csv on a project by project basis, but that rather defeats the object of it.

One option could be to get datafaker not to generate tables that don't have any data in the source.

Tim points out that this makes config.yaml less shareable to other projects. Ideally you want to have row_generators set even for your empty tables. But for those tables you want to be empty in your specific destination database you want num_rows_per_pass: 0 to make it empty. But you don't want num_rows_per_pass to be 0 for other databases that do want generation.

[tim-band]

What we need is

1. for --spec to be able to set (for example) weighted choice on empty tables, where weighted choice is actually not possible.
2. This should be permitted to run on tables with num_rows_per_pass set to 0.
3. We also need that num_rows_per_pass automatically gets set to 0 for empty tables.
4. We need that if num_rows_per_pass is not set to zero where there are generators that require data, a useful error message is output, or the user can specify that no table should be created in such cases.

The priorities for this, therefore, are 1 first: this might produce errors, but the errors are work-roundable. 2 already works. 4 would achieve the vision in a slightly nasty way, allowing users to produce output regardless or configuration problems like this. 3 would be a "nicer" way to do this.