Processing Algorithms for import/export/validate

[signedav]

Current implementation

This PR introduces process algorithms for data import / export / validate with ili2pg and ili2gpkg:

They contain all the settings provided by the Model Baker Plugin:

Path through

The input parameters concerning the database connection are passed through as output parameters, means you can easily connect these algorithms in the graphical modeler:

Situation with the input parameter

While we have at the moment just a plain list of input parameters, this is not that comfy to use when starting the algorithms directly. As well it would make sense to have an option to read the connection parameters (like we do it in model baker) from a layer. But not only. It should be optional to enter manually, read from layer or use an existing db connection (defined in the data source manager).

An option would be to create two more algorithms:

• parsing connection params from layer
• parsing connection params from profile db connection

But even nicer would be an almighty input parameter:

(Source: https://excalidraw.com/#json=vzYp9UoP3hOo3p19Iz7-e,He6nXlycWuRxn3h42DXUeQ )

But:

• Because this input parameter would make sense to introduce from the beginning, it would make sense to have it in (this would go over budget)
• But otherwise we would need multiple algorithms - old versions / new versions etc. that would be a mess...

Desicion

We decided against having this almighty input parameter, because it's very inflexible in the modeler. We would loose all the flexibility in getting the connection parameters. This is why in the end another "meta-algorithm" is part of the solution. Parsing connection params from layer (like mentioned above):

With this, such models can be built:

processingexample.zip

Choosing an input layer, deciding according to provider what algorithm should be run and pass the parsed connection settings.

To Do

• Import PG
• Import GPKG
• Validate PG
• Validate GPKG
• Export PG
• Export GPKG
• Test PG with Baskets
• Test PG without Baskets
• Test GPKG with Baskets
• Test GPKG without Baskets
• Image
• shortDescription/shortHelpString(self) in ValidatingAlgorithms
• super users
• db params are concerned
• Concept for a better input parameter

[signedav]

@gacarrillor what do you think about the decision with the meta algorithms in the end (after you pointed to the fact, that it's not super nice). I'm at the moment quite happy with the solution, but still open for other opinions.

If it's fine for you, then your review is appreciated. Thanks 🎃

[gacarrillor]

Very cool!
I have only minor inline comments and these 3 questions:

• The ili2db version that is running appears in the alg. log, right? Would be handy to be able to read it.

• Same as above for the actual ili2db command (including parameters) that is being run. I think it'd also appear in the alg. log?

• If I understood well, parameters like strokeArcs need to be defined by the user via ModelBaker GUI before running the Processing algs? If so, do you think this is clear for users right now from the alg. help?

[gacarrillor]

@gacarrillor what do you think about the decision with the meta algorithms in the end (after you pointed to the fact, that it's not super nice). I'm at the moment quite happy with the solution, but still open for other opinions.

If it's fine for you, then your review is appreciated. Thanks 🎃

BTW, I'm fine with the decision. I was just adding my 2 cents to the discussion.

[signedav]

• The ili2db version that is running appears in the alg. log, right? Would be handy to be able to read it.
• Same as above for the actual ili2db command (including parameters) that is being run. I think it'd also appear in the alg. log?

Like this:

Sufficient?

• If I understood well, parameters like strokeArcs need to be defined by the user via ModelBaker GUI before running the Processing algs? If so, do you think this is clear for users right now from the alg. help?

No. Schema import is not yet an algorithm. Only data.

[gacarrillor]

Sufficient?

Sure, nice!

No. Schema import is not yet an algorithm. Only data.

It seems strokeArcs would act on data import (from the docs):

Segmentiert Kreisbogen beim Datenimport. Der Radius geht somit verloren. Die Kreisbogen werden so segmentiert, dass die Abweichung der erzeugten Geraden kleiner als die Koordinatengenauigkeit der Stützpunkte ist.

[signedav]

It seems strokeArcs would act on data import (from the docs)

Oh... You are right! Still it's not implemented in Model Baker. Not sure if it's needed. I opened a feature request opengisch/QgisModelBaker#1098

And I think if it's not in Model Baker we wait on the processing tools. But thanks for pointing it out.

[signedav]

Thanks @gacarrillor I considered your inputs

[signedav]

I merge this. I can reproduce the failing test locally with the latest QGIS image. I did not find the reason yet, but it has nothing to do with this PR.