Add end-to-end tests with a LSF container?

[leoisl]

We have lots of small unit tests where we mock LSF behaviour to ensure our code behaves as we expect. This is really nice as it is very easy to add new features or change the behaviour of existing features without having to test in a real LSF system, and check if it works. With mocking, testing new code is very easy.

However, we can have issues if we incorrectly mock LSF behaviour. Also, having some end-to-end tests in a real LSF system (e.g. in a container) could also be valuable. The only other snakemake profile that has tests is slurm, and their tests are these end-to-end tests in a slurm container: https://github.com/Snakemake-Profiles/slurm/tree/master/tests

End-to-end tests in a container with proper setup and proper tests do take time and effort (way more complicated than using mock). I am unsure if we should add these tests, and what is the priority.

[mbhall88]

Yeah, this was my original intention. However, I couldn't seem to get an LSF container working.

If you can get one working I will worship you as a sysadmin god.

[leoisl]

Oh, I did not know it was hard to do :(
So this seems to be desirable, but it is a hard issue, will set a low priority tag in this, but keep the issue open. You can change the issue status if you disagree!

PS: I can't add labels and tag issues

[mbhall88]

PS: I can't add labels and tag issues

Sorry, I have asked @johanneskoester (#12) to add you as a contributor. I guess it probably got lost in the 100000 notifications he must get a day on GitHub 😅