Skip to content

Ro crate for the metaGOflow output #27

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hariszaf merged 29 commits into from
Mar 25, 2023
Merged

Ro crate for the metaGOflow output #27

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
a6ec747
init branch for issue #18
hariszaf Mar 22, 2023
36e80fb
software versions file
hariszaf Mar 22, 2023
5c85a97
remove unecessary output files; init ro-crate script
hariszaf Mar 22, 2023
78333a1
remove unecessary output files; init ro-crate script
hariszaf Mar 22, 2023
7ed085d
Merge remote-tracking branch 'origin/develop' into ro-crate
hariszaf Mar 22, 2023
4f5e357
minor edits
hariszaf Mar 22, 2023
29f5ff3
Merge branch 'develop' into ro-crate
hariszaf Mar 22, 2023
08c62cc
fix summary qc
hariszaf Mar 22, 2023
664b825
Merge remote-tracking branch 'origin/develop' into ro-crate
hariszaf Mar 22, 2023
e88632d
fix qc_summary output filenames
hariszaf Mar 22, 2023
cfc54fd
fix qc_summary output filenames
hariszaf Mar 22, 2023
e4dc458
remove prov output folder
hariszaf Mar 22, 2023
121c570
all test cases
hariszaf Mar 22, 2023
5d96e20
logic to address #18
hariszaf Mar 24, 2023
a3d5ffd
build ro-crate as zip #18
hariszaf Mar 24, 2023
74ab4f2
ro-crate example #18
hariszaf Mar 24, 2023
2821479
edits to keep only for the record; remove file afterwards
hariszaf Mar 24, 2023
06fa1f2
drop script
hariszaf Mar 24, 2023
19e53db
edit wf output to remove chunks and merge ips parts to single file
hariszaf Mar 24, 2023
746c603
add embrc records #18
hariszaf Mar 24, 2023
e5fa080
return ips chunks
hariszaf Mar 24, 2023
7a9eb65
set threads properly to exploit fat node on zorbas hpc
hariszaf Mar 24, 2023
d45aaca
edit ENA description #18
hariszaf Mar 24, 2023
9a537aa
minor edits reg #18
hariszaf Mar 25, 2023
a3db2c9
ignore dev outputs
hariszaf Mar 25, 2023
e2ceb74
cwd var
hariszaf Mar 25, 2023
c327600
add dataset metadata #18
hariszaf Mar 25, 2023
4e7f671
remove output dir so we keep only the .zip file that includes the pro…
hariszaf Mar 25, 2023
d042146
update the example ro-crate-metadata #18
hariszaf Mar 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,8 +116,15 @@ venv.bak/

# Ignore real-world test samples
test_input/SRR*
test_input/DB*

# Ignore dev output
TEST_*/
*.output

STELIOS_TEST/
marine_sediment_dbh/

slurm_run.sh


Binary file removed RunTimeResearchObject-f0b553d37e4255a3291393948f3e308bd88ed301.zip
View file
Open in desktop
Binary file not shown.
12 changes: 6 additions & 6 deletions config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
# Steps to go for
qc_and_merge_step: true
taxonomic_inventory: false
cgc_step: false
reads_functional_annotation: false
taxonomic_inventory: true
cgc_step: true
reads_functional_annotation: true
assemble: false

# Global
threads: 20
threads: 40
Copy link
Collaborator

@jprmachado jprmachado Mar 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am comfortable with changing config.yml paramters, just not sure if it is what is intended in this PR.

Copy link
Member Author

@hariszaf hariszaf Mar 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's zorbas-oriented.
Feel free to have a nice config in the documentation branch and we keep the one from there! 👍


# For parallelization cases might worth decrease global value of threads
interproscan_threads: 20
# As a rule of thumb keep that as floor(threads/8) where threads the previous parameter
Copy link
Collaborator

@jprmachado jprmachado Mar 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good rule. Yet is not assure that will always split in 8 instances, at least afters reading their code on cwltool. This may require fine tuning on the usage, but /8 is a general good starting point.

interproscan_threads: 4

# fastp parameters
detect_adapter_for_pe: false
Expand Down
212 changes: 0 additions & 212 deletions learning_cwl.md
View file
Open in desktop

This file was deleted.

Loading