Improve pipeline tests for different protocols

[apeltzer]

Description of feature

Should be done in two steps:

• Finding proper testdata that is published somewhere
• Integrating these into testData
• Using the new config profile(s) as in Move protocols to config profile(s) #364 and then testing these properly

[lpantano]

this could be an option, can you take a look and see if we can use some of these samples:

https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE149513

and script confirming the trimming:

https://github.com/besarka16/Benchmarking-of-small-RNA-seq/blob/master/scripts/1_trimming.sh

[apeltzer]

Looks like we can use these to me - lets see how easy it would be to add these to nf-core/testdatasets and then run them using the profiles. Shouldn't be too hard and we can get some proper tests then running.

[lpantano]

that would be great!

[atrigila]

Hi @lpantano, we are having some issues in CI testing due to, what I think is, making several requests from mirBase. Is it possible to use any other source for the hairpin.fa and the mature.fa? Ideally, we would host the data in the nf-core test-dataset repository if the licence allows for it. For example, could we use RNAcentral? Is it possible to obtain equivalent datasets from this source?
Thank you!

cc @nschcolnicov

[lpantano]

as far as I know, tools need those file from miRBase, we could potentially put them nf-core test-dataset, I don't think the license is a problem. I don't see anything in their webpage that will limit that. I think the hairpin.fa is the only one in RNAcentral, but as well we will need the GFF file from miRBase and the mature.

[apeltzer]

If the licence is no problem, we can certainly dump this on S3 and use that URl.

[atrigila]

@nschcolnicov Nico wrote an E-mail to confirm license

[nschcolnicov]

Yes! I sent an email to '[email protected]' last week, but I'm not sure when I will get an answer, I'll update this ticket as soon as I have news