snakemake-tibanna run not finding rule

[SooLee]

@johanneskoester Do you mind taking a quick look at this issue? You may find the problem more easily. 4dn-dcic/tibanna#256

[sengoku93]

Hi @johanneskoester ,

I have posted several comments in 4dn-dcic/tibanna#256. Please let me know if you have any suggestions. Thank you!

Best,

[rvalieris]

I just ran into the same problem, it looks like the snakemake target includes the S3 bucket as a prefix but the Snakefile does not have this prefix, so snakemake cant figure it out which rules produces this output.

example:
rule a: output: a.txt

ends up being submitted to tibanna as snakemake $S3_BUCKET/a.txt

but there's no rule that produces $S3_BUCKET/a.txt

[rvalieris]

I think I figured out.
there are 2 issues that can cause this rule not found error.

First, is that I think snakemake is assuming a bucket subdir is always present. if I try to use --default-remote-prefix=BUCKET it gives the error, but with --default-remote-prefix=BUCKET/dir it works.

Second, is that you can't mix rules that depend on the default prefix with rules that use explict S3.remote, that is you can either write:
rule a: output: "a.txt"; rule b: input: "a.txt" or
rule a: output: S3.remote("BUCKET/dir/a.txt"); rule b: input: S3.remote("BUCKET/dir/a.txt")
but you can't mix both styles, which is a bummer because rule all downloads the inputs to the local host if you are using the default prefix.
also I rather not use S3.remote at all to try to leave the pipeline as generic as possible.