consider renaming workdir to .work so it will be hidden

[kelly-sovacool]

pairs with #188

[slsevilla]

I don't know if this is a good thing - maybe for during the run but since we are having such an issue with space on CCBR I think leaving this exposed so that users can easily delete it post a run is probably a good idea?

[kelly-sovacool]

Yeah, I agree -- I think I'd rather users know about the work dir so they understand how the pipeline works.

@kopardev's idea was if we switch the config to use publish_dir_mode: link (hardlink), the space won't be counted twice for both work and results.

[slsevilla]

Yeah, if people aren't cleaning up the work dir, that's a prob. I THINK this is really the feature we want:

nextflow-io/nextflow#3818

[kelly-sovacool]

Yeah, if people aren't cleaning up the work dir, that's a prob. I THINK this is really the feature we want:

nextflow-io/nextflow#3818

Yes definitely, the lack of temporary outputs is a huge loss for nxf

[kelly-sovacool]

Another thing we should think about is whether we actually want to publish all process outputs (we are currently) or turn off publishing for certain intermediate rules so the outputs directory will be less cluttered, while still making sure users can find the files they may want/need

[slsevilla]

Option2 is my vote. Our Snakemake workflows never output everything. Standard was that if a user wanted something kept, they could request it and we'd add it back in. I've done this a few times already for CARLISLE - I think it's a an easy, space-conscious way to handle it.