Intermediate file compression and/or storage utilization forecasting

[joaobilro]

Description of feature

Hello, first of all congrats on a great work so far on NovelTree.
I have been using it to infer a species tree for fungi and I have come across this issue where the "work" folder gets bloated really quickly with the intermediate/temporary files from each of the steps in the pipeline, making it unfeasible to run unless you have no restrictions on storage space.

As an example, when I ran NovelTree with ~250 different isolates, the "work" folder had over 1 TB of files post-DIAMOND runs, and the OrthoFinder step failed on my end due to the lack of storage space. Considering that some people might not be able to easily extend their available storage space, it would be nice if there was some sort of file compression step added to the pipeline, and/or the implementation of storage utilization forecasting in order to know how much storage space is required to run the pipeline from start to finish.

Cheers,
João