[help] Possibility of setting resources in a target, based on tar_map value in static branch (or possibly on another target)

[koefoeden]

Help

• I understand and agree to https://books.ropensci.org/targets/help.html.

Description

Hi!

Context:
I like to run most of the targets in my pipeline in two different "versions" using tar_map static branching:

1. One with highly subsampled input, allowing for quick running and debugging
2. One with full input, generating the actual results of interest

Furthermore, I use the SLURM scheduler and set appropriate resource requirements for each target.

Problem:
However, the resource requirements for these targets are of course highly dependent on the input data size, i.e. which version, or in which static branch type, the target is in (case 1 vs 2). Therefore, it would be nice to change these resources dependent on the current active branch, to avoid requesting unnecesarrily large resources for these debugging targets. I tried to supply the tar_map value as an argument to the resources field of the targets, but it does not seem that this information is accessible here.

So in short, does anyone know of a way to acess the current values in a tar_map tibble from within the resources-field in the targets within the static branching?

[wlandau]

If you have only 2 sets of resources, you could split those targets into 2 different tar_map() calls. Or, to map over different resource sets, you could use tar_eval(): https://books.ropensci.org/targets/static.html#metaprogramming

[koefoeden]

Thanks for these suggestions! Using another tar_map is probably the easiest approach here - but I should get comfortable with the tar_eval function nonetheless! Again, thanks for such a nice package and your invaluable support!