Environment variables and the hashing process

[18kimn]

Hi,

Firstly thanks so much for such a great package, I'm having so much fun and learning so much using it. Everything about it from the central concepts to the progress messages and documentation are just so intuitive and convincing, I'm already a targets evangelist.

I'm just wondering if the package has a dedicated way to track changes across environment variables, and/or if the developer team has a preferred or canonical method of doing so using the existing functions. It is impossible and nearly always undesirable to ensure a completely frozen environment, but there are cases where we'd want to detect changes of specific environment variables. I see the two big scenarios being configurations with R and certain package workflows like with renv or targets itself, and secondly API keys and other secrets that may provide different levels of access or rate limits (my use case).

Users can do this if they really want by simply defining a target that is the environment variable, but the naive way to do so risks exposing secrets. Users or the package itself could rely on a one-way hash of the data so that targets can detect changes (and the data store shared) without storing what the target value itself is, but this doesn't really seem to fit the general package pattern. In any case, I'm just looking for other people's thoughts, so please let me know of any guidelines or suggestions ~

[wlandau]

Thanks for the feedback. In targets, automatically tracking environment variables is out of scope. I went through this a while ago trying to track changes to the R packages dependencies of a pipeline, and I learned there were a lot of disadvantages. Pipelines became brittle when an update to a package could invalidate all the targets, it was a lot of information for targets to track, and it became less clear to users where targets was looking. Locking down the packages with renv turned out to be simpler and more reliable. So in this case, I think a simpler and more reliable solution would be to containerize the data analysis project to control the environment variables.

[18kimn]

thanks so much!!