Using targets from 2 different machines always causes outdated targets

[MarcellGranat]

Help

• I understand and agree to https://books.ropensci.org/targets/help.html.

Description

Hi!

I have currently set the store to a specific cloud folder to run the pipeline from two different machines. But the issue is that the targets are seen as built, but always outdated if I wish to continue that from the other machine. Any clue why is that? It would be a help if someone could explain how the package validates whether a target is outdated or not.

Thanks in advance

[noamross]

This may be related to #1184, though I'm not sure. Two things it is best to do when working across machines or people is making sure that bucket versioning is turned on, and setting `tar_option_set(repository_meta = "local").

[colebrookson]

Curious about this and want to make sure I understand this. So say you're working across two machines, a laptop and a server of some kind that are talking to each other via a git repo. You use the server to do some larger simulations, and all the targets are up to date. Then you pull the repo to your laptop (for example), and the targets run on the server will be out of date. Is the only option here to use an AWS bucket? Maybe I'm missing something?

[noamross]

To clarify - the cloud folder is a shared folder? Like a dropbox shared between the two machines? Is it the _targets/ folder, or the _targets/objects folder?

Second, are you committing and/or sharing the meta file? (_targets/meta/meta)

The standard options for this are (a) include the targets and the metafile in the git repo itself. This is inefficient storage-and-transfer-wise, especially of your target objects are large, but it does work. Git LFS (large file storage) is also an option here. (b) Have a shared folder outside the repository - on a shared drive or similar, that you use to share targets and manage via https://docs.ropensci.org/gittargets/ (I've not used this approach). (c) Use an S3-compatible bucket, on AWS or another service, which I find works even better if versioning is turned on. One possibility for this is to fire up a local S3-server, and you could transfer its storage folder between machines. I have a quick example of this here: https://github.com/noamross/targets-minio-versioning

[MarcellGranat]

Thank you for your answers.

Yes, the cloud folder is a shared folder. I set the store in the targets.yaml like:

main:
  store: ~/OneDrive - Neumann János Egyetem/exchange-rate

After days of experiment, I finally found the root of the bug. I used box and the module created by that seems to be new for the machines. Everything else works perfectly, but this. box::use imports the modules into the global environment, but if it is imported on the other computer it seems to be outdated.

[wlandau]

Glad you figured it out. box and targets have different systems for managing objects in environments, and unfortunately there are ways in which they are not compatible.