Add new groups to data without invalidating downstream targets?

[jennysjaarda]

I was wondering what you would recommend if new groups are added to an upstream data set so that only the new groups are rerun. For example, borrowing from the example from tar_group_by, we have:

# _targets.R file:
library(targets)
library(tarchetypes)
produce_data <- function() {
  expand.grid(var1 = c("a", "b"), var2 = c("c", "d"), rep = c(1, 2, 3))
}
list(
  tar_group_by(data, produce_data(), var1, var2),
  tar_target(result, analyze_fn(data), pattern = map(data))


)

if later new groups are added to the dataset:

produce_data <- function() {
  expand.grid(var1 = c("a", "b", "c"), var2 = c("c", "d"), rep = c(1, 2, 3))
}

Now the data target is outdated (understandably) but every result branch is also rebuilt. I guess targets doesn't know the contents of the data, so what would be the best way around this case? Best to be mapping over var1 directly and have that as input into the function?

[wlandau]

768afad should work. Please give it a try.

At first I thought this was just an instance of #507, then I realized expand.grid() creates an out.attrs attributes that depends on the whole data frame but gets propagated to each group/slice. 768afad removes the attribute. Another workaround would have been to install d6bac3a and use tidyr::expand_grid() instead of base::expand.grid().

[wlandau]

I would also set stringsAsFactors = FALSE because the adding a new slice with a new factor level could change the factor levels of the other slices and invalidate everything.

[jennysjaarda]

I just tried to install and run again, but I got a strange error:

Error : 'switch_config' is not an exported object from 'namespace:targets'

[wlandau]

Would you post a reprex? Not sure where that might come from. Development targets handles tar_config_set() and _targets.yaml differently.

[wlandau]

switch_config() should not be invoked at all. Maybe just restart your R session?

[jennysjaarda]

Sorry yes I guess it was just something with my R session as I couldn't reproduce with reprex. I still haven't managed to solve the original question with the update though. Will try and get a reprex for that.

[wlandau]

The update changes how row groups are hashed, which automatically invalidates all the branches from an expand.grid() call. However, that invalidation due to the update only happens once per old project. So if expand.grid() was the only factor at play, then I think subsequent runs should skip the correct slices if previous tar_make() also used the update.