[help] tar_make() "completed successfully" but does not update outdated targets

[wvictor14]

Help

• I understand and agree to https://books.ropensci.org/targets/help.html.

Description

I am wondering why my targets remain outdated / do not get made. I run tar_make() then tar_outdated() and see that there are still outdated targets. But tar_make() did not return any errors and says pipeline completed successfully.

I wonder if it is at all related to my usage of tarchetypes::tar_files()

I'm on targets 1.4.1

> tar_make()
- The project is out-of-sync -- use `renv::status()` for details.
[conflicted] Will prefer dplyr::filter over any other package.
✔ skipped target excel_data_raw_Batch_2
✔ skipped target excel_data_raw_Batch_3
✔ skipped target report_template
▶ dispatched target internal_bismark_full_genome_files
Warning: program compiled against libxml 211 using older 209
● completed target internal_bismark_full_genome_files [1.792 seconds]
▶ dispatched target tidied_excel_files_files_Batch_2
● completed target tidied_excel_files_files_Batch_2 [0.157 seconds]
▶ dispatched target tidied_excel_files_files_Batch_3
● completed target tidied_excel_files_files_Batch_3 [0.046 seconds]
✔ skipped target report_Batch_2
✔ skipped target report_Batch_3
✔ skipped branch internal_bismark_full_genome_d798aff7
✔ skipped branch internal_bismark_full_genome_be1ecb9a
✔ skipped branch internal_bismark_full_genome_7489c930
✔ skipped branch internal_bismark_full_genome_1d0f3df8
✔ skipped branch internal_bismark_full_genome_8a816d9a
✔ skipped branch internal_bismark_full_genome_89c944f2
✔ skipped branch internal_bismark_full_genome_a7c7e7ad
✔ skipped branch internal_bismark_full_genome_edc9aecc
✔ skipped branch internal_bismark_full_genome_76b40218
✔ skipped branch internal_bismark_full_genome_f171f019
✔ skipped branch internal_bismark_full_genome_3a546b2c
✔ skipped branch internal_bismark_full_genome_1585fdbf
✔ skipped branch internal_bismark_full_genome_510d0bf3
✔ skipped branch internal_bismark_full_genome_4abf2095
✔ skipped branch internal_bismark_full_genome_6e5feca0
✔ skipped branch internal_bismark_full_genome_10987ddd
✔ skipped branch internal_bismark_full_genome_cad7c919
✔ skipped branch internal_bismark_full_genome_eee62d42
✔ skipped branch internal_bismark_full_genome_113c76dc
✔ skipped branch internal_bismark_full_genome_aaa4899f
✔ skipped branch internal_bismark_full_genome_aa5e0aeb
✔ skipped branch internal_bismark_full_genome_35acacab
✔ skipped branch internal_bismark_full_genome_9c190e40
✔ skipped branch internal_bismark_full_genome_9307fb8e
✔ skipped branch internal_bismark_full_genome_36ee47e0
✔ skipped branch internal_bismark_full_genome_fcf7c39f
✔ skipped branch internal_bismark_full_genome_0b925166
✔ skipped branch internal_bismark_full_genome_ffdb5823
✔ skipped branch internal_bismark_full_genome_5a7bb78f
✔ skipped branch internal_bismark_full_genome_c3a64b5b
✔ skipped branch internal_bismark_full_genome_d3a927f9
✔ skipped branch internal_bismark_full_genome_64305718
✔ skipped branch internal_bismark_full_genome_e12a3532
✔ skipped branch internal_bismark_full_genome_a750a50a
✔ skipped branch internal_bismark_full_genome_ef0fdcdb
✔ skipped branch internal_bismark_full_genome_a876e866
✔ skipped branch internal_bismark_full_genome_31fa80c2
✔ skipped branch internal_bismark_full_genome_deabd8ba
✔ skipped branch internal_bismark_full_genome_ce9a1483
✔ skipped branch internal_bismark_full_genome_6d610584
✔ skipped branch internal_bismark_full_genome_f6adf388
✔ skipped branch internal_bismark_full_genome_414f396f
✔ skipped branch internal_bismark_full_genome_7839f815
✔ skipped branch internal_bismark_full_genome_8adfb6a4
✔ skipped branch internal_bismark_full_genome_37c6d439
✔ skipped branch internal_bismark_full_genome_5bc2974f
✔ skipped branch internal_bismark_full_genome_08c3c996
✔ skipped branch internal_bismark_full_genome_abb9ef07
✔ skipped branch internal_bismark_full_genome_a9ce429a
✔ skipped branch internal_bismark_full_genome_f13e3398
✔ skipped branch internal_bismark_full_genome_c9df2840
✔ skipped branch internal_bismark_full_genome_c477f366
✔ skipped branch internal_bismark_full_genome_c1cb8d09
✔ skipped branch internal_bismark_full_genome_abc97223
✔ skipped branch internal_bismark_full_genome_7e124a31
✔ skipped branch internal_bismark_full_genome_fded22d2
✔ skipped branch internal_bismark_full_genome_3396a9c6
✔ skipped branch internal_bismark_full_genome_010a9cec
✔ skipped branch internal_bismark_full_genome_c7e65a04
✔ skipped branch internal_bismark_full_genome_d7223e92
✔ skipped pattern internal_bismark_full_genome
✔ skipped branch tidied_excel_files_Batch_2_1d3ed5a8
✔ skipped branch tidied_excel_files_Batch_2_9579f2ff
✔ skipped branch tidied_excel_files_Batch_2_06f17da2
✔ skipped branch tidied_excel_files_Batch_2_78b16183
✔ skipped pattern tidied_excel_files_Batch_2
✔ skipped branch tidied_excel_files_Batch_3_d398ae99
✔ skipped branch tidied_excel_files_Batch_3_26a9c988
✔ skipped branch tidied_excel_files_Batch_3_54116196
✔ skipped branch tidied_excel_files_Batch_3_fcd70138
✔ skipped pattern tidied_excel_files_Batch_3
▶ completed pipeline [2.47 seconds]
 
> tar_outdated()
- The project is out-of-sync -- use `renv::status()` for details.
[conflicted] Will prefer dplyr::filter over any other package.
[1] "internal_bismark_full_genome_files" "internal_bismark_full_genome"      
[3] "tidied_excel_files_files_Batch_2"       "tidied_excel_files_files_Batch_3"      
[5] "tidied_excel_files_Batch_2"             "tidied_excel_files_Batch_3"              

Here is my targets file, I renamed some filenames

library(targets)
library(tarchetypes)
tar_option_set(
  packages = packages,
  format = "qs",
  #   controller = crew::crew_controller_local(workers = 2)
)
#options(clustermq.scheduler = "multicore")
tar_source()

# define input files ----
params <- list(
  'Batch_2' =
    list(
      '.date' = '2023-12-22',
      'file' = here::here(
        'data-raw', 'Batch-2',
        'data.xlsx')
    ),
  'Batch_3' =
    list(
      '.date' = '2024-02-02',
      'file' = here::here(
        'data-raw', 'Batch-3',
        'data.xlsx')
    )
) |>
  dplyr::bind_rows(.id = 'name')

# define pipeline ----
mapped_pipeline <- tar_map(
  values = params,
  names = "name",

  # read and parse excel data ----
  tar_target(
    excel_data_raw,
    read_cpg_data(file)),

  # write out tidied data ----
  tar_files(
    tidied_excel_files,
    excel_data_raw |>
      imap_chr(
        ~write_multiple(
          out_dir = here::here('outs', name), .df = .x, file = .y
        )
      )
  ),

  # knit report ----
  tar_target(
    report, format = 'file',
    {
      out_file <- here::here(
        'reports', paste(.date, name, 'report.html', sep = ' '))
      rmarkdown::render(
        input = report_template,
        output_file = out_file,
        params = list(data = excel_data_raw, batch_date = .date, report_title = name)
      )
      return(out_file)
    }
  )
)


list(
  tar_target(
    report_template, format = 'file', here('report.Rmd'),
  ),
  mapped_pipeline,

  # bismark analysis
  tarchetypes::tar_files(
    internal_bismark_full_genome,
    here::here(
      'data-raw',
      'results_bismark_full_genome_skip_dedup',
      'methylation_calls',
      'methylation_coverage') |>
      list.files(full.names = TRUE) |>
      str_subset('.*\\.cov\\.gz$')
  )
)

sessionInfo:

> sessionInfo()
R version 4.3.2 (2023-10-31)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 22.04.3 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.10.0 
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.10.0

locale:
 [1] LC_CTYPE=C.UTF-8       LC_NUMERIC=C           LC_TIME=C.UTF-8       
 [4] LC_COLLATE=C.UTF-8     LC_MONETARY=C.UTF-8    LC_MESSAGES=C.UTF-8   
 [7] LC_PAPER=C.UTF-8       LC_NAME=C              LC_ADDRESS=C          
[10] LC_TELEPHONE=C         LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C   

time zone: America/Los_Angeles
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base     

other attached packages:
 [1] TSDR_0.0.0.9000    conflicted_1.2.0   DT_0.31            extrafont_0.19    
 [5] janitor_2.2.0      gt_0.10.1          forcats_1.0.0      ggpmisc_0.5.5     
 [9] ggpp_0.5.6         scales_1.3.0       patchwork_1.2.0    pals_1.8          
[13] paletteer_1.6.0    here_1.0.1         ggh4x_0.2.8        magrittr_2.0.3    
[17] readxl_1.4.3       ggplot2_3.4.4      purrr_1.0.2        stringr_1.5.1     
[21] glue_1.7.0         tidyr_1.3.1        dplyr_1.1.4        readr_2.1.5       
[25] tibble_3.2.1       tarchetypes_0.7.11 targets_1.4.1      devtools_2.4.5    
[29] usethis_2.2.2     

loaded via a namespace (and not attached):
 [1] polynom_1.4-1       remotes_2.4.2.1     testthat_3.2.1      rematch2_2.1.2     
 [5] rlang_1.1.3         snakecase_0.11.1    compiler_4.3.2      callr_3.7.3        
 [9] vctrs_0.6.5         maps_3.4.2          quantreg_5.97       profvis_0.3.8      
[13] crayon_1.5.2        pkgconfig_2.0.3     fastmap_1.1.1       backports_1.4.1    
[17] ellipsis_0.3.2      utf8_1.2.4          promises_1.2.1      rmarkdown_2.25     
[21] sessioninfo_1.2.2   tzdb_0.4.0          ps_1.7.6            bit_4.0.5          
[25] MatrixModels_0.5-3  xfun_0.41           cachem_1.0.8        jsonlite_1.8.8     
[29] later_1.3.2         parallel_4.3.2      R6_2.5.1            stringi_1.8.3      
[33] qs_0.25.7           extrafontdb_1.0     pkgload_1.3.4       brio_1.1.4         
[37] lubridate_1.9.3     cellranger_1.1.0    Rcpp_1.0.12         knitr_1.45         
[41] timechange_0.3.0    httpuv_1.6.13       Matrix_1.6-3        splines_4.3.2      
[45] igraph_1.6.0        tidyselect_1.2.0    rstudioapi_0.14     dichromat_2.0-0.1  
[49] yaml_2.3.8          stringfish_0.16.0   codetools_0.2-19    miniUI_0.1.1.1     
[53] processx_3.8.3      pkgbuild_1.4.3      lattice_0.22-5      shiny_1.8.0        
[57] withr_3.0.0         evaluate_0.23       desc_1.4.3          survival_3.5-7     
[61] RcppParallel_5.1.7  urlchecker_1.0.1    xml2_1.3.6          pillar_1.9.0       
[65] renv_1.0.3          generics_0.1.3      vroom_1.6.5         rprojroot_2.0.4    
[69] hms_1.1.3           munsell_0.5.0       RApiSerialize_0.1.2 xtable_1.8-4       
[73] base64url_1.4       mapproj_1.2.11      tools_4.3.2         data.table_1.14.10 
[77] SparseM_1.81        visNetwork_2.1.2    fs_1.6.3            grid_4.3.2         
[81] Rttf2pt1_1.3.12     colorspace_2.1-0    cli_3.6.2           fansi_1.0.6        
[85] gtable_0.3.4        digest_0.6.34       htmlwidgets_1.6.4   memoise_2.0.1      
[89] htmltools_0.5.7     lifecycle_1.0.4     mime_0.12           bit64_4.0.5        
[93] MASS_7.3-60  

[wlandau]

For your full scaled-up pipeline, tar_sitrep() can help explain why certain targets are not up to date.

Nothing jumps out as alarming here just by reading the code you shared. Unfortunately I do not have access to your functions and data, so it's hard to say what might be going on. It would help if you could post a simpler and more runnable reprex: https://books.ropensci.org/targets/help.html#write-a-reprex