Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

packrat::restore deletes sources then complains it can't find them #560

Open
torbjorn opened this issue May 24, 2019 · 6 comments
Open

packrat::restore deletes sources then complains it can't find them #560

torbjorn opened this issue May 24, 2019 · 6 comments

Comments

@torbjorn
Copy link

I'm experiencing odd behavior.

I using packrat to control the production R environment, but deploying to production and calling restore does strange things. At first go it removes sources of the package I want to restore. It then complains it can't find the sources.

I do the following:

  1. packrat::install_local("ga.data") # on development system
  2. rsync the packrat dir using --delete, over to the production environment's packrat dir
  3. start R in production environment's packrat dir
  4. packrat::restore() # on production system
  5. go back to repeat the rsync
  6. restore again, this time it works.

Odd. In my setup this is reproducing every time.

Here is the text output from a terminal as I go through the steps:

    ## After rsync of development system to the production system, sources
    ##   of a particular package, ga.data, looks like this:

    tl@gamap(/var/www/profileplotter/perl/R_Home)$ ll packrat/src/ga.data/
    total 6492
    -rw-rw-r--+ 1 tl tl 1070060 May  9 10:14 ga.data_1.3.7-10.tar.gz
    -rw-r--r--+ 1 tl tl 1070072 May  9 10:14 ga.data_1.3.7-24.tar.gz
    -rw-r--r--+ 1 tl tl 1070288 May  9 10:14 ga.data_1.3.7-30.tar.gz
    -rw-r--r--+ 1 tl tl 1131170 May  9 10:14 ga.data_1.3.8-3.tar.gz
    -rw-r--r--+ 1 tl tl 1131817 May  9 10:14 ga.data_1.3.9-3.tar.gz
    -rw-r--r--+ 1 tl tl 1131821 May 24 13:39 ga.data_1.3.9-5.tar.gz

    ## entering R to restore - it complains it can't find 1.3.9_5

    tl@gamap(/var/www/profileplotter/perl/R_Home)$ R -q
    Packrat mode on. Using library in directory:
    - "/var/www/profileplotter/perl/R_Home/packrat/lib"
    > packrat::restore()
    Replacing ga.data (upgrade 1.3.9-3 to 1.3.9-5) ... Error in installPkg(pkgRecord, project, repos, lib) : 
      Failed to install ga.data (1.3.9-5): sources missing at /var/www/profileplotter/perl/R_Home/packrat/src/ga.data/ga.data_1.3.9-5.tar.gz
    In addition: Warning message:
    In packrat::restore() :
      The most recent snapshot was generated using R version 3.5.3
    > packrat::restore()
    Installing ga.data (1.3.9-5) ... 
    Error in installPkg(pkgRecord, project, repos, lib) : 
      Failed to install ga.data (1.3.9-5): sources missing at /var/www/profileplotter/perl/R_Home/packrat/src/ga.data/ga.data_1.3.9-5.tar.gz
    In addition: Warning message:
    In packrat::restore() :
      The most recent snapshot was generated using R version 3.5.3
    > 
    Save workspace image? [y/n/c]:

    ## And indeed, now that particular source (and a bunch of other sources is gone)

    tl@gamap(/var/www/profileplotter/perl/R_Home)$ ll packrat/src/ga.data/
    total 1108
    -rw-rw-r--+ 1 tl tl 1129979 May 24 13:42 ga.data_1.3.9-1.tar.gz


    ## I repeat rsync the packrat dir from development system, now the sources are back again:

    tl@gamap(/var/www/profileplotter/perl/R_Home)$ ll packrat/src/ga.data/
    total 6492
    -rw-rw-r--+ 1 tl tl 1070060 May 24 13:42 ga.data_1.3.7-10.tar.gz
    -rw-r--r--+ 1 tl tl 1070072 May 24 13:42 ga.data_1.3.7-24.tar.gz
    -rw-r--r--+ 1 tl tl 1070288 May 24 13:42 ga.data_1.3.7-30.tar.gz
    -rw-r--r--+ 1 tl tl 1131170 May 24 13:42 ga.data_1.3.8-3.tar.gz
    -rw-r--r--+ 1 tl tl 1131817 May 24 13:42 ga.data_1.3.9-3.tar.gz
    -rw-r--r--+ 1 tl tl 1131821 May 24 13:42 ga.data_1.3.9-5.tar.gz

    ## And this time restore works:

    tl@gamap(/var/www/profileplotter/perl/R_Home)$ R -q
    Packrat mode on. Using library in directory:
    - "/var/www/profileplotter/perl/R_Home/packrat/lib"
    p> packrat::restore()
    Installing ga.data (1.3.9-5) ... 
            OK (built source)
    Warning message:
    In packrat::restore() :
      The most recent snapshot was generated using R version 3.5.3
    > 
    Save workspace image? [y/n/c]:

    tl@gamap(/var/www/profileplotter/perl/R_Home)$ 
@kevinushey
Copy link
Contributor

To the best of my knowledge, Packrat won't ever try to automatically remove any downloaded source packages from the src folder. Of course, I could be wrong.

Is there any chance you could distill this into a small, reproducible example? E.g. using a snippet of code that I could run on my own machine to reproduce the problem.

@torbjorn
Copy link
Author

I tried creating a minimal setup with only a locally installed package. That did not reproduce this behaviour.

I did notice however that after a couple of iterations, my locally installed package has multiple source tar balls on the dev system under packrat/src, while on the prod system, after restore, there is only the one current tar ball. Even if I rsync'ed them all over.

Could that be at play above? The mechanism which removes sources not used or needed?

@torbjorn
Copy link
Author

I had another stab at this one, this time running the ::restore with strace -e trace=unlink, below is what unfolded. As you can see, right under Replacing ga.data (upgrade 1.3.9-5 to 1.3.9-9) ... it goes on to delete all the sources, including the one it should upgrade to. Once I run rsync again it does it just fine. Is it picky on file mtimes? Could there be slight time sync differences between two systems that causes this?

    strace -e unlink R -e 'packrat::restore()'
    --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=28943, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
    --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=28944, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
    --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=28947, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
    unlink("/tmp/tmpfnBdNQq")               = 0

    R version 3.5.1 (2018-07-02) -- "Feather Spray"
    Copyright (C) 2018 The R Foundation for Statistical Computing
    Platform: x86_64-redhat-linux-gnu (64-bit)

    R is free software and comes with ABSOLUTELY NO WARRANTY.
    You are welcome to redistribute it under certain conditions.
    Type 'license()' or 'licence()' for distribution details.

      Natural language support but running in an English locale

    R is a collaborative project with many contributors.
    Type 'contributors()' for more information and
    'citation()' on how to cite R or R packages in publications.

    Type 'demo()' for some demos, 'help()' for on-line help, or
    'help.start()' for an HTML browser interface to help.
    Type 'q()' to quit R.

    --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=28951, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
    > packrat::restore()
    unlink("/tmp/RtmplM6VjB/file710e4deaa62b") = 0
    unlink("/tmp/RtmplM6VjB/file710e1807a70a") = 0
    unlink("/tmp/RtmplM6VjB/file710e434ebd67") = 0
    unlink("/tmp/RtmplM6VjB/file710e7bcf92b5") = 0
    unlink("/tmp/RtmplM6VjB/file710e4bdebb10") = 0
    unlink("/tmp/RtmplM6VjB/file710e40ff47a") = 0
    unlink("/tmp/RtmplM6VjB/file710e2a75150e") = 0
    unlink("/tmp/RtmplM6VjB/file710e74ce4d51") = 0
    unlink("/tmp/RtmplM6VjB/file710e1aee383a") = 0
    unlink("/tmp/RtmplM6VjB/file710e4a30f515") = 0
    unlink("/tmp/RtmplM6VjB/file710e5f217cdf") = 0
    unlink("/tmp/RtmplM6VjB/file710e27bc877c") = 0
    unlink("/tmp/RtmplM6VjB/file710e1b108ab5") = 0
    unlink("/tmp/RtmplM6VjB/file710e6aa25025") = 0
    unlink("/tmp/RtmplM6VjB/file710e566db79b") = 0
    unlink("/tmp/RtmplM6VjB/file710e31a5eb87") = 0
    unlink("/tmp/RtmplM6VjB/file710e779f773e") = 0
    unlink("/tmp/RtmplM6VjB/file710e348d3edb") = 0
    unlink("/tmp/RtmplM6VjB/file710e3468d6e") = 0
    unlink("/tmp/RtmplM6VjB/file710e504a6836") = 0
    unlink("/tmp/RtmplM6VjB/file710e7125aa80") = 0
    unlink("/tmp/RtmplM6VjB/file710e2990a8ec") = 0
    unlink("/tmp/RtmplM6VjB/file710e3eb30f20") = 0
    unlink("/tmp/RtmplM6VjB/file710e39cd47f4") = 0
    unlink("/tmp/RtmplM6VjB/file710e15359ca4") = 0
    unlink("/tmp/RtmplM6VjB/file710e4fbe396f") = 0
    unlink("/tmp/RtmplM6VjB/file710e7ab1ac9e") = 0
    unlink("/tmp/RtmplM6VjB/file710e33651309") = 0
    unlink("/tmp/RtmplM6VjB/file710e33357736") = 0
    unlink("/tmp/RtmplM6VjB/file710e2d85800c") = 0
    unlink("/tmp/RtmplM6VjB/file710e19dff764") = 0
    unlink("/tmp/RtmplM6VjB/file710e1201d61") = 0
    unlink("/tmp/RtmplM6VjB/file710e458d2716") = 0
    unlink("/tmp/RtmplM6VjB/file710e5d2eb4cb") = 0
    unlink("/tmp/RtmplM6VjB/file710e7cefb016") = 0
    unlink("/tmp/RtmplM6VjB/file710e116be227") = 0
    unlink("/tmp/RtmplM6VjB/file710e613ea945") = 0
    unlink("/tmp/RtmplM6VjB/file710e2764c525") = 0
    unlink("/tmp/RtmplM6VjB/file710e63a2f78") = 0
    unlink("/tmp/RtmplM6VjB/file710e7c2ce17f") = 0
    unlink("/tmp/RtmplM6VjB/file710e7195ba3a") = 0
    unlink("/tmp/RtmplM6VjB/file710e655bac57") = 0
    unlink("/tmp/RtmplM6VjB/file710e23e968fc") = 0
    unlink("/tmp/RtmplM6VjB/file710eca644f0") = 0
    unlink("/tmp/RtmplM6VjB/file710e4ffdfc7c") = 0
    unlink("/tmp/RtmplM6VjB/file710e7a572097") = 0
    unlink("/tmp/RtmplM6VjB/file710e3e4c3077") = 0
    unlink("/tmp/RtmplM6VjB/file710e479d73bb") = 0
    unlink("/tmp/RtmplM6VjB/file710e2ee45f72") = 0
    unlink("/tmp/RtmplM6VjB/file710e4192bde5") = 0
    unlink("/tmp/RtmplM6VjB/file710e17e7dbf1") = 0
    unlink("/tmp/RtmplM6VjB/file710e200a09f2") = 0
    unlink("/tmp/RtmplM6VjB/file710e6b2366d1") = 0
    unlink("/tmp/RtmplM6VjB/file710e569aeb11") = 0
    unlink("/tmp/RtmplM6VjB/file710e59d751e7") = 0
    unlink("/tmp/RtmplM6VjB/file710e590376") = 0
    unlink("/tmp/RtmplM6VjB/file710e26592480") = 0
    unlink("/tmp/RtmplM6VjB/file710e5488fe85") = 0
    unlink("/tmp/RtmplM6VjB/file710e33be167f") = 0
    unlink("/tmp/RtmplM6VjB/file710e598e9bb6") = 0
    unlink("/tmp/RtmplM6VjB/file710e20e7e92") = 0
    unlink("/tmp/RtmplM6VjB/file710e4d9e0de3") = 0
    unlink("/tmp/RtmplM6VjB/file710e5aaeb917") = 0
    unlink("/tmp/RtmplM6VjB/file710e479ba5a8") = 0
    unlink("/tmp/RtmplM6VjB/file710e2accc2ae") = 0
    unlink("/tmp/RtmplM6VjB/file710e579e692e") = 0
    unlink("/tmp/RtmplM6VjB/file710e590787cf") = 0
    unlink("/tmp/RtmplM6VjB/file710ec0b6bf3") = 0
    unlink("/tmp/RtmplM6VjB/file710e7f032e53") = 0
    unlink("/tmp/RtmplM6VjB/file710e5f41b747") = 0
    unlink("/tmp/RtmplM6VjB/file710e8384d73") = 0
    unlink("/tmp/RtmplM6VjB/file710e7098e88d") = 0
    unlink("/tmp/RtmplM6VjB/file710e449d639e") = 0
    unlink("/tmp/RtmplM6VjB/file710e2c21b66f") = 0
    unlink("/tmp/RtmplM6VjB/file710e7d3f2d7d") = 0
    unlink("/tmp/RtmplM6VjB/file710e149b601b") = 0
    unlink("/tmp/RtmplM6VjB/file710e2678d706") = 0
    unlink("/tmp/RtmplM6VjB/file710e3b8b5df5") = 0
    unlink("/tmp/RtmplM6VjB/file710e5c38d3d6") = 0
    unlink("/tmp/RtmplM6VjB/file710e555d3678") = 0
    unlink("/tmp/RtmplM6VjB/file710e7d1e1bda") = 0
    unlink("/tmp/RtmplM6VjB/file710e7420afc7") = 0
    unlink("/tmp/RtmplM6VjB/file710e7567406b") = 0
    unlink("/tmp/RtmplM6VjB/file710e684182ac") = 0
    unlink("/tmp/RtmplM6VjB/file710e4abb9ad9") = 0
    unlink("/tmp/RtmplM6VjB/file710e4f3e9252") = 0
    unlink("/tmp/RtmplM6VjB/file710e689a8622") = 0
    unlink("/tmp/RtmplM6VjB/file710e7114bf59") = 0
    unlink("/tmp/RtmplM6VjB/file710e23c790d7") = 0
    unlink("/tmp/RtmplM6VjB/file710e1c589ca1") = 0
    unlink("/tmp/RtmplM6VjB/file710e4aa35b10") = 0
    unlink("/tmp/RtmplM6VjB/file710e25d60f69") = 0
    unlink("/tmp/RtmplM6VjB/file710e69f6aa85") = 0
    unlink("/tmp/RtmplM6VjB/file710e25521427") = 0
    unlink("/tmp/RtmplM6VjB/file710e6d71b512") = 0
    unlink("/tmp/RtmplM6VjB/file710e14c36d33") = 0
    unlink("/tmp/RtmplM6VjB/file710e7cf07d55") = 0
    unlink("/tmp/RtmplM6VjB/file710e46793ce1") = 0
    unlink("/tmp/RtmplM6VjB/file710e20ced927") = 0
    unlink("/tmp/RtmplM6VjB/file710e7bf3aba8") = 0
    unlink("/tmp/RtmplM6VjB/file710e25baf429") = 0
    unlink("/tmp/RtmplM6VjB/file710e2907269a") = 0
    unlink("/tmp/RtmplM6VjB/file710e6c8c9436") = 0
    unlink("/tmp/RtmplM6VjB/file710e6a5857c7") = 0
    unlink("/tmp/RtmplM6VjB/file710e5528dd09") = 0
    unlink("/tmp/RtmplM6VjB/file710e69cbc1b3") = 0
    unlink("/tmp/RtmplM6VjB/file710e7ef3b7e2") = 0
    unlink("/tmp/RtmplM6VjB/file710e7ba1b40f") = 0
    unlink("/tmp/RtmplM6VjB/libloc_228_ccadf3d65a81c52b.rds") = -1 ENOENT (No such file or directory)
    Replacing ga.data (upgrade 1.3.9-5 to 1.3.9-9) ... unlink("/var/www/ga-biobank/perl/R_Home/packrat/src/ga.data/ga.data_1.3.9-5.tar.gz") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/src/ga.data/ga.data_1.3.7-30.tar.gz") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/src/ga.data/ga.data_1.3.7-20.tar.gz") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/src/ga.data/ga.data_1.3.9-9.tar.gz") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/src/ga.data/ga.data_1.3.7-24.tar.gz") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/biobank_db.json") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/kitlot_mapping.json") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_probe_levels.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_lx200_probe_diff.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_probe_categories.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_model_pca.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_kitlot_map.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_customer_bacteria_names.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/data/bacteria_limits_table_revision2.RData") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/data/bacteria_limits_table_revision3.RData") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_kitlots.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_genus_annotations.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_bacteria_names.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_probe_set.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_batch_correction.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_model_data.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat.R") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/help/ga.data.rdx") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/help/AnIndex") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/help/ga.data.rdb") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/help/aliases.rds") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/help/paths.rds") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/Meta/links.rds") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/Meta/hsearch.rds") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/Meta/package.rds") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/Meta/nsInfo.rds") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/Meta/Rd.rds") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/Meta/features.rds") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/bacteria_db.json") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/probe_categories.json") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/INDEX") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/html/R.css") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/html/00Index.html") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/DESCRIPTION") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/NAMESPACE") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/R/ga.data.rdx") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/R/sysdata.rdx") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/R/ga.data") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/R/ga.data.rdb") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/R/sysdata.rdb") = 0
    --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=28960, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
    Error in installPkg(pkgRecord, project, repos, lib) : 
      Failed to install ga.data (1.3.9-9): sources missing at /var/www/ga-biobank/perl/R_Home/packrat/src/ga.data/ga.data_1.3.9-9.tar.gz
    Calls: <Anonymous> -> restoreImpl -> playActions -> installPkg
    In addition: Warning message:
    In packrat::restore() :
      The most recent snapshot was generated using R version 3.5.3
    Execution halted
    --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=28971, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
    +++ exited with 1 +++

@kevinushey
Copy link
Contributor

Thanks -- I think I understand what's going on now.

When a package is upgraded / downgraded, Packrat attempts to remove the old version of the package, and then install the new one. Part of that implies calling removePkgs(), e.g.

removePkgs(project, pkgRecord$name, lib)

and that will also imply removing all cached sources for that package:

packrat/R/restore.R

Lines 597 to 601 in ffd8106

removePkgs <- function(project, pkgNames, lib.loc = libDir(project)) {
unlink(file.path(srcDir(project), pkgNames), recursive = TRUE)
remove.packages(pkgNames, lib.loc)
pkgNames
}

It looks like this was intentional:

#19

but in hind-sight it seems over-aggressive. I don't think we should remove the package sources during a restore; they should only be cleaned up manually (e.g. by the user or by a smart-enough clean() function)

@torbjorn
Copy link
Author

Ok, then.

So is my chosen route for deploying my R based app wrong then?

My production system typically doesn't have access to a checked out dev. directory of package sources that my development system has, and from which it can install_local, and I'd be hesitant to provide it.

The other R-package-bundle approach is definitely an over kill. We make incremental updates to select few packages, recreating and reinstalling the entire bundle would be an insane waste of time for a small update. If I understand that approach right.

But why does it work the other time around? All I have to do, annoying as it is, is to have the restore() process remove the sources, complain loudly that they can't be found after that, then rsync them back, and run restore again, and this time its super happy.

The intended pattern for how this should work may or may not be optimal (goes without saying I think it isn't), but it ought to be consistent.

@kevinushey
Copy link
Contributor

I don't think you're doing anything wrong. Packrat seems to have some unexpected / sub-optimal behavior here, and I think it should work without the extra orchestration that seems to be required in your case. I'll plan to fix this in Packrat directly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants