sync script breaks hab builds

[qubitrenegade]

Hello,

It appears following the instructions to sync packages causes subsequent attempts to build to fail.

Steps to reproduce

• Stand up on-prem-builder
• Bootstrap cluster: ./scripts/on-prem-archive.sh populate-depot http://${APP_HOSTNAME_OR_IP}
• attempt to execute build, build should succeed with hab 0.65.0 (which is what is packaged in the LATEST.tar.gz bundle)
• sync packages with ./scripts/on-prem-archive.sh sync-packages http://${APP_HOSTNAME_OR_IP
• habitat is no longer able to find requisite packages for starting hab studio, hab sup, hab launcher, etc.

Results:

$ hab pkg install core/shellcheck --force --binlink
» Installing core/shellcheck
☁ Determining latest version of core/shellcheck in the 'stable' channel
↓ Downloading core/shellcheck/0.6.0/20190205022536
☛ Verifying core/shellcheck/0.6.0/20190205022536
↓ Downloading core/acl/2.2.53/20190115012136
☛ Verifying core/acl/2.2.53/20190115012136
↓ Downloading core/attr/2.4.48/20190115012129
☛ Verifying core/attr/2.4.48/20190115012129
↓ Downloading core/binutils/2.31.1/20190115003743
☛ Verifying core/binutils/2.31.1/20190115003743
↓ Downloading core/coreutils/8.30/20190115012313
☛ Verifying core/coreutils/8.30/20190115012313
↓ Downloading core/gcc-libs/8.2.0/20190115011926
☛ Verifying core/gcc-libs/8.2.0/20190115011926
↓ Downloading core/glibc/2.27/20190115002733
☛ Verifying core/glibc/2.27/20190115002733
↓ Downloading core/gmp/6.1.2/20190115003943
☛ Verifying core/gmp/6.1.2/20190115003943
↓ Downloading core/grep/3.1/20190115012541
☛ Verifying core/grep/3.1/20190115012541
↓ Downloading core/libcap/2.25/20190115012150
☛ Verifying core/libcap/2.25/20190115012150
↓ Downloading core/libffi/3.2.1/20190115154244
☛ Verifying core/libffi/3.2.1/20190115154244
↓ Downloading core/libtool/2.4.6/20190115012815
☛ Verifying core/libtool/2.4.6/20190115012815
↓ Downloading core/linux-headers/4.17.12/20190115002705
☛ Verifying core/linux-headers/4.17.12/20190115002705
↓ Downloading core/pcre/8.42/20190115012526
☛ Verifying core/pcre/8.42/20190115012526
↓ Downloading core/sed/4.5/20190115012152
☛ Verifying core/sed/4.5/20190115012152
↓ Downloading core/zlib/1.2.11/20190115003728
↓ Downloading core/zlib/1.2.11/20190115003728
↓ Downloading core/zlib/1.2.11/20190115003728
↓ Downloading core/zlib/1.2.11/20190115003728
↓ Downloading core/zlib/1.2.11/20190115003728
✗✗✗
✗✗✗ We tried 5 times but could not download core/zlib/1.2.11/20190115003728. Giving up.
✗✗✗
ERROR: Job failed: exit code 1

$ hab sup run
∵ Missing package for core/hab-sup/0.74.0
» Installing core/hab-sup/0.74.0
☁ Determining latest version of core/hab-sup/0.74.0 in the 'stable' channel
↓ Downloading core/hab-sup/0.74.0/20190204225911
☛ Verifying core/hab-sup/0.74.0/20190204225911
↓ Downloading core/busybox-static/1.29.2/20190115014552
☛ Verifying core/busybox-static/1.29.2/20190115014552
→ Using core/bzip2/1.0.6/20190115011950
→ Using core/cacerts/2018.12.05/20190115014206
→ Using core/gcc-libs/8.2.0/20190115011926
→ Using core/glibc/2.27/20190115002733
↓ Downloading core/libarchive/3.3.2/20190116022211
↓ Downloading core/libarchive/3.3.2/20190116022211
↓ Downloading core/libarchive/3.3.2/20190116022211
↓ Downloading core/libarchive/3.3.2/20190116022211
↓ Downloading core/libarchive/3.3.2/20190116022211
✗✗✗ We tried 5 times but could not download core/libarchive/3.3.2/20190116022211. Giving up.

Is there an easy way to fix this?

I'm thinking I'm going to have to rm -rf /hab and start over and only run the packages provided in the bootstrap? (i.e.: no syncing latest packages, so we'll be stuck on hab 0.65 for a while?) I'm assuming/hoping the database is stored on /hab too?

Thanks

• Q

[bdangit]

@qubitrenegade, I've noticed that sync-packages does not update to 0.74.0. I even set it with base-plans only as well.

I had to manually download the 0.74.0 baseline and upload them into my on-prem.

[qubitrenegade]

@bdangit how did you do that? Maybe we can update the sync-packages script? My problem was identifying which packages are required

[bdangit]

Here is the list (it may not be everything):

hab
hab-backline
hab-studio
hab-sup
hab-launcher

From a setup that can access public bldr:

hab pkg install <pkg>

wish there was a pkg download

Locate all the harts in /hab/cache/artifacts

[qubitrenegade]

Ok, this seems to have gotten me back in business... it's a bit hamfisted (e.g.: I'm fairly certain there's a better way to s/r a string than piping it to perl...) but I think it at least alleviates the issue.

# if you've set auth_token in your cli.toml for your internal builder
# you will be unable to download from public supermarket, even with `--auth-token ''`
perl -pi -e 's/(auth_token.*)/#\1/' /etc/hab/cli.toml ~/.hab/etc/cli.toml

# let's remove everything in the cache dir just for GP
rm -vf "/hab/cache/artifacts/*.hart"

# combined with the first step _should_ ensure you're able to download the requisite packages
HAB_AUTH_TOKEN='' hab pkg install core/hab core/hab-backline core/hab-studio core/hab-sup core/hab-launcher --url http://bldr.habitat.sh

# Fix our cli.toml so we can upload to our internal bldr
perl -pi -e 's/#(auth_token)/\1/' /hab/etc/cli.toml ~/.hab/etc/cli.toml

# upload all the packages in /hab/cache/artifacts
for file in /hab/cache/artifacts/*.hart; do 

  # Strip the filename from the path
  filename=$(basename -- "$file"); 

  # convert our filenames to package names... there's gotta be a better way? 
  pkg_id=$(echo $filename | perl -pe 's/-x86_64-linux\.hart//; s{core-}{core/}; s{-(?=\d)}{/}g;'); 

  # upload and promote our packages to our internal bldr
  hab pkg upload "$file"; 
  hab pkg promote "$pkg_id" stable; 
done

[chefsalim]

Thanks for the report, we are looking into this..