selfupdate behind MITM HTTPS Proxy

[marc0der]

@chriswhite199 commented on Thu Feb 09 2017

The selfupdate process uses a pipe to curl option, the endpoint of which (https://api.sdkman.io/1/selfupdate?beta=false) doesn't respect the config setting for sdkman_insecure_ssl, and if run behind a corp proxy that uses MITM ssl proxy 'magic', causes the selfupdate to fail.

# fetch new distribution and check integrity
download_url="${SDKMAN_SERVICE}/res?platform=${sdkman_platform}&purpose=selfupdate&version=${SDKMAN_VERSION}"
__sdkman_echo_debug "Download new scripts from: ${download_url}"
__sdkman_echo_debug "Download new scripts to: ${sdkman_tmp_zip}"
curl --location --progress-bar "$download_url" > "$sdkman_tmp_zip"

That curl should probably have some logic in it similar to the __sdkman_secure_curl function to add the --insecure curl flag if the sdkman_insecure_ssl env variable is true

Here's the output from running sdk selfupdate:

$ sdk selfupdate

Updating SDKMAN...
Download new scripts from: https://api.sdkman.io/1/res?platform=Linux&purpose=selfupdate&version=5.1.18+191
Download new scripts to: /home/cswhite/.sdkman/tmp/res-5.1.18+191.zip

curl: (60) server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.
[/home/cswhite/.sdkman/tmp/res-5.1.18+191.zip]
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of /home/cswhite/.sdkman/tmp/res-5.1.18+191.zip or
        /home/cswhite/.sdkman/tmp/res-5.1.18+191.zip.zip, and cannot find /home/cswhite/.sdkman/tmp/res-5.1.18+191.zip.ZIP, period.
Downloaded zip archive corrupt. Are you connected to the internet?

If problem persists, please ask for help on https://gitter.im/sdkman/user-issues

To work around this, for every upgrade i do i have to fudge in the following to the __sdk_selfupdate function to ensure the --insecure flag is added prior to the pipe to bash:

__sdkman_secure_curl "${SDKMAN_LEGACY_API}/selfupdate?beta=${sdkman_beta_channel}" | sed 's/--location/--location --insecure/g' | bash

---

@ColinHarrington commented on Mon Feb 27 2017

It looks like __sdkman_secure_curl function should be honoring the sdkman_insecure_ssl flag. You mention setting an environment variable. Do you set this externally? Can you double check your ~/.sdkman/etc/config file and post the contents if the issue persists.
http://sdkman.io/usage.html#config

Alternatively you could install mitm's ca cert manually which would probably be safer than just skipping all of the cert checks in the first place. The same mechanisms could be applied for a corporate CA too.

---

@chriswhite199 commented on Tue Feb 28 2017

Right, the secure_curl function does account for that, but as originally noted, the https://api.sdkman.io/1/selfupdate?beta=false endpoint uses a straight curl command without any respect to the insecure_ssl flag:

# fetch new distribution and check integrity
download_url="${SDKMAN_SERVICE}/res?platform=${sdkman_platform}&purpose=selfupdate&version=${SDKMAN_VERSION}"
__sdkman_echo_debug "Download new scripts from: ${download_url}"
__sdkman_echo_debug "Download new scripts to: ${sdkman_tmp_zip}"
curl --location --progress-bar "$download_url" > "$sdkman_tmp_zip"

That last line is just a curl, i'm assuming it should be __sdkman_secure_curl.

As you note, installing the MITM certs is probably a better solution, once you know that's the problem, and the current last portion of the error message doesn't hint to that being the issue (granted if you look further up it's more apparently obvious):

unzip:  cannot find zipfile directory in one of /home/cswhite/.sdkman/tmp/res-5.1.18+191.zip or
    /home/cswhite/.sdkman/tmp/res-5.1.18+191.zip.zip, and cannot find /home/cswhite/.sdkman/tmp/res-5.1.18+191.zip.ZIP, period.
Downloaded zip archive corrupt. Are you connected to the internet?

A middle ground suggestion for this ticket would be to see if you can use the cURL exit codes to detect SSL errors like this (exit code 60), and produce an error message that's more relevant to the actual issue.

---

@ColinHarrington commented on Tue Feb 28 2017

Oh I see, you want an parameter sent to api.sdkman.io that configures the self-update script to add the --insecure flag. Something like https://api.sdkman.io/1/selfupdate?beta=false&insecure=true. So that would be a change in this repo adding that extra parameter and a change to the hosted selfupdate script: https://github.com/sdkman/sdkman-candidates-legacy/blob/master/src/main/scripts/selfupdate.sh served via https://github.com/sdkman/sdkman-candidates-legacy/blob/master/src/main/groovy/server.groovy#L71-L80 (I think)

---

@marc0der commented on Thu Mar 02 2017

I think it would be a better option to change the selfupdate script to simply honour the configuration as is done elsewhere. The selfupdate script is hosted in the candidates legacy app and can simply use the existing curl wrappers.

I do agree with @ColinHarrington that you should rather install the appropriate ca certs, as the insecure option was only introduced for extreme cases and is generally frowned upon from a security perspective.

Moving this issue to the legacy candidates app.