The NBSSLWebClient.ino example works only on webhook.site and aftenposten.no, but not on most other HTTPS sites

[jontingvold]

When I run examples/NBSSLWebClient/NBSSLWebClient.ino, the simple example for connecting to an HTTPS/SSL web server, it only works on a few sites.

It works on:

• webhook.site
• aftenposten.no

It does not work on:

• caddyserver.com
• nginx.org
• letsencrypt.org
• google.com
• arduino.cc
• www.nytimes.com
• ieeexplore.ieee.org
• vg.no

If i enable debugging mode (change line 28 from NB nbAccess; to NB nbAccess(true)), I get the following error:

00:33:42.216 -> AT+USOSEC=0,1,0
00:33:42.216 -> OK
00:33:42.406 -> AT+USECPRF=0,0,1
00:33:42.406 -> OK
00:33:42.616 -> AT+USOCO=0,"letsencrypt.org",443
00:33:42.909 -> ERROR
00:33:43.016 -> +UUSORD: 0,0
00:33:43.016 -> +UUSORD: 0,0
00:33:43.016 -> +UUSOCL: 0
00:33:43.125 -> AT+USOCL=0
00:33:43.125 -> ERROR
00:33:43.125 -> connection failed

For webhook.site, which works fine, I get

00:34:17.097 -> AT+USOSEC=0,1,0
00:34:17.097 -> OK
00:34:17.282 -> AT+USECPRF=0,0,1
00:34:17.282 -> OK
00:34:17.504 -> AT+USOCO=0,"webhook.site",443
00:34:18.797 -> OK
00:34:18.797 -> connected

I tried to change NBClient.cpp line 128 from MODEM.send("AT+USECPRF=0,0,1"); to MODEM.send("AT+USECPRF=0,0,0"); like Issue 90 suggested, but this did not work.

I don't have any clue what the issue can be about. webhook.site uses Let's encrypt and Nginx, but the web site of Let's encrypt and Nginx (which I guess use their own newest technology) does not work. It might be that webhook.site uses an older Nginx version, but this is hard to verify. aftenposten.no uses Let's Encrypt and Varnish and has a u89-varnish-abo-01 header. But vg.no, another major Norwegian news site, owned by the same parent company, also uses Varnish (it has a u89-varnish-01 header), but this site does not work. I also tired to see if their was any commonality between the certificate issuer or the signature algorithm for the sites that worked, but I always find an other sites with the same factor that did not work.

[aentinger]

This is very likely an issue of a missing or outdated certificate. @Rocketct can you provide help re updating of the internally stored certificates?

[johnyesberg]

I have similar issues: my MKR NB1500 with NBSSLWebClient.ino will connect to webhook.site, but not to an Azure influxdata.com site.
When I check on a browser, it seems that both webhook and influxdata use the same Let's Encrypt CA certificate, and the same ISRG root certificate.
This makes me think my problem is not a missing or outdated certificate.

When I try connecting to influxdata, I end up with an error 11 (EWOULDBLOCK / EAGAIN - Current operation would block, try
again.)
When I try connecting to arduino.cc, I receive an error 9 (EBADF - Bad file descriptor (internal error)).
Neither seems to suggest (to me) any particular avenues for investigation.
I'd appreciate any thoughts.

11:02:17.151 -> OK
11:02:17.378 -> AT+USOSEC=0,1,0

11:02:17.378 -> OK
11:02:17.576 -> AT+USECPRF=0,0,1

11:02:17.576 -> OK
11:02:17.769 -> AT+USOCO=0,"eastus-1.azure.cloud2.influxdata.com",443

11:02:20.179 -> ERROR
11:02:20.275 -> 
11:02:20.275 -> +UUSORD: 0,0
11:02:20.405 -> AT+USOCL=0

11:02:20.405 -> ERROR
11:02:20.405 -> connection failed
11:02:20.405 -> 
11:02:20.405 -> +UUSOCL: 0
11:02:20.439 -> AT+USOER

11:02:20.439 -> +USOER: 11
11:02:20.439 -> 
11:02:20.439 -> OK

and

16:28:50.952 -> OK
16:28:51.179 -> AT+USOSEC=0,1,0

16:28:51.179 -> OK
16:28:51.373 -> AT+USECPRF=0,0,1

16:28:51.373 -> OK
16:28:51.569 -> AT+USOCO=0,"arduino.cc",443

16:28:51.764 -> ERROR
16:28:51.861 -> 
16:28:51.861 -> +UUSOCL: 0
16:28:51.995 -> AT+USOCL=0

16:28:51.995 -> ERROR
16:28:51.995 -> connection failed
16:28:52.026 -> AT+USOER

16:28:52.026 -> +USOER: 9
16:28:52.026 -> 
16:28:52.026 -> OK

[johnyesberg]

Further to the above, I used nmap --script ssl-enum-ciphers -p 443 arduino.cc to see which TLS versions and cipher suites are supported by the different servers. Cipher suite options listed in the ublox 2018 manual rev 09 (page 206 of 307):

o 0 (factory-programmed value): (0x0000) Automatic the cipher suite will be negotiated in
the handshake process
o 1: (0x002f) TLS_RSA_WITH_AES_128_CBC_SHA
o 2: (0x003C) TLS_RSA_WITH_AES_128_CBC_SHA256
o 3: (0x0035) TLS_RSA_WITH_AES_256_CBC_SHA
o 4: (0x003D) TLS_RSA_WITH_AES_256_CBC_SHA256
o 5: (0x000a) TLS_RSA_WITH_3DES_EDE_CBC_SHA
o 6: (0x008c) TLS_PSK_WITH_AES_128_CBC_SHA
o 7: (0x008d) TLS_PSK_WITH_AES_256_CBC_SHA
o 8: (0x008b) TLS_PSK_WITH_3DES_EDE_CBC_SHA
o 9: (0x0094) TLS_RSA_PSK_WITH_AES_128_CBC_SHA
o 10: (0x0095) TLS_RSA_PSK_WITH_AES_256_CBC_SHA
o 11: (0x0093) TLS_RSA_PSK_WITH_3DES_EDE_CBC_SHA
o 12: (0x00ae) TLS_PSK_WITH_AES_128_CBC_SHA256
o 13: (0x00af) TLS_PSK_WITH_AES_256_CBC_SHA384
o 14: (0x00b6) TLS_RSA_PSK_WITH_AES_128_CBC_SHA256
o 15: (0x00b7) TLS_RSA_PSK_WITH_AES_256_CBC_SHA384

Arduino.cc server does not support any of those:

Host is up (0.037s latency).
Other addresses for arduino.cc (not scanned): 18.155.216.45 18.155.216.15 18.155.216.86
rDNS record for 18.155.216.94: server-18-155-216-94.bne50.r.cloudfront.net

PORT    STATE SERVICE
443/tcp open  https
| ssl-enum-ciphers:
|   TLSv1.2:
|     ciphers:
|       TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (ecdh_x25519) - A
|     compressors:
|       NULL
|     cipher preference: server
|   TLSv1.3:
|     ciphers:
|       TLS_AKE_WITH_AES_128_GCM_SHA256 (ecdh_x25519) - A
|       TLS_AKE_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
|       TLS_AKE_WITH_CHACHA20_POLY1305_SHA256 (ecdh_x25519) - A
|     cipher preference: server
|_  least strength: A

webhook.site supports:

Nmap scan report for webhook.site (46.4.105.116)
Host is up (0.33s latency).
rDNS record for 46.4.105.116: app02.webhook.site

PORT    STATE SERVICE
443/tcp open  https
| ssl-enum-ciphers: 
|   TLSv1.0: 
|     ciphers: 
|       TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (ecdh_x25519) - A
|       TLS_DHE_RSA_WITH_AES_256_CBC_SHA (dh 2048) - A      
|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (ecdh_x25519) - A
|       TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 2048) - A      
|       TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
|     compressors:
|       NULL
|     cipher preference: server
|   TLSv1.1:
|     ciphers:
|       TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (ecdh_x25519) - A
|       TLS_DHE_RSA_WITH_AES_256_CBC_SHA (dh 2048) - A
|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (ecdh_x25519) - A
|       TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 2048) - A
|       TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
|     compressors:
|       NULL
|     cipher preference: server
|   TLSv1.2:
|     ciphers:
|       TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (ecdh_x25519) - A
|       TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (dh 2048) - A
|       TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (dh 2048) - A
|       TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (ecdh_x25519) - A
|       TLS_DHE_RSA_WITH_AES_256_CCM_8 (dh 2048) - A
|       TLS_DHE_RSA_WITH_AES_256_CCM (dh 2048) - A
|       TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 (dh 2048) - A
|       TLS_DHE_RSA_WITH_AES_256_CBC_SHA (dh 2048) - A
|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (ecdh_x25519) - A
|       TLS_DHE_RSA_WITH_AES_128_CCM_8 (dh 2048) - A
|       TLS_DHE_RSA_WITH_AES_128_CCM (dh 2048) - A
|       TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 (dh 2048) - A
|       TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 2048) - A
|       TLS_RSA_WITH_AES_256_GCM_SHA384 (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_GCM_SHA256 (rsa 2048) - A
|       TLS_RSA_WITH_AES_256_CCM_8 (rsa 2048) - A
|       TLS_RSA_WITH_AES_256_CCM (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_CCM_8 (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_CCM (rsa 2048) - A
|       TLS_RSA_WITH_AES_256_CBC_SHA256 (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_CBC_SHA256 (rsa 2048) - A
|       TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A
|       TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
|     compressors:
|       NULL
|     cipher preference: server
|   TLSv1.3:
|     ciphers:
|       TLS_AKE_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
|       TLS_AKE_WITH_CHACHA20_POLY1305_SHA256 (ecdh_x25519) - A
|       TLS_AKE_WITH_AES_128_GCM_SHA256 (ecdh_x25519) - A
|     cipher preference: server
|_  least strength: A

influxdata also doesn't support any of the suites that the ublox manual mentions.

Nmap scan report for eastus-1.azure.cloud2.influxdata.com (52.224.79.155)
Host is up (0.24s latency).

PORT    STATE SERVICE
443/tcp open  https
| ssl-enum-ciphers:
|   TLSv1.2:
|     ciphers:
|       TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
|       TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (ecdh_x25519) - A
|     compressors:
|       NULL
|     cipher preference: server
|   TLSv1.3:
|     ciphers:
|       TLS_AKE_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
|       TLS_AKE_WITH_CHACHA20_POLY1305_SHA256 (ecdh_x25519) - A
|       TLS_AKE_WITH_AES_128_GCM_SHA256 (ecdh_x25519) - A
|     cipher preference: server
|_  least strength: A

Nmap done: 1 IP address (1 host up) scanned in 15.29 seconds

Although I don't see why there should be different error messages, the lack of common cipher suites seems to be a reasonable explanation for the lack of success in connecting to arduino.cc and influxdata.

When I look at the latest version of the manual rev 27, I see there are many more suites available (pages 308-311 of 536) - enough to match the TLSv1.2 protocol (although none of the TLSv1.3 suites) for arduino.cc and influxdata. It looks like I will have to try updating the firmware.

[MayarNour]

Hello @johnyesberg I'm facing the same issue while trying to connect to azure iot hub, where you able to solve the issue?

[johnyesberg]

No, I gave up. I found that I could submit to a google form, which could store data in a spreadsheet.

…

On Thu, 30 May 2024 at 07:08, MayarNour ***@***.***> wrote: Hello @johnyesberg <https://github.com/johnyesberg> I'm facing the same issue while trying to connect to azure iot hub, where you able to solve the issue? — Reply to this email directly, view it on GitHub <#106 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADVCZWB5E22W7OULZ6DN53ZEY7WHAVCNFSM6AAAAAAR37B5Y2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZYGI3DONRTHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[baszn]

Well, another one bites the dust, this took me way too long to figure out, but thanks @jontingvold and @johnyesberg for providing all the information of the things you have already tried. I have looked endlessly at the Ublox manual, nmap and OpenSSL output and tried many different sites in order to figure out what the hell was going on here. Although I'm not 100% sure I found the root cause, a lot of sites that were mentioned here now seem to work or have an explanation why they don't work.

Add your root certificates

This one is kind of obvious, the root certificates that are in NBRootCert.cpp are incomplete and often outdated. Fortunately, the NBSSLClient makes it quite easy to pass your own root certificates.

Copy NBRootCert.cpp, change the name of the class and pass it to the NBSSLClient through its constructor parameters (like NBSSLClient client(NB_ROOT_CERTS_CUSTOM, NB_NUM_ROOT_CERTS_CUSTOM);) . For this you need to download the root certificate you want to import as a DER certificate and convert it to a hexadecimal representation (as can be seen in NBRootCert.cpp). The command xxd -i cert.der is of great help here.

This library adds the Root certificates using the AT+USECMNG command, saving it to the persistent modem storage. Once added, it stays there. You can view the saved root certificates by executing AT+USECMNG=3 .

Don´t use EC pubkeys in your intermediate certificates

This seems to be a bug in how the SARA-R410M verifies certificates. When you look for example at the certificate chain of Let's Encrypt (https://letsencrypt.org/certificates/), the chain X1->R10/R11 will work because the root and intermediate certificates use RSA for the pubkeys, but the chain X1->E5/E6 use an EC pubkey for the E5/E6 intermediate certificates so it fails.

Why does this verification fail? No clue at the moment. The modem seems to support EC crypto algorithms as can be seen in the ciphersuits it supports. Some sites like google.com, which uses an EC pubkey in their server certificate (but not in their root/intermediate certificates) seem to work correctly. It looks like it gets confused when a combination of EC and standard RSA pubkeys get used in the certification chain. It is hard to pinpoint exactly what's going on without having detailed debug logs.

So what is the solution? Preferably use an certificate chain without any EC pubkeys, but when this isn't possible you can also disable certificate checking using AT+USECPRF=0,0,0. Note that this is a big security risk as your essentially disabling some fundamental protections (like MitM) of SSL/TLS!

Set the SNI

Some sites don´t seem to work (correctly) when you don't specify the Server Name Indication (sites like nginx.org or pcgamer.com). Specifying the SNI can be done by using the AT command AT+USECPRF=0,10,"www.example.com".

I won't explain how SNI works exactly, there is an explanation in the manual.

Furthermore....

Well I can write a blog post of all the things I tried an did, but I'll leave it at this for now. It's extremely frustrating to have a modem only telling you 'internal error' or 'tls/ssl error' without specifying exactly what has been going wrong. It's also a shame that Arduino didn't update the root certificates in this library for 4 years and not taking the effort to investigate and fix these bugs making this device almost unusable when you need secure connections.

For people willing to dig further into this, here are some tips:

• This Ublox Sara modem has its own proprietary HTTP AT commands like AT+UHTTP which are very useful in executing HTTP request easily through the serial monitor. These also have their own set of error codes which can be requested with AT+HTTPER=0.
• SSL/TLS is mainly managed through the AT+USECPRF commands. It is worth reading through this part of the manual as it gives a lot of configuration options and insights in how SSL/TLS works on this modem.
• Use openssl s_client -connect example.com:443 -showcerts -tls1_2 to show information about the certificates and the certificate chain. If it mentions any id_ecPublicKey it can be problematic for this modem. TLSv1.3 is not supported on this modem, hence the -tls1_2 flag.