HS100 Hardware V4.1 Firmware 1.1.0 - No Longer Working - No Port 9999 #81

ghostseven · 2020-11-12T10:50:04Z

I was not aware that these had an automatic firmware update but it is only what I can assume has happened. I have some HS100 devices that present at hardware version 4.1 (I have other that are 2.0 and 2.1 and all work fine).

Suddenly they now not longer work and NMAP shows nothing on port 9999 any more.

The units work fine in the Kassa app but that is it, I can only assume a firmware update has done this.

Result for UDP nmap scan

blake@nash:~$ sudo nmap -p0-65535 192.168.0.154 -sU
Starting Nmap 7.80 ( https://nmap.org ) at 2020-11-12 08:46 UTC
Nmap scan report for BM-TVConsole.ghost7.com (192.168.0.154)
Host is up (0.0058s latency).
Not shown: 65535 closed ports
PORT STATE SERVICE
20002/udp open|filtered commtact-http
MAC Address: CC:32:E5:A6:E5:64 (Unknown)

Nmap done: 1 IP address (1 host up) scanned in 35.37 seconds
blake@nash:~$

Result for TCP nmap scan

blake@nash:~$ sudo nmap -p0-65535 192.168.0.154 -sT
Starting Nmap 7.80 ( https://nmap.org ) at 2020-11-12 08:48 UTC
Nmap scan report for BM-TVConsole.ghost7.com (192.168.0.154)
Host is up (0.0044s latency).
Not shown: 65535 closed ports
PORT STATE SERVICE
80/tcp open http
MAC Address: CC:32:E5:A6:E5:64 (Unknown)

Nmap done: 1 IP address (1 host up) scanned in 30.08 seconds
blake@nash:~$

corautem · 2020-11-12T13:21:33Z

The same thing happened to me yesterday morning.

ghostseven · 2020-11-12T14:25:35Z

Ok so I have setup a hotspot on a Raspberry Pi and the only device on it is a HS100. This is a pcap from wireshark of everything on the wlan side. I did a turn on and a turn off, to me it looks like nothing is going over port 9999 (which makes sense as it is closed). Though that said it does not make a lot of sense to me as I see nothing coming in over port 80.

I am either doing something wrong (this seems likely) or I am failing to understand the capture properly (also likely).

PCAP in a zip attached.

HS100-H4.1-F1.1.0-On-Off.pcap.zip

Happy to do more tests or run anything that would help debug.

jneilliii · 2020-11-12T16:47:07Z

I have a couple of users reporting the same for my OctoPrint plugins (linked above). I'm using the same code to communicate with TPLinkSmartplug devices and appears firmware has finally broken.

synman · 2020-11-12T17:17:35Z

Thank you for sharing this. I scrambled quickly to implement some deny rules on my firewall to not allow my plugs to reach out to the internet as a result of this....

Looks like I caught it in time:

{
"system": {
"get_sysinfo": {
"sw_ver": "1.5.2 Build 180611 Rel.080914",
"hw_ver": "2.0",
"type": "IOT.SMARTPLUGSWITCH",
"model": "HS100(US)",
"mac": "zzz",
"dev_name": "Smart Wi-Fi Plug",
"alias": "TP-LINK_Smart Plug_F159",
"relay_state": 0,
"on_time": 0,
"active_mode": "none",
"feature": "TIM",
"updating": 0,
"icon_hash": "",
"rssi": -52,
"led_off": 0,
"longitude_i": 0,
"latitude_i": 0,
"hwId": "xxx",
"fwId": "00000000000000000000000000000000",
"deviceId": "xxx",
"oemId": "xxx",
"next_action": {
"type": -1
},
"err_code": 0
}
}
}

synman · 2020-11-12T17:27:56Z

I'm also going quite a ways back here... I do remember messing with some settings quite some time back but the details escape me. A history | grep on one of my PI's did show me trying to override things at some point. I can't remember where I found this, or even if it worked as it does not show up via the getinfo call:

tplink_smartplug.py -t xxx -j '{"cnCloud":{"set_server_url":{"server":"127.0.0.1"}}}'

You can see my intent here was to blackhole any update attempts. Perhaps that's why both of my plugs are still on such older versions firmware wise.

cp2004 · 2020-11-12T18:16:45Z

I have a couple of users reporting the same for my OctoPrint plugins

One of these users here 🙂
Doesn't work with OctoPrint, or this library on its own, or https://github.com/python-kasa/python-kasa either. If there is anything I can do to help, I will be happy to assist.

jneilliii · 2020-11-12T18:42:35Z

from what I'm seeing this might only be effecting UK models.

plasticrake · 2020-11-13T01:19:51Z

Related: home-assistant/core#43088

dragon2611 · 2020-11-13T15:30:03Z

@ghostseven You might need to put both the phone and the plug on the same network and block internet access to force it to communicate locally.

I may try and get a packet capture later but it will require me to reconfigure some network hardware so I can't do it at the moment.

ghostseven · 2020-11-13T16:17:22Z

@dragon2611 completely true, we are now seeing direct activity to port 80 on the device. See attached PCAP. It is a little noisy but just my iphone and the device on the network with internet access blocked.

HS100-H4.1-F1.1.0-On-Off-Local.pcap.zip

dragon2611 · 2020-11-13T16:19:47Z

I'll try and have a look when I'm on a computer, might be using some kind of multicast though

ghostseven · 2020-11-13T16:24:48Z

Ok there is some interesting UDP data packets on port 20002. This much better progress now its local only

k<§¿÷°u^õ2EÿÄjÀ¨7À¨7N"á·sÙ[Ã³b{"result":{"ip":"192.168.55.2","mac":"B0-95-75-5E-F5-32","device_id":"5889F7E50D5EBAC24CD4B1829EFAF927","owner":"98E11A6B2B3C68B89EB60704BE5A57C5","device_type":"IOT.SMARTPLUGSWITCH","device_model":"HS110(UK)","hw_ver":"4.1","factory_default":true,"mgt_encrypt_schm":{"is_support_https":false,"encrypt_type":"KLAP","http_port":80}},"error_code":0}

rytilahti · 2020-11-13T18:20:58Z

Some initial findings from a quick peek into that pcap file, the message format for discovery is

                          16B header
Broadcast: 020000010000000000000000463cb5d3 on 20002/udp.
                                     ^ checksum, maybe?
Response:  02000001015b000000000000c3b3625c
                    ^ 0x15b = 347 length of the payload excluding the header

The payload is the json encoded info as shown in @ghostseven's comment, which contains encrypt_type (no clue about what KLAP would be) and the port for the management controls. After the discovery, the communication commences using HTTP requests with binary payloads. The requests are HTTP POSTed on /app/request?seq=1307806992 (I first thought seq would be a unix timestamp, but then again this plug would be living in 2011. Seems to be just incremented integer.)

wilsonnkwan · 2020-11-14T15:26:56Z

Any ideas how to communicate with udp port?

jneilliii · 2020-11-14T20:06:46Z

based on @ghostseven's information it seems the the kasa device is acting like a tapo device on this firmware now. A similar approach seems to be happening on them as implemented here and discussed here.

dragon2611 · 2020-11-16T12:16:33Z

Some initial findings from a quick peek into that pcap file, the message format for discovery is
                          16B header
Broadcast: 020000010000000000000000463cb5d3 on 20002/udp.
                                     ^ checksum, maybe?
Response:  02000001015b000000000000c3b3625c
                    ^ 0x15b = 347 length of the payload excluding the header
The payload is the json encoded info as shown in @ghostseven's comment, which contains encrypt_type (no clue about what KLAP would be) and the port for the management controls. After the discovery, the communication commences using HTTP requests with binary payloads. The requests are HTTP POSTed on /app/request?seq=1307806992 (I first thought seq would be a unix timestamp, but then again this plug would be living in 2011. Seems to be just incremented integer.)

If you know Java then maybe decompiling the android app might yield some further clues?

snecklifter · 2020-11-16T13:07:15Z

@dragon2611 I have an 80Mb decompiled zip from the apk and in sources/com/tplinkra/tpcommon/tpclient/klap/TPKLAPClient.java I can see:

HttpClient httpClient = new HttpClient(requestId2, "http://" + this.iotContext.getDeviceContext().getIPAddress() + ":" + 80 + "/app/request?seq=" + sessionInfoByDeviceIdMD5.getSeq());

ghostseven · 2020-11-16T13:21:40Z

@dragon2611 I have an 80Mb decompiled zip from the apk and in sources/com/tplinkra/tpcommon/tpclient/klap/TPKLAPClient.java :

HttpClient httpClient = new HttpClient(requestId2, "http://" + this.iotContext.getDeviceContext().getIPAddress() + ":" + 80 + "/app/request?seq=" + sessionInfoByDeviceIdMD5.getSeq());

That looks interesting, I have grabbed the APK and if I get a chance tonight I will decompile and try and replicate some of the app behaviour.

I think we are moving forward so that is a good thing 👍

snecklifter · 2020-11-16T13:23:24Z

That looks interesting, I have grabbed the APK and if I get a chance tonight I will decompile and try and replicate some of the app behaviour.

http://www.javadecompilers.com/data/16.11.20/5178dd3c928e00253f0a53c20b058f30/Kasa_base_source_from_JADX.zip

ghostseven · 2020-11-16T13:28:45Z

That looks interesting, I have grabbed the APK and if I get a chance tonight I will decompile and try and replicate some of the app behaviour.

http://www.javadecompilers.com/data/16.11.20/5178dd3c928e00253f0a53c20b058f30/Kasa_base_source_from_JADX.zip

Thank you :)

chriswheeldon · 2020-11-16T16:45:38Z

@ghostseven @snecklifter I went through that process over the weekend and managed to get through the new two step handshake to start an encrypted session with my upgraded HS110. Here's a small proof of concept python script demonstrating discovery, handshake and sending one command to get the device info: https://gist.github.com/chriswheeldon/3b17d974db3817613c69191c0480fe55

I also successfully turned the plug on and off 🎉

In summary:

UDP broadcast with fixed payload on port 20002 to discover devices.
Requests are plaintext HTTP on TCP port 80 with encrypted contents
Two step handshake to determine encryption key and base IV to use for later requests.
Key and IV seem to be derived from the device email and password. Presumably this is for the account registered for TP-Link cloud functionality. I haven't registered and so the email and password are empty strings. See https://gist.github.com/chriswheeldon/3b17d974db3817613c69191c0480fe55#file-hs110-py-L76
AES-128-CBC for request and response body encryption.
HTTP requests include an incrementing sequence number as a query parameter. This is used to determine the per-request IV.
Encrypted request and response bodies are prefixed with a HMAC.
Pre-documented requests seem to work e.g. {"system":{"get_sysinfo":null}}
There's some time-sensitive state in the plug that results in repeat calls in quick succession to the script generating 403 responses. Not sure what is causing that (too many connections, cookies, &c). Note that each time the script runs it generates a new session cookie with the device.

cp2004 · 2020-11-16T16:55:36Z

@chriswheeldon Thanks for that, script confirmed working with my HS110 - Mine's associated with a TPLink account, so I fiddled with this bit and it worked flawlessly.

Edit: formatting

ghostseven · 2020-11-16T16:55:42Z

@ghostseven @snecklifter I went through that process over the weekend and managed to get through the new two step handshake to start an encrypted session with my upgraded HS110. Here's a small proof of concept python script demonstrating discovery, handshake and sending one command to get the device info: https://gist.github.com/chriswheeldon/3b17d974db3817613c69191c0480fe55

I also successfully turned the plug on and off 🎉

In summary:

UDP broadcast with fixed payload on port 20002 to discover devices.

Requests are plaintext HTTP on TCP port 80 with encrypted contents

Two step handshake to determine encryption key and base IV to use for later requests.

Key and IV seem to be derived from the device email and password. Presumably this is for the account registered for TP-Link cloud functionality. I haven't registered and so the email and password are empty strings. See https://gist.github.com/chriswheeldon/3b17d974db3817613c69191c0480fe55#file-hs110-py-L76

AES-128-CBC for request and response body encryption.

HTTP requests include an incrementing sequence number as a query parameter. This is used to determine the per-request IV.

Encrypted request and response bodies are prefixed with a HMAC.

Pre-documented requests seem to work e.g. {"system":{"get_sysinfo":null}}

There's some time-sensitive state in the plug that results in repeat calls in quick succession to the script generating 403 responses. Not sure what is causing that (too many connections, cookies, &c). Note that each time the script runs it generates a new session cookie with the device.

This is fantastic, literally just started picking through the encryption side and now I don't have to!

Thank you for putting the work in, I will have a play!

ghost · 2020-11-17T23:49:44Z

I've got a number of these and my experience is that the firmware update only applies to hardware version 4.1, which I believe is exclusively seen in the UK.

Weirdly one of my plugs got updated to Firmware revision 1.1.0 but the other which is also on hardware version 4,1 is still on an old version and even using the App to try to trigger a firmware update doesn't present it as an option. It could be that they've pulled the firmware of it could be something else entirely.

I'll be interested to see how you progress with this issue. Unfortunately it now means my plugs don't work in conjunction with Home Assistant so for the time being i've enabled the TPLink Alexa Skill so I can at least use voice control for some of my equipment.

TheDK · 2020-11-18T16:22:08Z

Can confirm that HW 4.0 (sold in Germany / Central Europe) does not get the FW update just yet, most recent is v1.1.5 which still has port 9999 open (works with HA). Really looking forward to the new mechanism as I guess we will be getting it as well and I really like the TP Link switches.

ghost · 2020-11-19T11:27:17Z

I actually opened a support ticket with TP-Link because the one that had upgraded to 1.1.0 was showing the wrong time-zone (Dawson/America) and it wouldn't allow me to change it. They have asked for some more information relating to this issue, but interestingly in the same e-mail they have specifically told me to NOT upgrade the other 4.1 device I have.

For the firmware version still in 1.0.4, if the app didn't have the new firmware upgrade available, please keep it. Because the 1.1.0 version was to avoid the potential attacks and security risks with 3rd party software, so currently, you don't need to upgrade it.

dragon2611 · 2020-11-19T13:26:39Z

Interestingly one of mine has that timezone, I thought I'd messed up readding it after resetting it to test an unpaired plug.

Support just asked me for the Mac of the plugs so I wonder if they're going to push a downgrade

jneilliii · 2020-11-19T16:50:25Z

Because the 1.1.0 version was to avoid the potential attacks and security risks with 3rd party software

you mean like this python module...lol.

watfordjc · 2020-11-20T03:07:48Z

@ghostseven @snecklifter I went through that process over the weekend and managed to get through the new two step handshake to start an encrypted session with my upgraded HS110. Here's a small proof of concept python script demonstrating discovery, handshake and sending one command to get the device info: https://gist.github.com/chriswheeldon/3b17d974db3817613c69191c0480fe55

I also successfully turned the plug on and off 🎉

There's some time-sensitive state in the plug that results in repeat calls in quick succession to the script generating 403 responses. Not sure what is causing that (too many connections, cookies, &c). Note that each time the script runs it generates a new session cookie with the device.

Using your gist, I bound to the IP address of an interface, used a fixed IP address for a specific plug, and tried to narrow down the timing issue.

I removed the time.sleeps from Handshake.

I modified main so it has a retry loop, and tried changing things until the number of retries was 0 or less than the maximum. This code results in no retries, or an average of 2-3 (at least for my network) when the first attempt results in a 403:

if __name__ == '__main__':
  s = requests.Session()
#  ip = discover()
#  print('Found device at {}'.format(ip))
  ip = '192.168.88.138'
  handshake = Handshake(ip)

  retry = 0
  while retry < 15:
    print("retry: ", retry)
    time.sleep(0.25)
    encryption = handshake.perform(s)
    (msg, seq) = encryption.encrypt('{"system":{"get_sysinfo":null}}')
    res = s.post('http://{}:80/app/request'.format(ip), params={'seq': seq}, data=msg)
    if res.status_code == 200:
      break
    retry += 1
  assert(res.status_code == 200)
  print(encryption.decrypt(res.content))

When encryption = handshake.perform(s) is outside of the retry loop, the number of retries is either 0 or 14 (it either succeeds on first try, or never succeeds). The only assert I noticed failing was the last one. Assuming encryption.encrypt() is correct, that leaves s.post().

I just tried adding a 5 second sleep before s.post() and it still resulted in some failures so I think it might be something (session invalidation?) happening on the plug. Does Kasa/cloud send retries or do they always succeed first time?

Key and IV seem to be derived from the device email and password. Presumably this is for the account registered for TP-Link cloud functionality. I haven't registered and so the email and password are empty strings. See https://gist.github.com/chriswheeldon/3b17d974db3817613c69191c0480fe55#file-hs110-py-L76

I am registered and can confirm filling in my TP-Link/Kaza e-mail address and password on line 81 works, whereas empty strings (and forgetting which password I used) resulted in 403 errors.

ghost · 2020-11-26T13:06:08Z

It might be possible to get the local API reinstated it seems: https://twitter.com/home_assistant/status/1331003814477000705

I've got a ticket open with them and they've confirmed my device MAC addresses are on their list, but the way it's worded makes it sound like they're working on another firmware which will then be sent to the MAC addresses they have recorded. No update on mine yet but i'll report back if things change.

ghostseven mentioned this issue Nov 12, 2020

HS100 Hardware V4.1 Firmware 1.1.0 - No Detection Even When IP Manually Specified plasticrake/homebridge-tplink-smarthome#154

Open

jneilliii mentioned this issue Nov 12, 2020

[BUG]: UK Models Running Firmware 1.1.0 No Longer Work With Plugin jneilliii/OctoPrint-TPLinkSmartplug#218

Closed

rytilahti mentioned this issue Nov 13, 2020

HS110(UK) not discoverable python-kasa/python-kasa#113

Closed

rytilahti mentioned this issue Nov 13, 2020

Lost communication with TP Link Kasa Plugs after firmware update home-assistant/core#43088

Closed

rytilahti mentioned this issue Nov 16, 2020

Implement the new protocol (HTTP over 80/tcp, 20002/udp for discovery) python-kasa/python-kasa#115

Closed

watfordjc mentioned this issue Nov 20, 2020

Add support for new handshake protocol #82

Open

limpep mentioned this issue Mar 27, 2021

Not getting any metrics fffonion/tplink-plug-exporter#12

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HS100 Hardware V4.1 Firmware 1.1.0 - No Longer Working - No Port 9999 #81

HS100 Hardware V4.1 Firmware 1.1.0 - No Longer Working - No Port 9999 #81

ghostseven commented Nov 12, 2020

corautem commented Nov 12, 2020

ghostseven commented Nov 12, 2020 •

edited

Loading

jneilliii commented Nov 12, 2020

synman commented Nov 12, 2020 •

edited

Loading

synman commented Nov 12, 2020

cp2004 commented Nov 12, 2020

jneilliii commented Nov 12, 2020

plasticrake commented Nov 13, 2020

dragon2611 commented Nov 13, 2020

ghostseven commented Nov 13, 2020 •

edited

Loading

dragon2611 commented Nov 13, 2020

ghostseven commented Nov 13, 2020

rytilahti commented Nov 13, 2020 •

edited

Loading

wilsonnkwan commented Nov 14, 2020

jneilliii commented Nov 14, 2020 •

edited

Loading

dragon2611 commented Nov 16, 2020

snecklifter commented Nov 16, 2020 •

edited

Loading

ghostseven commented Nov 16, 2020

snecklifter commented Nov 16, 2020

ghostseven commented Nov 16, 2020

chriswheeldon commented Nov 16, 2020 •

edited

Loading

cp2004 commented Nov 16, 2020 •

edited

Loading

ghostseven commented Nov 16, 2020

ghost commented Nov 17, 2020

TheDK commented Nov 18, 2020

ghost commented Nov 19, 2020

dragon2611 commented Nov 19, 2020

jneilliii commented Nov 19, 2020

watfordjc commented Nov 20, 2020 •

edited

Loading

ghost commented Nov 26, 2020

HS100 Hardware V4.1 Firmware 1.1.0 - No Longer Working - No Port 9999 #81

HS100 Hardware V4.1 Firmware 1.1.0 - No Longer Working - No Port 9999 #81

Comments

ghostseven commented Nov 12, 2020

corautem commented Nov 12, 2020

ghostseven commented Nov 12, 2020 • edited Loading

jneilliii commented Nov 12, 2020

synman commented Nov 12, 2020 • edited Loading

synman commented Nov 12, 2020

cp2004 commented Nov 12, 2020

jneilliii commented Nov 12, 2020

plasticrake commented Nov 13, 2020

dragon2611 commented Nov 13, 2020

ghostseven commented Nov 13, 2020 • edited Loading

dragon2611 commented Nov 13, 2020

ghostseven commented Nov 13, 2020

rytilahti commented Nov 13, 2020 • edited Loading

wilsonnkwan commented Nov 14, 2020

jneilliii commented Nov 14, 2020 • edited Loading

dragon2611 commented Nov 16, 2020

snecklifter commented Nov 16, 2020 • edited Loading

ghostseven commented Nov 16, 2020

snecklifter commented Nov 16, 2020

ghostseven commented Nov 16, 2020

chriswheeldon commented Nov 16, 2020 • edited Loading

cp2004 commented Nov 16, 2020 • edited Loading

ghostseven commented Nov 16, 2020

ghost commented Nov 17, 2020

TheDK commented Nov 18, 2020

ghost commented Nov 19, 2020

dragon2611 commented Nov 19, 2020

jneilliii commented Nov 19, 2020

watfordjc commented Nov 20, 2020 • edited Loading

ghost commented Nov 26, 2020

ghostseven commented Nov 12, 2020 •

edited

Loading

synman commented Nov 12, 2020 •

edited

Loading

ghostseven commented Nov 13, 2020 •

edited

Loading

rytilahti commented Nov 13, 2020 •

edited

Loading

jneilliii commented Nov 14, 2020 •

edited

Loading

snecklifter commented Nov 16, 2020 •

edited

Loading

chriswheeldon commented Nov 16, 2020 •

edited

Loading

cp2004 commented Nov 16, 2020 •

edited

Loading

watfordjc commented Nov 20, 2020 •

edited

Loading