Homie mqtt protocoll integration

[nebkrid]

following PR from osresearch#26 (comment)

• added Homie convention and automatically identifies settable features and renaming.
• code restructuring: renaming some variables to better following what is meant (feature (json entry of a feature), feature_name, uid, ...).
• method calls between hc2mqtt are all uid based. Renaming is completely done in hc2mqtt (avoid conflicting short names)
• renaming is done within config: Manually add a json entry "hc2mqtt"! see updated READM.
• --mqtt_prefix changed to "homie/" for autodetection in smart-home systems (see homie specification)
• start program is added by using the mqtt topic .../ActiveProgram

Default is to expose values according to [homie mqtt protocoll] {https://homieiot.github.io/specification} for auto-detection in smart home systems.
for backwards compatibility:
a) call hc2mqtt with --publish_as_json option to expose a full json to one MQTT value
b) renaming of some parameters: define in config (see above)
c) call with --mqtt_prefix "homeconnect/"

[Skons]

There are a few things in regards to this PR.

First homie is added, but it seems to be mainly focused on homie. The mqtt default topic is changed, and the shebang. Wouldn't it be better to have a -homie parameter or something like that, which prepares the use for homie? instead of the use a set of parameters? And does changing the shebang have implications on not homie users?

Second: --log_debug is ugly imo. This should be --loglevel debug/info(default)/warning .... I already though that print should not be used, but logger should be used. But i thought it was a to big of a change to introduce with what i did. The use of different way of logging could be introduced later, but i thing the parameter should be changed.

[nebkrid]

Thank you for your feedback.
I can see your point that it seems to be mainly focused on homie. Actually it was not the intention to focus on homie, but on MQTT. To my understanding of MQTT it is not meant to have one topic in which a big json string is exposed, but to have different topics for each parameter. Therefore I think - although it is a big change - it is useful to not expose one json but many topics as default. The homie-topics (the ones with "$") are only an addition for automatic detection of these channels and shouldn't disturb other users, since they don't need to query them. However, if there are scenarios where these additional topics are disturbing, this could be changed with some additional work to expose these homie-topics only with an additional e.g. homie-parameter.
On the other hand I disliked the change of the main topic to "homie/", too, but it was necessary for automatic detection... So here I totally agree to make this change dependent based on e.g. a homie-parameter.
Another option would be to let the user define the main topic in the hc2mqtt json part of the config file which I introduced mainly for choosing and renaming of the parameters instead of having this hard-coded in the source code. This was already suggested in an issue. What do you think would be the better option?

Changing of the shebang was not intended. I thought I reverted this... Didn't this commit synchronize?

Regarding the debug option: I am completely open to any changes here. I don't have experience with best practices for logging and especially not which options python offers.

PS: I won't have time for this in the next two to three weeks, so please don't be irrititated if my next message will take some time. I will definitly come back to this code! :)

[Skons]

Ah, the shebang has been reverted indeed. I read that wrong.

That debug should be changed, which later on it could get better logging. Now print is used, which should be replaced by logging imo. Thats something for the future.

I created my PR hoping it would land in home assistant. Having one repository managing homie and home assistant is probably not very feasible. But i dont think home assistant will be integrated into this code. But it would be nice if homie is a choice, and not an obligation or a default. The default should be just mqtt...

And yes, the config could do so much more then just smart devices :)

Take your time :)

[JW5PX]

I agree that the Homie integration (what is Homie?) should be optional, although I appreciate this implementation because with certain modifications, I was able to get everything I needed for my custom Raspberry-based home automation system, including dishwasher start and scheduling based on the cheapest hourly electricity pricing.

[nebkrid]

Sorry for the even longer delay than I thought. But now I finally found some time :)

That debug should be changed, which later on it could get better logging. Now print is used, which should be replaced by logging imo. Thats something for the future.

Yes, let's shift this for future and get the rest combined :)

@Skons: I created my PR hoping it would land in home assistant.
@JW5PX: (what is Homie?)

My itention is very similar, but for openhab instead home assistant (on the other hand I read that they are quite similar / nearly the same? But I am not sure in how fare they are technically the same...). However, in openhab there is an MQTT binding which automatically can import the so-called homie-specifiation. (PS: I didn't know homie before, either, but stumbled over it when importing a cloud free open-source implementation of cleaning robots firmware ("valetudo") and was impressed that all features/properties were automatically found without any specific openhab AddOn for valetudo itself. So I thought: This is the easiest way to get this script into smart home systems without need for someone else, who knows how to program a specific add for e.g. openhab.)
The basic idea of homie is, that e.g. a dish washer with MQTT topics
/topic/path/power
/topic/path/program
gets additional "meta data" topics -> values like
/topic/path/$properties-> power, program
/topic/path/power/$name -> 'power'
/topic/path/power/$datatype-> Boolean
/topic/path/program/$datatype-> Enum
...
With this additional meta-data topics it is possible to import the whole MQTT object (in this example already a "Power" attribut that is a boolean), without an additional add-on.

Having one repository managing homie and home assistant is probably not very feasible. [...] But it would be nice if homie is a choice, and not an obligation or a default. The default should be just mqtt...

I don't think that homie + home assistant exclude each other in this basic script. If anyone use it to create a add on for a specific smart-home implementation, he/she can use the appropriate options to choose how he/she wants to have it reported.
With the short explanation above is the main question is, how do we define "mqtt"? In my eyes the current version is not really MQTT, but JSON, since at the end it sends a big JSON object in one MQTT channel, which needs additional processing to get all the different properties from it. Therefore removing Homie would just result in removing the meta data topics. But the big change from one JSON object to many MQTT channels should be the default anyway. (However as it is simple, enabling the backwards-compatability with one JSON object can easily be kept, so you and others with the first implementation don't have to change your running systems)

And yes, the config could do so much more then just smart devices :)

Ok, then I will include the option with base topic choosing here.

To sum it up (Please add if I forgot something)
I plan to

• make default base topic "hc2mqtt" again. Name adaptable via config file.
• change ugly names of debug options to less ugly ones. Full logging options moved to future
• move export-all-as-one-big-json option in config file for backwards compatibility
• if easy to implement: "homie"-option in config data for turning on/off homie meta data topics.
• add bugfixes from @JW5PX

How do you both think about this? :)

[nebkrid]

@JW5PX

I was able to get everything I needed for my custom Raspberry-based home automation system, including dishwasher start and scheduling based on the cheapest hourly electricity pricing.

A little bit offtopic (is there a better way in github?): How do you get it to "manually" start your dishwasher when you filled it but then have a auto-start later on? I plan to do the same based on solar power production and thought maybe enabling the manual-remote-button on the machine can be used as a trigger in my smart home system to watch for solar power (or in your case cheap price).

[Skons]

@nebkrid thats a lot of information :P. First you can skip that debug you are planning on. It has already been fixed in my latest release.

As for homie and home assistant, they maybe not be that far apart, the implementation can be fastly different. I can agree on the way you want to implement the topics, but what are those $ signs exactly? And is this applicable for all devices? If i look at the documentation of the home connect api not every device has the same options. So if you implement the program topic, the program feature must be present.

what do you mean with:
move export-all-as-one-big-json option in config file for backwards compatibility

[nebkrid]

As for homie and home assistant, they maybe not be that far apart, the implementation can be fastly different.

Please have a look into the examples I added to the documentation. They don't interfere with each other. So if you try the commit I just pushed, as long as you enable the option "publish_as_json": true and adapt the renaming of features to your setup in the config file, it should work right away with your current configuration, since the hc2mqtt script then generates exactly the same MQTT output as your current one.

but what are those $ signs exactly?

This is the only thing which makes the implementation to a "homie" one. If I would explain more than I already tried it would be actually a copy-paste from the documentation, which I linked in my last post. It is not very complicate, maybe takes 15 min to understand. You can also have a look into example No. 4 in the documentation I added (or the example at the homie overview weppage)

And is this applicable for all devices? If i look at the documentation of the home connect api not every device has the same options.

That's the main thing of my whole pull request: I generate the topics from the config file. Here the HC devices define there options in a JSON format which is read by the hc2mqtt script.
E.g. for the door feature it reads that it can have the state "Open" and "Closed". (For program is indeed special and the detection from the script may have to adapted if anyone has other devices to do this.)

what do you mean with:
move export-all-as-one-big-json option in config file for backwards compatibility

The current implementation of "MQTT" publishing in your/the original script from osresearch. Now it is the `"publish_as_json" option of my last commit (see example No. 1 in the readme file.)