Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New API #3

Open
slajgaj opened this issue Sep 18, 2023 · 51 comments
Open

New API #3

slajgaj opened this issue Sep 18, 2023 · 51 comments
Assignees
@akbooer

Comments

@slajgaj
Copy link

slajgaj commented Sep 18, 2023

Hello!
I working with Netatmo plugin (Micasaverde 3)
From junius not working.
The Natatmo "said" , have a new API structure.
Can you planning make your plugin ?
@akbooer
Copy link
Owner

akbooer commented Sep 18, 2023

You need to use the latest version, which was updated at the start of 2023, to use the latest API.

If you still have an error, then I need to see more information about it.

@slajgaj
Copy link
Author

slajgaj commented Sep 18, 2023

I replaced the 2 jsons and now have error
image1
image0

@akbooer
Copy link
Owner

akbooer commented Sep 18, 2023

Some of the other files have changed. In particular, the Lua file. Update them all and restart to try again.

I see you are running UI5. This is very old, and it’s possible that some of the Lua modules to do with secure HTTP are not new enough versions. I don’t have a Vera with UI5 any more myself.

I would suggest you also try asking at the Smarthome forum https://smarthome.community/ if updating the files doesn’t work.

@akbooer akbooer self-assigned this Sep 18, 2023
@slajgaj
Copy link
Author

slajgaj commented Sep 18, 2023

Ok thank you
Than i trying the Vera upgrade from UI5 to

@Tim88-8
Copy link

Tim88-8 commented Jul 3, 2024

Hi. I hope you are still supporting this for Vera. My Netatmo had been running fine until recent and now I am currently having a problem where Vera is reporting 'Not Authorized' and when I click the Authorize button and sign in using my Netatmo credentials I get the following error: 'Failed to retrieve access token using authorization code'. Hopefully are still supporting this and can lend a hand.
Regards,
Tim

@akbooer
Copy link
Owner

akbooer commented Jul 3, 2024

I don't run Vera any more, but use this daily under openLuup. Have just fired up a test version and everything worked as expected. I haven't been informed of any recent changes from Netatmo. Has something on the Vera side changed recently?

@Tim88-8
Copy link

Tim88-8 commented Jul 3, 2024 via email

@akbooer
Copy link
Owner

akbooer commented Jul 3, 2024 •
edited
Loading

The Vera backend shouldn't be involved – it's simply a direct HTTPS call to the Netatmo website. I'm assuming that Vera hasn't made any firmware changes recently. It may just be a security level issue with an old library. What Vera, what firmware?

What output is produced when you run this code in Lua Test:


local socket  = require "socket"
local https 	= require "ssl.https"

print(socket._VERSION)
print(https._VERSION)

For further support on this, it might be good if we switch the conversation to smarthome.community since there may be others who can contrubte.

@Tim88-8
Copy link

Tim88-8 commented Jul 3, 2024 via email

@Tim88-8
Copy link

Tim88-8 commented Jul 3, 2024 via email

@akbooer
Copy link
Owner

akbooer commented Jul 3, 2024

My bad.... I think the Vera doesn't support print (whereas openLuup does!)

I get

Socket version = LuaSocket 3.0-rc1
HTTPS version = 0.9

I don't know if that's a smoking gun, or how long Vera has used 0.4.

I will try and investigate further...

@Tim88-8
Copy link

Tim88-8 commented Jul 3, 2024 via email

@SinSharks
Copy link

SinSharks commented Jul 4, 2024 •
edited
Loading

Sorry for the likely dumb question.

I was watching the web traffic when trying to "Authorize" or in this case re-"Authorize" my plugin and noticed the website does send an access token/refresh token it just doesn't appear to be updating in the "RefreshToken" field in Vera for whatever reason.

I tried to input in the field manually and that unfortunately didn't appear to take. I then restarted LUUP and am now getting this error on the netatmo site:
"Array is not allowed scope for oepnLuup."

That is likely from me messing with RefreshToken Variable and trying to put the new token on top of the previous using this format:
"Nothereallytoken|refreshtokennotreal"

akbooer -- Would it be possible to just input the token manually somewhere?

Thank you for akbooer for time and effort you put into making this wonderful plugin.

@akbooer
Copy link
Owner

akbooer commented Jul 4, 2024

@SinSharks

The RefreshToken variable is only for the refresh token, and it's only there so that the plugin survives Luup restarts without needing to be re-authorized (so long as the token has not expired.) The access token is never stored. Just entering a valid refresh token should work on restart.

Has this problem just started? You're on openLuup, so this might be a clue. Why were you looking at the web traffic? Did you originally get an authourize failure?

@SinSharks
Copy link

SinSharks commented Jul 4, 2024 •
edited
Loading

Hello akbooer,

I find myself in a situation similar to Tim88-8, using a Vera Plus with firmware version 1.7.5186(7.31). I started by examining web traffic to pinpoint where the issue with the original error 'Failed to retrieve access token using authorization code' might be. I wanted to verify whether the token was not being retrieved at all or if it was retrieved successfully but Netatmo made slight changes causing Vera to not recognize the token. (Netatmo recently underwent significant GUI changes for both their app and web interface, leading me to consider they might have inadvertently altered something on the API side during these updates, that is not yet listed in their documentation.)

@akbooer
Copy link
Owner

akbooer commented Jul 4, 2024

Can you both confirm that:

  1. you are using the latest version 2023.01.08
  2. you have deleted the old variables ClientID and ClientSecret

@Tim88-8
Copy link

Tim88-8 commented Jul 4, 2024 via email

@Tim88-8
Copy link

Tim88-8 commented Jul 4, 2024 via email

@Tim88-8
Copy link

Tim88-8 commented Jul 4, 2024 via email

@akbooer
Copy link
Owner

akbooer commented Jul 4, 2024

@Tim88-8
The plugin is designed to use your own ClientID and ClientSecret if they're specified in the device variables, and the app is enabled in your account on the DEV website. The fact that it's been working (I presume) probably indicates that it's something else which has recently changed. What's bothering me is that I'm having no problems right now (the app is refreshing every 2 hours.)

I may need to extend the diagnostics to discover more. Sorry about this!

@akbooer
Copy link
Owner

akbooer commented Jul 4, 2024 •
edited
Loading

OK, I have set up a new openLuup installation, installed the Netatmo plugin, and now it times out when trying to authorize. It's doesn't actually get to asking for login credentials, which seems wrong.

So something with their site HAS changed. I'm signed up to developer emails, but haven't heard anything about an API change. So it's not anything you're doing wrong.

@akbooer
Copy link
Owner

akbooer commented Jul 4, 2024

Further to this, I've seen this comment on the HA website

"I just received an email from Netatmo that they will update their policy regarding API keys and Access/Refresh Tokens as per May 29, 2024. "

It may be the issue. Why I never got this message, I don't know, but I will dig into this further.

@Tim88-8
Copy link

Tim88-8 commented Jul 4, 2024 via email

@Tim88-8
Copy link

Tim88-8 commented Jul 4, 2024 via email

@Tim88-8
Copy link

Tim88-8 commented Jul 4, 2024 via email

@SinSharks
Copy link

I do really appreciate your quick response and spending the time to look into this.
…
On Thu, Jul 4, 2024 at 1:02 PM AK @.> wrote: Further to this, I've seen this comment on the HA website <home-assistant/core#111938> "I just received an email from Netatmo that they will update their policy regarding API keys and Access/Refresh Tokens as per May 29, 2024. " It may be the issue. Why I never got this message, I don't know, but I will dig into this further. — Reply to this email directly, view it on GitHub <#3 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/BJTX5PV6VXIPMBLC2FFG5KTZKV535AVCNFSM6AAAAABKIPPIFSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBZGM2TKNRTHE . You are receiving this because you were mentioned.Message ID: @.>

100% agree thank you so much for looking at this.

When you find out what's going on and come up with a solution, I'd be more than happy to test it on my Vera controller.
…
On Thu, Jul 4, 2024 at 1:19 PM Tim @.> wrote: I do really appreciate your quick response and spending the time to look into this. On Thu, Jul 4, 2024 at 1:02 PM AK @.> wrote: > Further to this, I've seen this comment on the HA website > <home-assistant/core#111938> > > "I just received an email from Netatmo that they will update their > policy regarding API keys and Access/Refresh Tokens as per May 29, 2024. " > > It may be the issue. Why I never got this message, I don't know, but I > will dig into this further. > > — > Reply to this email directly, view it on GitHub > <#3 (comment)>, > or unsubscribe > https://github.com/notifications/unsubscribe-auth/BJTX5PV6VXIPMBLC2FFG5KTZKV535AVCNFSM6AAAAABKIPPIFSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBZGM2TKNRTHE > . > You are receiving this because you were mentioned.Message ID: > @.***> >

I'd also be more than willing to attempt any debug/testing. (Sadly, I'm not a super skilled at programming but can certainly help test anything you would like on my Vera and send debug logs.)

@akbooer
Copy link
Owner

akbooer commented Jul 5, 2024

Just an update: All appears to be working on my production system (openLuup in a Docker environment) and in my development system (openLuup in an iMac.) I have dug around in a junk box and found an old Vera Edge which I'll agtempt to fire up and try it on there. Most odd.

@Tim88-8
Copy link

Tim88-8 commented Jul 5, 2024 via email

@akbooer
Copy link
Owner

akbooer commented Jul 5, 2024

...not so sure now. It didn't work on Vera, but now it's failing on both my production system and development system. It;s obviously them, not us, but I have no idea how to proceed.

@akbooer
Copy link
Owner

akbooer commented Jul 5, 2024

Aha! We are not the only ones... Netatmo Help Centre

@Tim88-8
Copy link

Tim88-8 commented Jul 5, 2024 via email

@Tim88-8
Copy link

Tim88-8 commented Jul 5, 2024 via email

@akbooer
Copy link
Owner

akbooer commented Jul 5, 2024 •
edited
Loading

Yes, there seems to be an element of confusion here, I had tried reverting to a private ClientSecret and ClientID using your own app on the Netatmo site, as originally, but that hasn't worked either. I'm really annoyed my standard installation has just started to fail!

@akbooer
Copy link
Owner

akbooer commented Jul 6, 2024

Appreciate the offers for help with testing. At the moment, I'm working in the openLuup environment. So far, I find that the authentication viw the Netatmo website works OK and returns access and refresh tokens. However, using those tokens results in an "invalid grant" response. This looks like the tokens have expired, but they're the new ones, so, again, I'm at a bit of a loss.

@Tim88-8
Copy link

Tim88-8 commented Jul 6, 2024 via email

@Tim88-8
Copy link

Tim88-8 commented Jul 7, 2024 via email

@akbooer
Copy link
Owner

akbooer commented Jul 7, 2024

Again, I would refer you to smarthome.community for opinions other than my own (and mine too!)

@Tim88-8
Copy link

Tim88-8 commented Jul 7, 2024 via email

@akbooer
Copy link
Owner

akbooer commented Jul 7, 2024

Oh, that’s not good. I’ll try to notify them myself.

@smkess
Copy link

smkess commented Jul 19, 2024

Looks like this was resolved??? I was just able to authorize via my vera for the first time in a couple of months. Didn't change anything in my config.

@Tim88-8
Copy link

Tim88-8 commented Jul 20, 2024 via email

@SinSharks
Copy link

SinSharks commented Jul 20, 2024 •
edited
Loading

I'm not having any luck either I get this message:
image
but when I go back and check it still hasn't received the refresh token (with and without rebooting the luup engine.) and it still says it can't connect:
image
When I look at the Device Tree & Device List page I see this error:
[string "--module ("L_Netatmo", package.seeall) --..."]:572: bad argument #1 to 'pairs' (table expected, got nil)
and the Diagnostics page just displays this:
NETATMO CONFIGURATION PAGE at Sat Jul 20 04:20:16 2024

I really wish they would just roll this changes back... :(

@smkess
Copy link

smkess commented Jul 20, 2024

Mine still working…..yesterday, after I got the “access granted…..restart LUUP engine” I didn’t just restart LUUP, I rebooted the Vera ( for quite a while now I’ve noticed that whenever I change anything like add or remove a device, etc. if I don’t reboot the Vera the changes don’t get applied properly). After the reboot, initially the netatmo data came in but it was the data from the last time it refreshed before the auth failure and then a couple of minutes later it refreshed by itself to the current data and it’s been bringing current data in as recently as about two minutes ago. Don’t know if it will last or start failing again at some point but it’s working for now. Not sure why based on what you all are saying.

@akbooer
Copy link
Owner

akbooer commented Jul 20, 2024

Sadly, mine not working either.

In the circumstances, there doesn't seem any point in trying to fiddle with the code.

I have no idea what is going on here.

@Tim88-8
Copy link

Tim88-8 commented Jul 21, 2024 via email

@SinSharks
Copy link

SinSharks commented Jul 24, 2024 •
edited
Loading

Ok some interesting notes here. I was able to get the weather station to authorize on my Vera controller. What's interesting is I tried logging on to http://dev.netatmo.com and it redirected me to http://home.netatmo.com so it seems that the dev account that I had created is no longer necessary as I can not access it any longer. After the redirect I went to the partner apps section under my account ( 'Settings\My Account\Partner Apps') and noticed there were several entries for 'OpenLuup' so what I did was remove the oldest entry and it removed all the entries. There are also 5 entries named Tim's Netatmo Weather Station all with the same date. I did not remove these for fear of breaking something as I'm not sure what these are. After removing the entries I went back to the Netatmo app in my Vera controller and clicked authorize again and this time the device came back on line and is responding again.
…
On Sat, Jul 20, 2024 at 3:49 PM AK @.> wrote: Sadly, mine not working either. In the circumstances, there doesn't seem any point in trying to fiddle with the code. I have no idea what is going on here. — Reply to this email directly, view it on GitHub <#3 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/BJTX5PSNAAL3MUDDL647SK3ZNK5MVAVCNFSM6AAAAABKIPPIFSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBRGI3TMOJQGU . You are receiving this because you were mentioned.Message ID: @.>

More updates:

I tried your method/suggestions and it appears to have worked for moment. Yay!

Some things that differed in my case:

  1. I did not have something called "Sharks" weather station in my partner apps just 5 openluup entries.
  2. All of my openluup entries were from 2024/07/20.

-break
When I deleted one of the openluup entries they all got deleted same as yours.

Other items worth noting:
Device Tree & Device List - Display nothing just a white screen.
Diagnostics = Displays properly displays the same as it had before the whole token issue.

This worked about 20 mins ago I'll give another update if it holds for 24-48 hours.

@Tim88-8
Copy link

Tim88-8 commented Jul 24, 2024 via email

@smkess
Copy link

smkess commented Jul 24, 2024 •
edited
Loading

Same here........Still up and running as of 5mins ago. Netatmo must have changed something or rolled the changes back so the plugin started working again with a clean environment. I didn't change anything or reinstall anything. The only thing I did was try to reauthorize it and reboot the vera after.

@SinSharks
Copy link

Giving my update, system still seems to be working after 2 days, my scenes/triggers are all functioning as desired again. I'm not sure what Netatmo did on the back-end but whatever seems to be fixed the issue.

@akbooer
Copy link
Owner

akbooer commented Jul 26, 2024

@SinSharks

So, just to confirm, you have no private authorization apps, and you’re using the default one with no specified ClientID and ClientSecret.

@SinSharks
Copy link

SinSharks commented Jul 26, 2024 •
edited
Loading

@akbooer -- Yes, that is correct. (Note: I do have an app but client ID is 100% different from what Vera has.)

Here is what I did to get it working:

  1. Rolled Back to Older Vera Image before this issue was a thing, as I had reinstalled the plugin about 4-5 times and Vera was really unhappy.
  2. Followed @Tim88-8 suggestion:

Ok some interesting notes here. I was able to get the weather station to authorize on my Vera controller. What's interesting is I tried logging on to http://dev.netatmo.com and it redirected me to http://home.netatmo.com so it seems that the dev account that I had created is no longer necessary as I can not access it any longer. After the redirect I went to the partner apps section under my account ( 'Settings\My Account\Partner Apps') and noticed there were several entries for 'OpenLuup' so what I did was remove the oldest entry and it removed all the entries. There are also 5 entries named Tim's Netatmo Weather Station all with the same date. I did not remove these for fear of breaking something as I'm not sure what these are. After removing the entries I went back to the Netatmo app in my Vera controller and clicked authorize again and this time the device came back on line and is responding again.
…
On Sat, Jul 20, 2024 at 3:49 PM AK @.> wrote: Sadly, mine not working either. In the circumstances, there doesn't seem any point in trying to fiddle with the code. I have no idea what is going on here. — Reply to this email directly, view it on GitHub <#3 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/BJTX5PSNAAL3MUDDL647SK3ZNK5MVAVCNFSM6AAAAABKIPPIFSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBRGI3TMOJQGU . You are receiving this because you were mentioned.Message ID: @.>

  1. It started working after that and has been going strong for 2 days now.

Step by Step of what I did:

  1. Roll Back Vera image prior to app breaking.

  2. Went to Netatmo's Website

  3. Logged in.

  4. Went to: Settings > My Account > Partner Apps
    image

  5. Saw 5 separate instances of openLuup all dated 2024/07/20
    image

  6. Removed 1 of the openLuup apps which resulted in it and all the other instances being removed.
    image

  7. Went back to Vera/Netatmo plugin, and clicked the Authorize button.

  8. That took me to the Authorization Page and it said your app was successfully authorized.

  9. Rebooted Vera for good measure.

  10. Saw the app was reconnected and had updated the temps to match what I was seeing on Netatmo's webpage/app, I waited through a few 5 minute refresh cycles and confirmed it was in fact being updated every 5 mins.

  11. Checked periodically the last 48 hours and it keeps updating.

  12. Watched as my temperature based trigger/scenes go off correctly based on Netatmo's measurements.

Hopefully, that will get you up and running and/or at least help debugging you setup. I again can't thank you enough for making this wonderful plugin and taking the time to debug it, especially as it appears Netatmo themselves are the cause of this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@akbooer akbooer

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@akbooer @SinSharks @slajgaj @Tim88-8 @smkess