Python oauth implementation troubleshooting

[skimike02]

Following the steps from @uhthomas here, I came up with the following python3. It works until it gets to the MFA url, which just returns a blank list

b'{\n  "data": []\n}'

instead of the JSON expected. MFA is set up for the account in question.

Any help appreciated.

import os
import base64
import hashlib
import random
import string
import requests
from bs4 import BeautifulSoup

verifier_bytes = os.urandom(86)
challenge = base64.urlsafe_b64encode(verifier_bytes).rstrip(b'=')
challenge_bytes = hashlib.sha256(challenge).digest()
challenge_sum = base64.urlsafe_b64encode(challenge_bytes).rstrip(b'=')
state=''.join(random.choices(string.ascii_uppercase + string.digits, k=10))

authorize_url="https://auth.tesla.com/oauth2/v3/authorize"

params={
    'client_id':'ownerapi',
    'code_challenge':challenge_sum,
    'code_challenge_method':'S256',
    'redirect_uri':'https://auth.tesla.com/void/callback',
    'response_type':'code',
    'scope':'openid email offline_access',
    'state':state}

data={
    'identity': [email],
    'credential': [pw]}

s=requests.Session()

r=s.get(authorize_url,params=params,data=data)
soup = BeautifulSoup(r.content, 'html.parser')

for item in soup.find_all('input'):
    if item.get('name') not in data.keys():
        data[item.get('name')]=[item.get('value')]

headers = {'Content-type': 'application/x-www-form-urlencoded'}        
r=s.post(authorize_url,headers=headers,params=params)

mfa_uri='https://auth.tesla.com/oauth2/v3/authorize/mfa/factors'

r=s.get(mfa_uri,data={'transaction_id':data['transaction_id'][0]})

The last GET has

 'url': 'https://auth.tesla.com/oauth2/v3/authorize/mfa/factors',
 'body': 'transaction_id=[transactionID]',

where [transactionID] is an 8 character string with no brackets. entering random strings for [transactionID] gets the same result.

[uhthomas]

Sorry, maybe I should clarify the request for MFA devices is using a URL query, not an encoded form.

[skimike02]

Thank you. I tried that too, with the same result:

'url': 'https://auth.tesla.com/oauth2/v3/authorize/mfa/factors?transaction_id=[transactionID]',
'body': None,

It looks like my post request before MFA was hitting an error 400, so that is probably part of the problem. Perhaps I misinterpreted "Encode the map to the request body as a URL encoded form." I took this to mean the map we had created went in the body, and the other details noted went in the URL (shown below), though I also tried putting all items in the body.

{'method': 'POST',
 'url': 'https://auth.tesla.com/oauth2/v3/authorize?client_id=ownerapi&code_challenge[challenge]&code_challenge_method=S256&redirect_uri=https%3A%2F%2Fauth.tesla.com%2Fvoid%2Fcallback&response_type=code&scope=openid+email+offline_access&state=[state]',
 'headers': {'User-Agent': 'python-requests/2.25.1', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-type': 'application/x-www-form-urlencoded', 'Cookie': 'tesla-auth.sid=[cookie]', 'Content-Length': '181'},
 '_cookies': <RequestsCookieJar[Cookie(version=0, name='tesla-auth.sid', value=[value], port=None, port_specified=False, domain='auth.tesla.com', domain_specified=False, domain_initial_dot=False, path='/', path_specified=True, secure=True, expires=1612244422, discard=False, comment=None, comment_url=None, rest={'HttpOnly': None}, rfc2109=False)]>,
 'body': 'identity=[email]&credential=[credential]&_csrf=[csrf]&_phase=authenticate&_process=1&transaction_id=[transaction_ID]&cancel=',
 'hooks': {'response': []},
 '_body_position': None}

[heumn]

I have a working non-2FA and 2FA support here if you need some tips for how to do it in Python

https://github.com/enode-engineering/tesla-oauth2

[baysinger]

Thanks for your code. It does indeed work, but I'm trying to understand it. Why do you need to strip the padding (rstrip(b"=")) off of all the base64 representations?

Edit: To answer my own question: That's just a spec. RFC 7636 specifies the base64 encode function as this:

     static string base64urlencode(byte [] arg)
     {
       string s = Convert.ToBase64String(arg); // Regular base64 encoder
       s = s.Split('=')[0]; // Remove any trailing '='s
       s = s.Replace('+', '-'); // 62nd char of encoding
       s = s.Replace('/', '_'); // 63rd char of encoding
       return s;
     }

I'm still not sure why they do that. But they do!

[fkhera89]

The Mfa breaks or basically returns empty list if you have no Mfa devices enabled. If the goal is to just get the api working I would skip Mfa cause it makes this problem pretty complex and tracking refresh tokens etc.

I used @heumn example code and it worked great !

[skimike02]

Apparently mine was returning empty list because I was locked out of my account from too many failed login attempts (the result of too much testing). I thought it would give a 401, but apparently it gives a 200 with a blank list.

[jeffkowalski]

I've added a ruby implementation of the full flow, based on @heumn's work
enode/tesla-oauth2#4

[natrlhy]

How does this work if I have 2 MFA devices (Bitwarden and Duo) to select for the one time passcode?

[Pyromancer579]

@skimike02 Thank you for the code and the time and effort you've put in.

I'm trying to do much the same as @Natrhy, but want to swap between whatever mode my Powerwall is currently in to 'Back-Up' and visa versa as and when various conditions are met (to prevent my Powerwall discharging into my EV, when the EV charger conducts a timed charge).

I have slotted your code into mine (QuaterHourCheck.py - designed to be run by Cron every 15 minutes), so I can use my existing logic and conditions, and have put in a few print statements instead of the raise ValueError's, for the time being, so I can try and understand what is going on.

The best I can get is:

  GET authorization form successful after 1 attempt(s).
  POST authorization form unsuccessful after 10 attempts.
  No MFA Detected.

  Traceback (most recent call last):
  File "/home/pi/ZappiPowerwall/QuaterHourCheck.py", line 304, in <module>
  authenticate(TES_ID,TES_PW)
  File "/home/pi/ZappiPowerwall/QuaterHourCheck.py", line 161, in authenticate
  code = parse_qs(r.headers["location"])["https://auth.tesla.com/void/callback?code"]
  KeyError: 'https://auth.tesla.com/void/callback?code'

I've tried increasing MAX_ATTEMPTS to 20, to see if that would help, but no luck.

Any suggestions?

[dvermagithub]

First of thanks @skimike02 - you have saved me hours of sifting through XMLs and CSS. Not a front-end savvy person...so its greek to me.

@Pyromancer579 for line 68

  #Send email and pw
    for i in range(MAX_ATTEMPTS):
        r=s.post(authorize_url, headers=headers, data=data, params=params, allow_redirects=False)
        if r.ok and "<title>" in r.text:

change to

 #Send email and pw
    for i in range(MAX_ATTEMPTS):
        r=s.post(authorize_url, headers=headers, data=data, params=params, allow_redirects=False)
        if r.ok and r.status_code == 302:

or better yet

 #Send email and pw
    for i in range(MAX_ATTEMPTS):
        r=s.post(authorize_url, headers=headers, data=data, params=params, allow_redirects=False)
        if r.ok and (r.status_code == 302 or "<title>" in r.text):

[Pyromancer579]

@dvermagithub Thank you.

Got it working once I spotted the missing speach mark after <title> in the 'better yet' solution.

Now have a 'refresh_token' is not defined error, even though I can print the value of 'refresh token' so off to poke around to try and solve this, I may be back on here asking for help later - hopefully not.

Thanks again.

[dvermagithub]

Np. Fixed missing quote mark

[Pyromancer579]

The token retrieval/authentication seems to be working okay now - had to make a few variables global in the appropriate places.

But now my original code for changing the mode of my Powerwall isn't working, I've put the code below and the response I get:

256    async def Boost():
257        client = TeslaApiClient(TES_ID, TES_PW)
258        energy_sites = await client.list_energy_sites()
259        assert(len(energy_sites)==1)
260        N_MODE = await energy_sites[0].get_operating_mode()
261        if N_MODE==C_MODE:
262            await energy_sites[0].set_operating_mode('backup')
263            N_MODE = await energy_sites[0].get_operating_mode()
264        await client.close()

File "/home/pi/ZappiPowerwall/QuaterHourCheck.py", line 258, in Boost
energy_sites = await client.list_energy_sites()
File "/home/pi/.local/lib/python3.7/site-packages/tesla_api/init.py", line 106, in list_energy_sites
return [_class(self, products['energy_site_id']) for products in await self.get('products')]
File "/home/pi/.local/lib/python3.7/site-packages/tesla_api/init.py", line 79, in get
await self.authenticate()
File "/home/pi/.local/lib/python3.7/site-packages/tesla_api/init.py", line 67, in authenticate
expiry_time = timedelta(seconds=self.token['expires_in'])
KeyError: 'expires_in'

Looks to me like some sort of conflict between the old authentication code and the new system i.e. I'm still using the old system to try and get the required data and switch modes, but I don't know the appropriate code for doing the same under the new system.

Any guidance on how to alter the Powerwall mode would be much appreciated.

[labdel]

Hi All,
htanks for the python code, I pull the code and I ran it in my local PC pycharm and it works great I am able to get the taoken and refresh token. Problem I have is when I run same .py file in my AWS EC2 ubuntu machine it never returns in this line.
r=s.get(authorize_url,headers=headers,params=params)

I appeciate your feedback.

Thanks i nadvance - Leo.

[skimike02]

can you open up a python environment on the EC2 machine and run through line by line? How can you tell that is the line not working?

[labdel]

That is the line that kicks off in the traceback when I interrupt hte execution after a while:

File "teslaauthentication.py", line 60, in login
resp = session.get("https://auth.tesla.com/oauth2/v3/authorize", headers=headers, params=params)
File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 543, in get
return self.request('GET', url, **kwargs)
File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
httplib_response = self._make_request(
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 426, in _make_request
six.raise_from(e, None)
File "", line 3, in raise_from
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 421, in _make_request
httplib_response = conn.getresponse()
File "/usr/local/lib/python3.8/http/client.py", line 1322, in getresponse
response.begin()
File "/usr/local/lib/python3.8/http/client.py", line 303, in begin
version, status, reason = self._read_status()
File "/usr/local/lib/python3.8/http/client.py", line 264, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/local/lib/python3.8/socket.py", line 669, in readinto
return self._sock.recv_into(b)
File "/usr/local/lib/python3.8/ssl.py", line 1241, in recv_into
return self.read(nbytes, buffer)
File "/usr/local/lib/python3.8/ssl.py", line 1099, in read
return self._sslobj.read(len, buffer)
KeyboardInterrupt

[skimike02]

No idea. I would recommend trying the code in sections on the EC2 instance. and see if you get a response at all, and if so, what you get. You could also try just using requests to make sure you can get a response from the tesla url without any of the rest of the script, in case it is something about the network config of the instance.

[alexmaple]

This sounds very similar to the problem I am having - script also works locally but not on EC2. I also spun up a Digital Ocean server to check and it doesn't work there either. Did you get anywhere with this?

[labdel]

I worked with a Network admin who helpt to re-route the traffic so it is not coming from AWS. It worked so far. It seems that they are banninc traffic from AWS for the authentication.

[labdel]

Hi,
In this part of the code:

#Send email and pw
for i in range(MAX_ATTEMPTS):
r=s.post(authorize_url, headers=headers, data=data, params=params, allow_redirects=False)
if r.ok and "<title>" in r.text:
print(f"POST authorization form success after {i + 1} attempt(s).")
break
time.sleep(3)
else:
raise ValueError(f"POST authorization form unsuccessful after {MAX_ATTEMPTS} attempts.")

if r.ok and "<title>" in r.text: is True but later in

code = parse_qs(r.headers["location"])["https://auth.tesla.com/void/callback?code"]

I am not getting any 'location' in header.

Any suggestion?

Thank you.

[skimike02]

yeah. they changed what user agent they accept. delete the line: "User-Agent": UA, from the header definition and it should all start working again.

[Pyromancer579]

@skimike02 Thank you, that did the trick nicely.

[Pyromancer579]

@skimike02 Has Tesla changed something in their authentication systems again?

The python code I'm using worked fine up til and through 7th July 2021, but ceased working on the 8th!

[skimike02]

It looks like they added a captcha. Luckily I am using refresh tokens. I'm not sure if there is a way to bypass the captcha somehow...

[skimike02]

There is a discussion thread here: #390
I also modified my code here: https://github.com/skimike02/TeslaSolarData (you just need oauth.py) to download the captcha image and let you type it in.