Vehicle.commands.download() timeout when called multiple times on a telemetry link brokered by MAVProxy

[tribbloid]

When calling Vehicle.commands.download() and Vehicle.commands.wait_ready() repeatedly through MAVProxy, the wait_ready() always fail after a number of trials:

  File "/usr/local/lib/python2.7/dist-packages/dronekit/__init__.py", line 2650, in wait_ready
    return self._vehicle.wait_ready('commands', **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/dronekit/__init__.py", line 2182, in wait_ready
    timeout)
dronekit.APIException: wait_ready experienced a timeout after 30 seconds.

The above error is guaranteed to be thrown when running the following integration test:

https://gist.github.com/tribbloid/13d1bb022c0b681a9ee4a65f1067376e

I don't know if its a DK issue or MAVProxy issue. So I'll just submit a failed test to both projects. Please resolve it if DK is not the culprit.

[custeve]

I can verify that I am having this same problem. If after calling cmds.download() in DroneKit, I go to MavProxy and see "No waypoint load started" as a response to that requestion, then I issue a mp command "wp list" DroneKit unblocks, receives the download and continues. This isn't a problem with a direct connection or SITL.

This is causing me to be super stuck at the moment with no workaround.

[DarkmatterVale]

I get this issue as well. In addition, sometimes calling connect('connection_string', wait_ready=True) causes this issue as well.

@peterbarker Any help here would be appreciated. I am able to connect to Mission Planner from MAVProxy, but I cannot run a drone kit connection. I can provide any information you need for debugging the issue.

Thank you for the help!

[custeve]

Some additional information that we discovered in our application:

On SITL or direct serial connection, this wasn't a problem. On a radio connection, this was a huge problem. We also found that if we had a ground station (Mission Planner) using the same connection as DroneKit, we could get it to work. It was assumed that something in the exchange of data from Mission Planner through DroneKit was satisfying Wait Ready statements. Also, while the documentation for Vehicle.commands.wait_ready and Vehicle.wait_ready implies that the first just waits for a mission to be downloaded, it appears they both hang in the identical way. Also, the docs suggest doing a Vehicle.commands.wait_ready after a vehicle.commands.download, however a review of the code on GitHub suggests that the wait_ready has been added to the downloads function, and doesn't need to be added again.

[DarkmatterVale]

@custeve If you don't mind elaborating, how did you connect DroneKit and Mission Planner using the same connection? I can try to do this tomorrow to see if it fixes the problem on my end as well.

[custeve]

Run through MavProxy. Connect mavproxy with one input and two outputs, something like:

mavproxy.py --master={path to your radio, like /dev/ttyUSB0} --out=udpbcast:{ip of your mission planner pc}:14550 --out=udp:0.0.0.0:14551

Then, in DroneKit, connect to 0.0.0.0:14551 and in mission planner connect to 14550. Depending on your network and machine setup, you may have to experiment with your udp ins and udp outs. See the mavproxy docs for me.

[DarkmatterVale]

Interestingly, I was not able to get a UDP broadcast to work. I have been using Mission Planner and my script using TCP ports without any issues, however.

Each of my tests included the following setup:

1. Run MAVProxy, connect to the UAV
2. Connect Mission Planner to the MAVProxy instance running
3. Connect my script to the MAVProxy instance running

When I was ran my script in Python 3 (version 3.6.2), setting wait_ready=False in connect() caused none of the parameters to load. Although it fixed the timeout error, it broke the program since none of the parameters would load. Similarly frustrating, I could not get the parameters to load using wait_ready=True. Trying to run wait_ready=True caused a timeout after 30 seconds. Once I switched my program to Python 2, however, setting wait_ready=False successfully loaded the parameters. I was able to connect to my UAV, and use Mission Planner at the same time as my script. I will try setting wait_ready=True in Python 2 tomorrow, and I will add another comment with my testing results.

@custeve Are you running Python 2 or Python 3?

[DarkmatterVale]

Using Python 2 fixed this issue. Once I switched back to Python 2 and added the wait_ready=True to the waypoint download command as well as the connect command, nothing timed out. It appears there is a bug somewhere in the Python 3 implementation that causes DroneKit to time out.

[DarkmatterVale]

I still get this error with my UAV if I use Python 3.

Will this be fixed soon? It is forcing me to continue to use Python 2.

[tribbloid]

I only test it on Python 2 and got the error, DK's support for Python 3 is still beta (though it should be supported and Python 2 should be dropped).

[hamishwillee]

@DarkmatterVale DK isn't getting any great resource. However if "someone" puts in a PR to fix this we'll certainly review it.

[pietrodn]

I am still seeing this error intermittently on the SITL tests in Python 3.

[pietrodn]

I think I figured it out. mavlink.py silently swallowed an exception, deep down in the mavlink message encoding methods.
The exception was about struct.pack complaining about a str argument which should have been bytes (in Python 3, the semantics of str are different).
This fix appears to solve the problem.

[tizianofiorenzani]

Assuming that your connection string is ok, sometimes you just need to extend the timeout period to get rid of the issue. Here is what I do:

1. vehicle = connect(..., wait_ready=None) --> or do not set wait_ready
2. vehicle.wait_ready(True, timeout=<timeout_in_seconds>)

This will connect with the vehicle and then wait until all the main messages have been received up to timeout_in_seconds

[Guohao91]

I set timeout=180, and then still
Traceback (most recent call last):
File "mission_basic.py", line 38, in
vehicle.wait_ready(True, timeout=180)
File "/home/clickmox/.local/lib/python2.7/site-packages/dronekit/init.py", line 2375, in wait_ready
timeout)
dronekit.TimeoutError: wait_ready experienced a timeout after 180 seconds.
is that because possibly my pymavlink is the version of 2.4.1?

[eminakgun]

Having same problem in Python 2 with continuous wait_ready calls when waiting home_location to arrive to DK. I have serial connection through pixhawk cube. All modules up to date.

[DoneMan]

anyone solved this? im having the same issue

[eminakgun]

I tried to re-request the next waypoint when timeout occured and worked fine. Dronekit does not implement MAVLink's mission protocol properly. I will create a PR but I don't think it will be merged.

[nick-schafhauser]

@eminakgun could you please elaborate on what you meant by "re-request the next waypoint"?
I experienced this same issue yesterday and am looking into workarounds as well

[eminakgun]

#1039 is the implementation here.