-
Notifications
You must be signed in to change notification settings - Fork 158
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use a precompiled Portduino version of firmware for python integration testing? #643
Comments
I've thought similarly, so I definitely agree! There's still a SimRadio system built off portduino, used among other things by the simulator (Meshtasticator), as well as the actual linux-native As far as implicitly testing the firmware code, I think that's also good and we might even be able to build this such that it can optionally use real devices (somewhere far away from defaults/the public mesh), which could uncover device-specific bugs, potentially. I don't imagine we could systematize that without a lot of resources (especially given how many devices we support these days), but it'd be a cool option to have. Basically: yes, I like this! |
ooh Meshtasticator looks great (I didn't even know this hotness existed - awesome work @GUVWAF !) and is probably an indirect descendent of that old linux build I was mentioning. awesome! |
I'm game! I think some of the nominal testing that was originally there shook loose from the 1.3/2.0 days when we moved all of the config protos around. Long term, I'd actually like to have real IoT hardware in play for e2e testing. I think ESP32 based targets could work the easiest, since we have esptool access already in python. I wonder if a USB hub with programmatically switchable ports exists, because inevitably when something goes wrong, power cycling could save us some flakey test heartache. |
While I'm not proficient with Python testing, I'm happy to help with some Python glue code to create a basic test scenario of 3 Portduino nodes that talk to each other with a hop in between. I believe with that you can cover a lot of test cases for mesh communication (broadcasts, DMs with ACK, traceroute, etc.) already. |
Hi @ianmcorvidae and @thebentern,
I'm making this issue to discuss an idea. In the 'old days' we had a compiled C++ version of the firmware code that ran on linux using the portduino layer. We used that binary in the CI build of the python toolchain to do integration testing - because we could talk to it like a "real" device (because it was 95% the same). This was handy for integration test purposes.
I guess that somewhere over time that got deprecated/removed.
Now that there is slick codecoverage support that someone added (@ianmcorvidae ?) for python: what do you think about using that again?
The reason I ask is that in cleaning up my power measurement stuff for checkin I realized that the PowerStress stuff needs code coverage testing. I could learn how to use the python mocking framework, but I'm worried that even if I use that framework if I mock out the PowerStress class to pretend to be the real device I'm not really testing that much. Because PowerStress is mostly done on the device side.
Instead: (proposal)
Does this seem plausible? Would you like changes?
My current queue is:
The text was updated successfully, but these errors were encountered: