Create3 WaLI : Wall follower Looking for Intelligence

[slowrunner]

Jump to latest update

Create3 WaLi BLOG

Terrible Turmoil Totally Too Talkative (in my head)

After 7 years of working with my two GoPiGo3 robots, and that great community, I am way too overly excited to be building a new robot:

Create3 WaLi: Wall follower Looking for Intelligence

There is a heavy feeling of abandonment of my dreams for my current robots, Carl and ROSbot HumbleDave.

I participated in testing the beta (ROS 2 Galactic) Create3 simulation in October 2021 (only the software...was not chosen for the hardware beta), and have been contemplating this major shift for two years.

No robot is perfect, but I'm ready to see what I can accomplish in the next seven years with WaLi.

Plan:

• Phase 1: Raspberry Pi5 with Dockerized ROS 2Humble controlling Create3 - Characterize Create3
  • with an TTS "WaLi voice" (Dave and Carl have espeak-ng voices - want better TTS)
  • Move Oak-D-Lite from Dave to WaLi - for color camera
  • Investigate limits of dead-reckoning navigation
• Phase 2: Learn Behavior Tree Robot Control Architecture
• Phase 3: Characterize 3D to 2D point cloud mapping, localization from Oak-D-Lite depth cams/processing

[brianabouchard]

This sounds awesome! I'm looking forward to seeing the final results :)

[slowrunner]

Power and Data Connection Plan:

• Expect Pi5 will support VSLAM mapping, and VSLAM localization, w/o needing to add a LIDAR.

UPDATED: 2/24 - Oak-D-Lite powered from USB3 Crashing Pi5, added Pololu 5.1v 2A power supply and Oak-Y-Adapter

[slowrunner]

So excited - found a good quality TTS engine( rhasspy/piper ), chose a voice (en_US-arctic-medium) , which makes a good WaLi voice.

Both my current robots use espeak-ng voices, which are tough for my wife to understand. Hopefully this voice will be more understandable.

[slowrunner]

Started "off-board" wali_node to undock when recharged, sit idle, at 20% rotate to face dock, under 15% dock, and repeat.

Findings:
Playtime: 7h40m 3.4W load 26Wh available
Recharge: 1h38m current 1.07A restores "charge: 2.048"

Thus with an estimated "non-moving" power budget:

• Create3 3.4W
• Raspberry Pi 5 (3W idle): 50% 4W 100% 7.2W
• Oak-D-Lite (0.6W idle): 4W
• Audio Speaker: 0.5W
• (no LIDAR in plan)

Total Estimated "Playtime (100% to 14%) minimal movement":

• Idle, Standby: 3h 20m
• 50% processing, depthAI, localization: 2h 15m
• 100% processing, depthAI, localization, navigation, wali_node, and, and, and: 1h 45m

I haven't characterized driving loads yet, so these times are absolute maximums.

[KFW]

Very cool - I look forward to following your progress.
/K

[slowrunner]

My original plan with an $8 WALL-E character felt just not right:

So meet the new Create3-Wali:

At 37g he adds significant payload to the Create3. It will be interesting to see how he affects power needed to drive. Perhaps I should remove some ballast from the cargo bay.

[slowrunner]

"Dress Rehearsal"

Wali's 12v powered USB hub arrived today, so I did a dress rehearsal of all his planned parts and ran a full load Pi5 power test - NO PROBLEM! (Except playtime drops below 2 hours...)

[slowrunner]

I just did a 100% Pi5 load test powered from the Create3 USB-C port (no USB peripherals powered by the Pi5)

NO PROBLEM! Full Speed, Full Load, No Throttling - 12.7W total

Create3: 3.44W
Pi5: 8.46W
Audio Speaker (0.8W idle from 14.4v aux powered hub)
No driving
voltage: 15.54800033569336
temperature: 32.64999771118164
current: -0.8140000104904175
charge: 1.6330000162124634
capacity: 1.9220000505447388
design_capacity: 1.9220000505447388
percentage: 0.8500000238418579

12.7W max

********** Pi5DESK MONITOR ******************************
Saturday 12/16/23
18:15:51 up 18 min, 4 users, load average: 4.90, 3.15, 1.64
temp=69.2'C
frequency(0)=2400008192
throttled=0x0
total used free shared buff/cache available
Mem: 7.9Gi 562Mi 6.5Gi 33Mi 928Mi 7.3Gi
Swap: 99Mi 0B 99Mi

[slowrunner]

The fun continues with testing the IR sensors - took a bunch of readings at a bunch of distances

Created create3_ir_dist.dist_ir_reading() linear interpolation function.

And then modified the Tufts U. (Kate Wujciak) example sub_ir.py to use the distance from ir_intensity value:

Create3 IR sensor:
side_left            intensity:    29 dist:    0.436m
left                 intensity:    43 dist:    0.387m
front_left           intensity:    67 dist:    0.324m
front_center_left    intensity:    74 dist:    0.305m
front_center_right   intensity:    38 dist:    0.400m
front_right          intensity:    47 dist:    0.376m
right                intensity:     6 dist:  -99.000m

The distances are approximate because the sensor returns depend on the surface-reflectance (albedo), but they are in the ballpark for what I want to test - slam-toolbox mapping with resolution of 5cm (the default).

I need to test further because the front_left sensor seems to be returning shorter distances than expected. I built the reading to distance table using the front_center_left (3 deg offset ccw) sensor. I need to see how much variation from sensor to sensor to determine if one function will suffice or will need dedicated function for each sensor.

Next, I think, I have to use the distances to publish a /scan topic for feeding the slam-toolbox mapper.

[slowrunner]

Setback: Bluetooth Gamepad I purchased will not pair with Raspberry Pi5
Progress:

• Wireless USB SNES Gamepad from my robot Dave works with Raspberry Pi5 ("Sorry to steal from you Dave.")
• Major feeling of success! Configured Docker so teleop_twist_joy works with wireless USB gamepad! What a maze to solve, but finally success.
  

UPDATE: $30 Logitech F710 USB Wireless Gamepad works with the dockerized teleop_twist_joy to give fine control. (SNES only gave "go" with "go slow" or "go fast" modifier button)

[slowrunner]

Create3 IR Sensor /scan Topic Investigation

Major feeling of relief - I wanted to see if it is possible to use the Create3 IR "intensity" sensors as "rough" IR range sensors.

There are all kinds of "BUT This" and "But That" reasons against this idea: primarily - Intensity varies with the surface IR reflectivity, and varies with the incidence angle. Also I found that each IR sensor has a significant different reading to distance equation.

So after averaging three measurements for seven sensors at seven distances, I created my create3_ir_dist.dist_ir_reading(sensor,reading) function covering 15mm to 400mm from the bumper.

Next I created an /ir_intensity topic (62 Hz) subscriber that publishes a 20 Hz /scan topic with the seven "rough" distances adjusted to be distance from the Create3 base_link at the approximate sensor angle (e.g. 65.3 degrees as 65 degrees).

As a test, I drove "Create3-Wali" around a complex wall while running rviz2 with a long decay on the /scan and /odom topics.

AND FOR THE SWEET RESULT:

(My wife said "Wali painted a tree." Somehow that wasn't quite what I was hoping for...)

Another Wali Scan Adventure:

[KFW]

great progress!
/K

[slowrunner]

Create3-WaLi Became Truly Autonomous Today

Create3-WaLi became truly autonomous today. When he wakes up he starts being WaLi without my help - a service starts ROS 2 Humble Desktop in a Docker container and executes "ros2 run wali wali_node" - all when the Raspberry Pi 5 boots, which is when the Create3 powers on.

He doesn't use much of his existing intelligence yet, but he knows when to get off his dock (battery > 99%), when to prepare for re-docking (battery < 20%), when (battery < 15%) and how to dock. He also keeps a log of major events like booting, undocking, docking, and "safety shutdowns":

*** Create3-WALI TOTAL LIFE STATISTICS ***
Total Life:  338.88 hrs (since Dec 11, 2023)
Playtimes (Undocked-Docked): 44
Sessions (boot):  17
Average Session:  19.9 hrs
Safety Shutdowns:  3

Choosing the Raspberry Pi 5 is going to enable much more on-board processing, but at the start the fact that I have to run ROS 2 Humble in Docker really complicates the software architecture and configuration needed to start the Wali ROS 2 Humble wali_node. Everything takes a docker command. Want to open three terminals to try out your ROS creation? Run ./term_to_r2hdp.sh in three remote shells:

#!/bin/bash

# Start a term to already running ROS 2 Humble Desktop Plus Docker container named r2hdp
docker ps
echo -e "\nStarting BASH TERMINAL to r2hdp DOCKER container"
echo -e "To exit:  exit (will not terminate container)"
docker exec -it r2hdp bash
echo -e "Exited Docker Terminal"

And while you are in the Docker Terminal, only ROS 2 Humble commands, nano, and basic Ubuntu commands work. I have to exit the container for the PiOS stuff, and then usually forget to get back in for the ROS stuff.

BUT IT IS WORKING! Create3-WaLi is alive (and thinking Wali thoughts from the moment he wakes up).

[KFW]

Wow. Continued great process.
Using Docker sounds like a pain. It's not something I really have experience in. Although years ago when I lived in St. Louis and was part of a robotics club there, I remember there was a guy who ran everything in Docker (including ROS - this was before ROS2 became more common).
/K

[slowrunner]

WaLi Learned To Take A Nap and Wake Up

One of the Raspberry Pi5 new features is an integrated real time clock with battery backup. This allows setting a wake up time in the RTC, then halting the Pi5 processor with the assurance it will power back up when needed to control the Create3. This 3W savings can shorten Create3 recharge times and could save about 30Wh per day... ok so it only saves a kilowatt (US$0.16) a month.

I like to think of my robots as "alive" 24/7, and "playing" when off their dock. Humans are still considered alive when sleeping, so teaching WaLi to take a nap isn't actually inconsistent with being a 24/7 robot.

I tested it. It works. The Pi5 powered down to show 0.000A during the nap, and then booted at the designated end of the nap. Booting the Pi5 launches the ROS 2 Humble docker container and starts the WaLi "ROS 2 brain" (wali_node.py) again.

The Create3 does not have a "wake on alarm" feature, so WaLi cannot shutdown his Create3 while he naps. Since the house is dark at night, I really don't want WaLi wandering around in the dark. I probably will have Wali dock at midnight, nap on the dock till morning, then wake ready to play at 8AM. (That strategy will complicate my "life" logging...)

[KFW]

Can you use time stamps from the RTC for life logging?
/K

[slowrunner]

Actually, I can cheat.. I have a script for putting Wali out for a nap called

$ cmds/nap_pi5_for.sh 
Usage:  ./nap_pi5_for.sh NN.n (NN.n hours) 

Which puts the following in Wali's life.log:

2023-12-26 12:44|[logMaintenance.py.main]** WaLiPi5 is going to nap for 0.1 hours **

so with a little Awk magic to extract and sum the naps,
and bc to add the "totalNaps" to the "totalAwake":

$ more totallife.sh 
#!/bin/bash
#
# totallife.sh    print total hours and sessions of life in life.log
#
# requires bc  (sudo apt-get install bc)
#
echo "(Cleaning life.log first)"
/home/pi/wali_pi5/plib/cleanlifelog.py
echo " "
fn="/home/pi/wali_pi5/logs/life.log"
# fn="/home/pi/wali_pi5/utils/test_life.log"
totalAwake=`(awk -F':' '{sum+=$3}END{print sum;}' $fn)`
totalNaps=`(awk -F'nap for' '{sum+=$2}END{print sum;}' $fn)`
totalLife=`(echo "scale=1; ($totalAwake + $totalNaps)" | bc)`
echo "*** Create3-WALI TOTAL LIFE STATISTICS ***"
echo "Total Awake: " $totalAwake " hrs"
echo "Total Naps:    " $totalNaps " hrs"
echo "Total Life: " $totalLife "hrs (since Dec 11, 2023)"
echo "Playtimes (Undocked-Docked):" `(grep -c " Docking: success " $fn)`
booted=`(grep -c "\- boot \-" $fn)`
echo "Sessions (boot): " `(grep -c "\- boot \-" $fn)`
aveSession=`(echo "scale=1; ($totalLife / $booted)" | bc -l)`
echo "Average Session: " $aveSession "hrs"
safetyShutdowns=`(grep -c "safety shutdown" $fn)`
echo "Safety Shutdowns: " $safetyShutdowns 

I get it all:

*** Create3-WALI TOTAL LIFE STATISTICS ***
Total Awake:  386.48  hrs
Total Naps:     0.1  hrs
Total Life:  386.58 hrs (since Dec 11, 2023)
Playtimes (Undocked-Docked): 53
Sessions (boot):  22
Average Session:  17.5 hrs
Safety Shutdowns:  3

[slowrunner]

Create3 IR Sensor SLAM - First Tries

If I actually had a clue what each of the 52 parameters in the slam.yaml parameter file were for, perhaps I could know what to adjust to get the map to look like the sensor data.

My /ir_intensity to /scan node seems to output a reasonable representation of the walls when I drive Create3-Wali around.

When I first launch my create3_navigation slam.launch.py (turtlebot4_navigation renamed) with my create3_navigation/config/slam.yaml (turtlebot4 LIDAR parms modified for Create3 IR "distance" sensor scan), with Wali facing a wall, the map shows the wall - bouncing around in angle, but there is a wall on the map in front of bot.

As I proceed to drive Wali around the room, more walls enclose a "known to be open" space totally disconnected from the starting wall and known area.

In the back of my mind is the knowledge that mapping with the Oak-D-Lite and RTAB-Map will likely produce a more precise map with more precise localization, but my curiosity about the full potential of the base Create3 keeps me from abandoning this investigation.

(For the "elephant in the room" comparison - This is the same area mapped with a LIDAR on my GoPiGo3 ROSbot HumbleDave - rotated 180):

[slowrunner]

Create3 IR Sensor Mapping Investigation - There is a map!

It makes sense to get a map out of a SLAM, "simultaneous localization and mapping" package, that turning off all localization might leave only the mapping function working. Of course that didn't occur to me until I had attempted to understand the influence of the other 50 parameter values first. Turning off "use_scan_matching" and "do_loop_closure" gives a reasonable map!

• allowing for the no IR sensor return along the black bottom of the dishwasher
• and slam_toolbox does not use the Create3 diameter in computing the occupancy grid.

[slowrunner]

Another SLAM-TOOLBOX with Create3 IR sensors

Went for broke - mapped the entire main area of my home:

Running slam-toolbox with localization turned off (map.create3.yaml)

• same result as launching create3_navigation with config/slam.yaml (slam.create3.yaml)
• Memory Use - used: 1.2G avail: 6.9G of total: 8G
• Processor load: (CPU% are htop reported percent of 1 core of the 4 cores of Pi5)
  • uptime - 5m 0.25 - 0.32 max = 8% of Pi5 for all active processes
  • wali_node: CPU% 0.0 - 0.7
  • teleop_joy: CPU% 1.3
  • ir2scan shows CPU% 6-8
  • slam-toolbox at 0.5res, (use_scan_matching and do_loop_closure both false)
    htop shows CPU% 15-18
  • odometer CPU% 2
  • life log CPU% 0.0
  • docker container with ROS 2 Humble daemon: CPU% 0.7
• /odom
  • off by -0.4,-0.4 at completion of multi-room mapping
  • off by 0.01,0.06 at completion of 15m single room loop
  • heading accuracy appears to be roughly +/- 2 degrees after short or long excursions
  • 60m around main area (6 rooms with several non-orthogonal walls
  • w/carpet transitions and some one-wheel-on-carpet travel
  • approx 6000 degrees rotations
• Battery:
  • 60 meters in 26 minutes of very slow joystick driving decreased playtime by only 7 minutes
  • 3h 8m playtime (100% to 14% battery percentage)

My Conclusions:

• Create3 odometry is really very good, (probably sufficient for navigation without localization)
• Slam-toolbox with Create3 IR Sensors can build a "reasonable, rough map" with localization turned off
• I need to turn the Create3 safety feature off
  • to allow backing out of "under-furniture entrapments"
  • The mortar joints between our floor tiles sometimes trigger the cliff sensor

*** Create3-WALI TOTAL LIFE STATISTICS ***
Total Awake: 442.83 hrs
Total Naps: 33.95 hrs
Total Life: 476.78 hrs (since Dec 11, 2023)
Playtimes (Undocked-Docked): 61
Sessions (boot): 26
Average Session: 18.3 hrs
Safety Shutdowns: 3
Total Travel: 379.4 meters 1244.7 feet

[KFW]

Very cool. Real progress.

[slowrunner]

Floor Plan To Map Server - Dead Reckoning

Learned more about map representation enough to create a rough draft map from my home's floor plan.

I didn't realize how to get both X and Y dimensions to be accurate when I created the floorplan map - have to unlock the aspect ratio and scale in both X and Y dimensions, so my Y dimension is modeled 0.4m high in 12 meters.

Then there needs to be a map server, which localization loves to provide but I can't use localization with the IR intensity sensors, so had to learn how to launch the map_server and get the launch file, the launch config file, and the map file set to the floor plan map.

But then there is that other thing localization provides - the transform from the base_link frame of the robot to the map frame. Without localization, I have to launch the tf2 static_transform_publisher to lock the map to the odom frame.

So with the map being a little off, and no localization possible, I have something to show:

[slowrunner]

Wali Says "I'm Going To The Kitchen", and "I Want To Go Home Now"

Running ROS 2 in Docker really complicates things, especially access to the audio device. Finally managed to get Wali "walking and talking":

• "I'm headed to the kitchen now"
• "That was step 1"
• "On to the breakfast area"
• "And finally to the kitchen"
• "Look at me! Made it all the way to the kitchen by my self."

And not long afterwards:

• "I want to go home now."
• "There has to be a shorter way than this."
• "I'm sort of in the way here. Moving on."
• "Almost there."
• "Home again. I think I'll stay here for a while."

2024-01-06 23:00|[logMaintenance.py.main]** Dead-Reckoning To Kitchen From Undocked (and back) **
2024-01-06 23:02|[logMaintenance.py.main]** Arrived at Kitchen **
2024-01-06 23:03|[logMaintenance.py.main]** Dead-Reckoning Back To Undocked position From Kitchen **
2024-01-06 23:05|[logMaintenance.py.main]** Arrived Back at Undocked Position **

[slowrunner]

Text-To-Speech Service Added To Wali

ROS (1) had a package audio_common that provided a TTS engine and TTS service node, but that package does not appear to have been ported to ROS 2 Humble... and I wanted to use the Piper TTS engine for very natural sounding voices, so I created a /say service.

The say_node service logs all /say requests to wali_pi5/logs/say.log (to be with the life.log and odometer.log) and returns a boolean response.spoken True if spoken (or False if only logged due to "quiet time").

2024-01-09 09:49|hello, my name is wali - spoken: True
2024-01-09 10:20|hello - spoken: True

My other robots have used espeak-ng which allows a volume parameter, but Piper TTS does not have a volume parameter so I have to set volume using the ALSA mixer from the command line - cmds/set_vol_very_low.sh (10%) and cmds/set_vol_normal.sh (80%) but it is now working in Docker and in the PiOS outside of Docker.

ROS 2 service call from CLI: (cmds/call_say_svc.sh "hello")

#!/bin/bash

if [ "$#" -ne 1 ] ;
	then echo 'Usage:  ./call_say_svc.sh "string to speak" '
	exit
fi
ros2 service call /say wali_interfaces/srv/Say "saystring:  '${1}'"

and outside of ROS/Docker: (cmds/say.sh "hello")

#!/bin/bash

if [ "$#" -ne 1 ] ;
	then echo 'Usage:  ./say.sh "string to speak" '
	exit
fi


echo $1 | piper   --model /home/pi/wali_pi5/c3ws/models/piper-tts/en_US-arctic-medium.onnx  --output_raw | aplay -D plughw:2,0 -r 22050 -f S16_LE -t raw - 

[slowrunner]

Adventurous Wandering (Safely)

I set up what I considered a "safe playground" for Wali - no furniture, no black objects (file cabinet, trash can, UPS, chair and table legs), no mirrors, no dead-end traps.

After "exploring" the Create3_coverage action server for "Use only Create3 onboard sensors to safely wander" only to find Wali smashed against a cabinet, I attempted to diagram its complex state machine and transitions written in beautifully modular C++. I discovered it handled lots of special cases such as starting on the dock, and wandering toward the dock, protecting against falling off a cliff, (none of which are applicable to Wali's playground), but did not appear to use the IR intensity sensors to avoid bumping into walls. (At an "unsafe speed" without concern for objects mounted on top of the Create3.)

Since my C++ skills are over 30 years rusty, I set out to write a "simple wander avoiding walls and recessed kick panels". I forgot that while ROS is a very simply described architecture of topics, services, actions implemented in ROS publisher/subscriber nodes connected by a node discovery / message delivery mechanism, writing a ROS node quickly becomes "not a simple program". Over the years I have played with many robot control mechanisms from simple stimulus/response "Braitenburg Vehicles" to "Parallel Processed Blackboard Rule-based", and to trained neural nets, but I usually end up with a "deterministic finite state machine" as my favored controller. (I'm eager to learn the ROS behavior tree package - eventually.)

Leveraging my prior Create3 IR intensity sensor readings to distance estimation function, I created a "safely wander state machine" in Python. When I tried it out, it found the one "black hole" in the playground - a black kick panel under the dishwasher. A little white paper fix made the dishwasher visible again.

My program indeed would now wander safely, but quickly fell into a rut following the same path over and over. By definition a "Finite State Machine" is the epitome of "nothing is random".

With the addition of additional complexity, rotating for randomly chosen additional time in the wander_avoid state, and randomly choosing a slight left or right arc to the "wander_drive (straight)" state, Wali appears to safely explore his playground. I did not attempt to tune the randomness for efficiency, or even measure the area coverage per time.

I let him wander for two hours, and it only decreased his "playtime" by a few minutes - so driving at 0.1 m/s appears to be "nearly free". That was totally unexpected before "wandering" with the Create3.

[KFW]

Great way to experiment.
/K

[slowrunner]

Oak-D-Lite Mounted On Wali - Investigation Begins

The Oak-D-Lite RGB+Depth device is amazing - amazingly capable, amazingly versatile, and amazingly complex to wrap my head around.

My device is from the Kickstarter campaign, and does not have an IMU built in like the current device, so one of the four ROS 2 demo programs does not work. Another of the ROS 2 demo programs, yolov4, does not work, although the non-ROS version of the program does work so I'm guessing the issue will come down to setting one of the million parameters correctly.

My initial test of the ROS 2 wrapped mobilenet-ssd neural net, which has 21 object classes, is quite impressive with very high confidence scores when correctly recognizing people with no false positives seen yet. (I have seen a few false "cat" and "dog" detections - we have no pets but apparently chair legs can fool it.) This demo publishes 30Hz 300x300 pixel RGB /color/image and /color/mobilenet_detections topics, with nearly no load on the Raspberry Pi5.

I've run the stereo depth demo and verified the depth image is available, so I will be ready to try RTABmap with that program soon.

The Luxonis software support for the Oak cameras is impressive, which helps give me confidence in using the Oak-D-Lite camera with Create3-Wali.

[KFW]

very cool
/K

[slowrunner]

The Luxonis software support for the Oak cameras is impressive, which helps give me confidence in using the Oak-D-Lite camera with Create3-Wali.

The first part of that statement is still true. The Luxonis software for the Oak cameras is very impressive, but the second part of that statement about my confidence in using the Oak-D-Lite camera on Wali is deflated.

First, Luxonis removed the pointcloud method from their camera driver code.

Second, The code they removed relied on the internal IMU of the camera which my particular Oak-D-Lite being from the kickstarter campaign does not have.

Third, All the depth to pointcloud nodes I found are ROS not ROS 2.

There has been a background worry that even the Raspberry Pi 5 may not be up to 3D data processing, so seeing that the Oak-D-Lite could output pointcloud2 topics without loading the Raspberry Pi was encouraging. Now seeing that the Pi5 will have to convert the depth data to pointclouds, that worry is again concerning. I'm just not sure my brain can wrap the complexity either.

I did figure out how to run both the Yolo Neural Net and MobileNet as ROS 2 nodes to output detected class_id and position in the image. I will be able to use that for recognizing the presence of a person in Wali's view. Yolo has more class IDs, but not sure recognizing cow, sheep, horse, elephant, etc. are a useful improvement over the basic 20 classes of MobileNet.

All in all, making use of any of the packaged code in ROS is proving to be very challenging.

Wali's latest stats:

*** Create3-WALI TOTAL LIFE STATISTICS ***
Total Awake:  5142.75  hrs
Total Naps:     153.3  hrs
Total Life:  5296.05 hrs (since Dec 11, 2023)
Playtimes (Undocked-Docked): 216
Sessions (boot):  43
Average Session:  119.5 hrs
Safety Shutdowns:  4
Total Travel:  1529.6 meters 5018.3 feet

[slowrunner]

Don't Know How The Future Will Play Out

Don't know if the Create3 is really up for what I have planned for Wali - only the future will tell. As progress, I have figured out how to run the camera with MobileNet (or YOLO v4) object detections. The Create3 is reporting 94% CPU average and 100% CPU max but it survived at least for this test - see for whole story:

[slowrunner]

Apparently trying to power the Oak-D-Lite from the Pi5 USB3 port will not allow more than 50% Pi5 load when the Oak-D-Lite is running full out (Full Res RGB Video, 300x300 pixel MobileNet or YOLOv4, stereo depth at 30-35fps) which draws 3.2W average but can suck 4W (800mW).

When the Pi5 detects it is being run on a 3A capable supply, it tries to limit the USB output power to 600mW 3W. It is possible to "override" that limit in the boot configuration file, and for some Oak-D-Lite configurations and Pi5 loading the camera and Pi5 cooperate nicely, but for reliable "all out" use, I had to add a separate supply for the camera.

Pololu 5.1v 2A step-up/down switching power supply:

I used a "USB C Power Delivery (no data wires) Cable" which maintains the 5.1 voltage acceptably under load (5.14v to 5.09v) to the Oak-Y-Adapter which separates the power and data paths for the Oak-D-Lite camera.

Using this separate supply, my Pi5 survived all tests (depthai examples and depthai-ros driver launch files).

Perhaps now I can begin to study the RTABmap function.

[brianabouchard]

This is really awesome! Thanks for sharing.

[slowrunner]

Create3-WaLI Had A Nightmare

Last night at 3:30 AM my wife awoke me saying "WaLI is in distress."

I could hear a roaring sound and went to investigate. Indeed WaLI was having a nightmare. He had pushed his dock along the wall, and was frantically spinning his wheels trying to dock as commanded by his wali_node.py control program. His light ring was flashing bright red so I knew he was feeling it was urgent he get on that dock.

I lifted him away from the wall which invoked the Create3 "kidnapped" reflexes to stop the wheels spinning, and after restoring the dock to its proper position along the wall, I placed WaLI on the dock to have the green charging LED appear.

Having run to his aid without my eyeglasses, I had some difficulty logging his "rescue." Later in the morning I checked his log to reveal the full nightmare:

2024-02-09 03:10|wali_node.py| ** WaLI Ready To Dock at battery 17% **      <-- WaLI noticed battery below 18%, turned to face dock
2024-02-09 03:11|wali_node.py| ** WaLI Ready To Dock at battery 16% **
2024-02-09 03:12|wali_node.py| ** WaLI Ready To Dock at battery 16% **      
2024-02-09 03:15|wali_node.py| ** Rotate Angle Goal failed with status: 6     <--  WaLI couldn't see the dock and tried to fix problem 
2024-02-09 03:16|wali_node.py| ** WaLi dock goal result - Docking: failed **  <<--  DOCKING FAILED - BAD SITUATION
2024-02-09 03:16|wali_node.py| ** WaLI Ready To Dock at battery 13% **
2024-02-09 03:17|wali_node.py| ** WaLI Ready To Dock at battery 12% **
2024-02-09 03:19|wali_node.py| ** WaLI Ready To Dock at battery 10% **
2024-02-09 03:19|wali_node.py| ** WaLI Ready To Dock at battery 10% **
2024-02-09 03:23|wali_node.py| ** Rotate Angle Goal failed with status: 6    <<--  WaLI couldn't see dock and tried to turn to face it
2024-02-09 03:23|wali_node.py| ** Rotate Angle Goal failed with status: 6
2024-02-09 03:24|wali_node.py| ** WaLI Noticed Docking: success at battery 8% after 2.3 hrs playtime **
2024-02-09 03:27|[logMaintenance.py.main]** Wali got stuck against wall while trying  to dock **

Wali left two piles of rubber and a mark on the baseboard from his nightmare:

Analysis

Nightmare was a result of "Parking a Bicycle On WaLI's Dock"

I was running an overnight test with the Oak-D-Lite camera doing the MobileNet Object Detection and had "given WaLI a bicycle to recognize" while he was on the dock (Wedging the target between WaLI and the Dock to hold it up):

When Wali got off the dock, the bicycle fell onto the dock preventing WaLI from juicing up when he next needed.

Lessons Learned

• Don't do tests overnight
• Don't leave WaLI's toys laying around

[alain-lnpcrd]

Hello Slowrunner
To start, a photo of my create3 with a temporary installation to check that everything works.

I see that I have found a master in computer science and your work can certainly teach me a lot of things.
Besides the create3 with its A1M8 lidar and its OAK D Lite camera there is an RPi4B-8Go in the hold but I also have a Pi5 which I hardly use at the moment.
So please, in a few lines to start, how to install and get started with https://github.com/slowrunner/wali_pi5

[slowrunner]

@alain-lnpcrd Looking good. You should give it a name and start a "Show And Tell" blog for your bot.

have found a master in computer science

No you have found someone in the process of "mastering frustration trying to smooth rifts in the fabric of the robot universe" making progress but not having solved "robotics world peace".

create3 with its A1M8 lidar and its OAK D Lite camera there is an RPi4B-8Go in the hold

You have built a TurtleBot 4 Lite - so:

your work can certainly teach me a lot of things.

Your work can certainly teach me as well

Just yesterday I was reading the turtlebot 4 manual on network configuration with the FastDDS Discovery Server.

As you progress with your Create3, I predict you will need to continue following the TurtleBot 4 Lite path,
and I will learn from you and the Turtlebot 4 community how to workaround the Create3's four processors just barely keeping up with the additional tax running ROS 2.

in a few lines to start, how to install and get started with https://github.com/slowrunner/wali_pi5

I can do you one better than a few lines - one word: "Don't." I am just beginning this misguided, "off the beaten path", adventure with two major impediments:

• Running on Pi5 requires Dockerized ROS 2 Humble ( Or build Humble from sources over Ubuntu 23.10)
• I don't want to put a LIDAR on my robot
  • Primarily to force myself to put my "efforts where my mouth is"
    I have been "preaching" for the last 6 years that we as humans don't need to know the exact distance to the walls in order to not bump into them, and having a sensor that cannot see my black trash bin, my black UPS, my black dining chair legs, my black tower computer, my black filing cabinet, or for that matter anything below the LIDAR plane or above the LIDAR plane, is not going to provide as much information about the environment as a single look with our eyes (or frame from a Pi Camera or Oak-D-Lite camera).
  • Secondarily, the "playtime" running a Raspberry Pi 4 or 5, with the Oak-D-Lite camera (running fulltime), processing the sensor info adds 8W load to the Create3 battery giving a 1.7 hour playtime and a 3.8 hour recharge time. Adding battery load from a LIDAR (running fulltime) would shorten the playtime and requires solving the power delivery / USB back power issue I could not solve. (and would hit the 4-hour Create3 recharge limit - obvious solution shutdown the LIDAR and Camera on the dock, but a complication none-the-less.)

Since you have a LIDAR you don't need my IR Intensity sensors to /scan node.

If you want to make your robot a 24/7 robot with a "life log", you could bring the wali_node.py into your work space and run it. (It is not packaged for distribution, so it is not just copy one file, build and launch - I don't think.) I'm a little surprised no one on the Turtlebot 4 discussion forum has posted a "make your TB4 a 24/7 robot" thread, but I have been underwhelmed by the TB4 "community" period, (and disappointed that ClearPath left nearly everything Oak-D-Lite/Pro to Luxonis).

You could take a look at my wali_pi5/c3ws/cmds/ folder - there are example command lines to do many common things: e.g.

• echo_battery_state.sh
• echo_oak_nn_detections.sh
• launch_yolo4_publisher.sh
• launch_camera.sh (with params/camera.yaml - does full FOV video with mobilenet and stereo depth)
• echo_dock_status.sh
• reset_pose.sh (use when Wali is on dock before starting any mapping or navigation test)
• set_safety_override_backup.sh (sometimes have to run it a few times before Create3 listens to it)

My odometer.py node probably is usable standalone. It logs the distance traveled by your robot whenever it moves.

I can help you to solve specific issues or give my opinion on how to achieve a goal for your bot, but I really don't believe anyone should try to follow my path at this point. I have painfully been making steady progress since getting my robot in December, but the pain is real and continues. I love the Create3, but like every commercial educational robot it has limitations that must be discovered and conquered.

[alain-lnpcrd]

Hello Slowrunner
I'm very busy at the moment with stupid incidents.
For my pseudo turtlebot4 lite, my problem is rplidar which crashes so no frame map (although it works with create3_lidar_slam).
All this to say that I haven't had the time to look into your project although I find it enormously interesting in the means you put in place to create mapping.
Needless to say, I have a big problem which is my age. I'm 75 years old and I'm looking forward to 5 years max at best, so I limit my ambition to copying what I find with adapting. When it comes to learning, I'm drowning in tools that I can't master, such as foxglove rqt rviz gbd... and what's more, what I learn is volatile. My dream is to make a robot that talks to me and cares about me the day I become senile. I think that at my speed it will end up on Amazon.fr. Perhaps the advantage will be that I will have an idea of how it works. But the feeling of having an ax above your head is an obstacle to progress.
I'm bad in English and I hope "chromium" does an understandable translation of my French. Its English to French translation is not bad which is not the case with Firefox which invents a lot of words unknown to dictionaries. And so I am wary of using it to translate French into English.
To return to my original question about the installation of your project. I have the impression that you have automated a lot of things for the RPi5 under bookworm (but maybe I'm just imagining things). So your directory, do you install it under your home folder or in a docker container? Do you then run a shell program that will install everything properly? ...
This is just to start correctly.

[slowrunner]

Wife Demands WaLi Sleep Through The Night

While I am not lacking for topics to investigate with WaLi, one "feature" has just had its priority boosted. Since implementing a "nap for x hours" feature available with having a Raspberry Pi5 real-time-clock (RTC) as WaLi's compute board, I have generally been manually commanding WaLi to nap through the night. I prefer to leave him operating so sometimes I "forget to put him to sleep".

My wife just boosted the case for automating WaLi's sleep schedule. Time to teach WaLi to "watch the clock" and "decide when to go to sleep" He only sleeps the Raspberry Pi5, not the Create3, so WaLi needs to be sure to sleep on the dock.

[slowrunner]

Going off the radar for a bit

[shamlian]

Be well!

[alain-lnpcrd]

Hi @slowrunner
you seem to be discussing the use of nav2 so can you tell me if you can synchronize the create3 and the rapsberrypi because I'm losing my mind.
We offer examples and nothing works as it should due to lack of synchronization of messages.
#572