Objective:

This README file showcases a guide for a Stretch robot utilizing Amazon Kinesis Video Streams, and Amazon Kinesis Video Streams with WebRTC. The robot's image ROS-topic is converted to an RTSP stream using the ROS-RTSP ROS-package. This work is based off Amazon's AWS KVS workshop.

Setup

Have an Ubuntu 18.04 machine with ROS Melodic installed and AWS credential setup as env variables on the local machine. Specifically

• AWS_REGION
• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_SESSION_TOKEN (optional)

Create a new IAM user

Create a new IAM user for your Stretch robot.

1. Sign in to the AWS Management Console and open the IAM console at https://console.aws.amazon.com/iam/

2. In the navigation panel, choose Users and then choose Add users.

3. Type Stretch followed by its id number for the user name. This is the sign-in name for AWS and will help keep track of your device in the AWS console.

4. Select Programmatic access to create an access key for the stretch robot.

5. Click on Next:Permissions

6. On the Set permissions page, select the Attach existing policies directly tab. In the search toolbar, type kinesis to see the list of policies that is relevant to KVS. Select the policy name, AmazonKinesisVideoStreamsFullAccess.

7. Click on Next:Tags. Currently, there is no need to attach a tag to this policy.

8. Click on Next:Review to see all of the choices you made up to this point. When you are ready to proceed, choose Create user.

9. Once the IAM user is added, the following page will provide you the robot's Access Key ID and Secret access key. To view the users' access keys (access key IDs and secret access keys), choose Show next to each password and access key that you want to see. To save the access keys, choose Download .csv and then save the file to a safe location.

IMPORTANT This is your only opportunity to view or download the secret access keys, and you must provide this information to your users before they can use the AWS API. Save the user's new access key ID and secret access key in a safe and secure place. You will not have access to the secret keys again after this step.

For further information about creating an IAM user in your AWS account, please check out Creating IAM users.

Application setup

Clone this repository provided in this workshop onto the development environment, preferably under /home/hello-robot/catkin_ws/src/

cd /home/hello-robot/catkin_ws/src
git clone https://github.com/hello-robot/stretch_aws_robotics_video_streaming.git

Run the following command to install corresponding libararies, software and the robot and kvs applications. This may take up to 15 minutes.

cd stretch_aws_robotics_video_streaming/user_scripts
sudo bash setup_with_sudo.bash
sudo bash setup_as_user.bash

Once the above command is completed, change the sample application code present in the sample application source code present in your environment. You can open the file with the following commands if you used the default setup.

cd /home/hello-robot/catkin_ws/src/stretch_aws_robotics_video_streaming/amazon-kinesis-video-streams-webrtc-sdk-c/samples/
gedit kvsWebRTCClientMasterGstreamerSample.c

The code change is shown in the before and after pictures below

You can find the file with the code to copy here , with the current code as

                pipeline = gst_parse_launch(
                        "rtspsrc location=rtsp://0.0.0.0:8554/back short-header=TRUE ! rtph264depay ! "
                        "video/x-h264,stream-format=byte-stream,alignment=au,profile=baseline ! "
                        "appsink sync=TRUE emit-signals=TRUE name=appsink-video",
                        &error);

Update ROS RTSP Configuration File

Edit the stream_setyp.yaml file to the appropriate camera topic that will be broadcasted.

cd /home/hello-robot/catkin_ws/src/stretch_aws_robotics_video_streaming/deps/ros_rtsp/config
gedit stream_setup.yaml

Simply change the source to match Stretch's camera plugin name, /camera/color/image_raw_upright_view.

Launch Applications

Launch ROS Application

Open a new terminal and run the following launch file.

roslaunch stretch_deep_perception stretch_detect_faces.launch

While the detect faces launch file is running, in another terminal run the following launch file command.

roslaunch stretch_core upright_camera_view.launch

This should setup an rviz setup like the image below. If the camera feed in the bottom left corner in the rviz window is not shown, simply click on the Add button and include the Camera to the display. You then select the same Image Topic you defined in RTSP configuration file.

Then run the following command in another terminal to provide a real-time video feed of the Stretch robot's camera.

roslaunch ros_rtsp rtsp_streams.launch

Troubleshooting

A common error that arises running the ros_streams.launch file is that there is a failure to load the nodelet because the library coressponding to plugin image2rtsp/Image2RTSPNodlet can not be found. If this is the case, then the likely cause of this error is that the gstreamer libraries used by kvs and ros-rtsp were not properly installed. Simply type in the following commands

sudo apt-get install -y libgstreamer-plugins-base1.0-dev libgstreamer-plugins-good1.0-dev libgstreamer-plugins-bad1.0-dev libgstrtspserver-1.0-dev

sudo apt-get install -y libssl-dev libcurl4-openssl-dev liblog4cplus-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev gstreamer1.0-plugins-base-apps gstreamer1.0-plugins-bad gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly gstreamer1.0-tools

Then catkin_make your workspace. This should help fix the previous error in the ros_streams.launch file.

cd /home/hello-robot/catkin_ws
catkin_make

Set AWS credentials

Include your aws credentials (access key, secret access key, and region) to the creds_from_default_file_stretch() function in the utility_bash_functions file. Make sure not to add a space between the = sign and aws credentials.

cd /home/hello-robot/catkin_ws/src/stretch_aws_robotics_video_streaming/user_scripts
gedit utility_bash_functions

Because the utility_bash_functions file was modified, you need to reload your .bashrc configuration with the following command.

source ~/.bashrc

Then setup your credentials by running the following command.

creds_from_default_file_stretch

To make sure the credentials are set use the following IN THE SAME TERMINAL WHERE YOU SETUP CREDENTIALS.

aws sts get-caller-identity

The IAM user account information should be displayed in the terminal.

Launch Webrtc application

Once the credentials are setup, you can setup the webrtc application by running the following IN THE SAME TERMINAL WHERE YOU SETUP CREDENTIALS

# build code again since we modified it with new gst-pipeline

echo "building webrtc code"
APP_DIRECTORY=/home/hello-robot/catkin_ws/src/stretch_aws_robotics_video_streaming
mkdir $APP_DIRECTORY/amazon-kinesis-video-streams-webrtc-sdk-c/build
cd $APP_DIRECTORY/amazon-kinesis-video-streams-webrtc-sdk-c/build
cmake ..
make

# Set environment variable for log level
export AWS_KVS_LOG_LEVEL=3

# move to location of built code
echo "launching webrtc app"
cd $APP_DIRECTORY/amazon-kinesis-video-streams-webrtc-sdk-c/build/samples

# Create signalling channel
echo "create signalling channel, in case it doesnt exist"
aws kinesisvideo create-signaling-channel --channel-name stretch_webrtc_stream
sleep 3

# Launch the application
./kvsWebrtcClientMasterGstSample stretch_webrtc_stream

This will create a KVS webrtc connection between the robot and your browser. You can view it on the Console page by selecting the corresponding signaling channel clicking on Media playback viewer.

Launch KVS video stream application

Setup the KVS stream from the same terminal from which you have the credentials using the following commands

# set env vars to recognize the kvs plugin
APP_DIRECTORY=/home/hello-robot/catkin_ws/src/stretch_aws_robotics_video_streaming
export GST_PLUGIN_PATH=$GST_PLUGIN_PATH:$APP_DIRECTORY/amazon-kinesis-video-streams-producer-sdk-cpp/build
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$APP_DIRECTORY/amazon-kinesis-video-streams-producer-sdk-cpp/open-source/local/lib

export AWS_KVS_LOG_LEVEL=3

# Make create stream api everytime for simplicity. It just fails if it already exists.
echo "Creating a new stream if it doesnt exist"
aws kinesisvideo create-stream --stream-name "stretch_kvs_stream" --data-retention-in-hours "120"
sleep 3
echo "setting up kvs streaming"
gst-launch-1.0 -v rtspsrc location=rtsp://0.0.0.0:8554/back drop-on-latency=true use-pipeline-clock=true do-retransmission=false latency=0 ! rtph264depay ! h264parse ! kvssink stream-name="stretch_kvs_stream" storage-size=512 aws-region=$AWS_REGION

You can then view the current and historical data that is streamed from the robot on the AWS console by selecting the corresponding video stream on the kvs page, as shown in the gif below.

All the commands for launching applications are available from this file

Clean up

You can kill the processes by running Ctrl+C on all the tabs. The Webrtc process does not die untill the ROS application is killed.