QT Robot

Contents:

QT Basics

Computers and sensors

Head computer

QT’s head computer is a Raspberry Pi. Things to note:

  • Has access to speakers and a microphone (which is right next to a fan)
  • Has access to the screen on QT’s head
  • Has gesture files
  • Hosts the ROS core
  • Turns off and on the body computer
  • Internet isn’t very reliably

Body computer

QT’s body computer is an Intel NUC. Things to note:

  • This is a powerful computer
  • Internet is reliable

Turning QT on and off

To turn on QT, just plug in power to QT and it will boot.

To turn off QT, there are two options:

  1. Press the button on the backside of QT, near its feet.
  2. Login to the head computer (see below) and do sudo shutdown.

If you do sudo shutdown on the body computer, you only turn off the body computer—the head computer is still on.

Note

If you unplug QT to restart QT, it may mess up the boot timing of the two computers. Probably one of them takes longer because it boots in recovery mode. This screws up how the head and body computer network. You will not be able to connect to the head computer from the body computer. If this occurs, simple restart QT by pushing the button on its backside.

Accessing QT’s body computer

When you connect a monitor to QT and turn QT on, you will start on QT’s body computer.

Accessing QT’s head computer

To setup the head, you must Secure-SHell into it (SSH) from QT’s body computer. To do this

  1. Turn on QT.

  2. Open a terminal.

  3. Type the following and hit return:

    ssh qtrobot@192.168.100.1

Setting up QT

What you’ll need

  • A monitor
  • A mouse and keyboard
  • Wireless internet

QT has two ports: one USB-C and one USB-A. You’ll need to use the USB-C port for display and figure out how to control a mouse and keyboard. I suggest a USB-C hub that has a display port that you can use (USB-C or HDMI, for example) and has USB-A ports for your mouse and keyboard. Alternatively, you can use a USB-A hub to connect your mouse and keyboard to QT.

Note

If you have trouble using your mouse or keyboard through a USB-C port, try flipping the USB-C input going into QT. In theory, USB-C should go both ways, but in practice, sometimes not.

Head Computer Setup

Turning off the default face

  1. If you haven’t already, SSH into QT’s head computer:

    ssh qtrobot@192.168.100.1

  2. Update QT:

    cd ~/robot/packages/deb
git pull
sudo dpkg -i ros-kinetic-qt-robot-interface_1.1.8-0xenial_armhf.deb

Note

If the git pull step fails, the head computer might be having trouble with it its network. You can check this with ping google.com. If there’s nothing, there is a problem with the network. To fix this, the best think we’ve found is to restart QT: sudo reboot.

  1. Edit a configuration file to turn off QT’s default face:

    1. Open the configuration file:

      sudo nano /opt/ros/kinetic/share/qt_robot_interface/config/qtrobot-interface.yaml

    2. Change the line that says disable_interface: false to disable_interface: true

    3. Save and exit nano by hitting Ctrl+x, then typing ‘y’, and then hitting Enter twice to confirm things.

Note

You can reboot to see these changes take effect, or continue on and we’ll reboot eventually.

Setting up our code

  1. Secure-Shell (SSH) into QT’s head computer:

    ssh qtrobot@192.168.100.1

  2. Install our project’s dependencies:

    git clone https://github.com/robotpt/qt-robot.git
bash ~/qt-robot/scripts/pi_setup.bash

  3. Increase the swap size, so we’re able to build without running out of virtual memory:

    1. Turn off your swap memory:

      sudo /sbin/dphys-swapfile swapoff

    2. Open your swap configuration file:

      sudo nano /etc/dphys-swapfile

    3. Set CONF_SWAPFACTOR to 2 by changing the line that says #CONF_SWAPFACTOR=2 to CONF_SWAPFACTOR=2, that is by deleting the # character to uncomment the line.

    4. Save and exit nano by hitting Ctrl+x, then typing ‘y’, and then hitting Enter twice to confirm things.

    5. Turn the swap file back on:

      sudo /sbin/dphys-swapfile swapon

  4. Clone our repositories and build them:

    1. Go to the source code directory in the catkin workspace:

      cd ~/catkin_ws/src

    2. Clone our repositories:

      git clone https://github.com/robotpt/cordial
git clone https://github.com/robotpt/qt-robot

    3. Build our workspace:

      cd ~/catkin_ws
catkin_make

    Note

    It takes around five minutes for this command to finish. You can setup QT’s body computer at the same time as it runs, if you like.

  5. Setup our code to run when QT’s head computer turns on.

    1. Copy the autostart script into the correct directory:

      roscp qt_robot_pi start_usc.sh /home/qtrobot/robot/autostart/

    2. Enable the autostart script:

      1. Open a webbrowser on QT (e.g., Firefox) and go to http://192.168.100.1:8080/.
      _images/qt_menu.png

      QT’s configuration menu.

      1. Click ‘Autostart’. You’ll be prompted for a username and password. Enter qtrobot for both.
      2. Click the ‘Active’ checkbox next to start_usc.sh.
      _images/autostart_checked.png

      QT’s autostart menu with our script, start_usc.sh, checked.

      1. Click ‘Save’ and then ‘Return’ twice.

Note

You can reboot to see these changes take effect, or continue on and we’ll reboot eventually.

If you’d like, you can confirm that things are running after a reboot by opening a terminal and running the following command. You should see both /sound_listener and /start_face_server:

rosnode list | grep "/\(sound_listener\|start_face_server\)"
_images/head_nodes_running.png

What you should see if the head nodes are running correctly.

Body

Getting your Amazon Web Service credentials

For QT to speak, we use Amazon Polly, which requires an Amazon Web Services account. At our current usage, using Amazon Polly is free up to a certain level), but you will need a credit card to create an account.

  1. Create an Amazon Web Services account.
  2. Once you sign in, in the top right of the page, click your account name (mine says “Audrow”), then in the drop-down menu click “My Security Credentials,” then click “Create New Access Key.”
  3. Record your access key and keep it somewhere safe. You can do this by downloading this or just viewing it and copy-pasting it to somewhere for later reference.

Note

It is best practice to create separate accounts with less access than your root account and use those access keys, see Amazon’s security best practices.

Setting up our interaction in a Docker container

  1. Change your system timezone to be in your current timezone. To do this, you can click the time in the upper-right of the desktop on QT and then click ‘Time & Date settings…’

  2. Open a terminal and clone this repository onto QT’s body computer:

    git clone https://github.com/robotpt/qt-robot.git

  3. Run a script to allow for updates:

    sudo bash ~/qt-robot/scripts/nuc_setup.bash

Warning

If this step fails, try the following commands before rerunning:

sudo apt install --reinstall python3-six
sudo apt install --reinstall python3-chardet

Note

This step takes five minutes or so.

  1. Setup Cordial by following this guide.

Setting up remote access to QT

Get Dataplicity login credentials from Audrow and sign on. Go to the devices tab and then click “+ Add New Device”. Copy or enter this command into a terminal on QT’s body PC and enter QT’s password ‘qtrobot’. After that runs, remote access should be setup. You can confirm this by clicking the added device and confirming that you can explore the file system (e.g., ls /home/qtrobot).

Note

When you login on Dataplicity, you will be signed in with the user dataplicity. You should then switch to the user qtrobot with su qtrobot and then enter the password, ‘qtrobot’.

Troubleshooting

Unable to contact ROS master at 192.168.100.1:11311

This means that the two computers in QT are not talking correctly because they did not boot in the right order. To fix this, restart QT by pressing down QT’s power button for a second or two. QT should slump forward and the face screen should go off. If the face screen doesn’t go off, power QT on and repeat the process. I haven’t ever had to do this more than twice, as by the second time they are synced.

To test if the computers are talking correctly, you should be enter the following command and see a list of outputs. If you get an error, restart QT.

rostopic list

The tablet doesn’t connect

This can be two things in my experience:

  1. The tablet is on the wrong wifi network
  2. The interaction isn’t running

Tablet is connecting to the wrong wifi

It is easiest to check that the tablet is on the correct wifi. When QT turns off, it turns off the wireless network that it is hosting. When this happens, most tablets will auto-join other networks that they have been connected to. What makes this worse is that web browsers often rememeber (cache) websites to make them quicker to load, which makes it look like you’re connecting to the website hosted by QT, but you’re not. To fix this, go into the tablet’s settings and turn off autojoin for all wireless networks except for QT (or just the networks you expect the tablet to see while the interaction is running). Once this is set, you should be able to connect to and prompt the interaction on QT, including after QT restarts (if you’ve changed the container’s restart policy).

QT’s head computer doesn’t have internet

If you can SSH into QT’s head computer, but are unable to clone a repository or make an update, check if you have internet. The following command should show you that messages are being sent and responses are received, it just hangs there, you are not connected to the internet:

ping google.com

At the moment, just restart QT and hopefully the problem will be fixed. LuxAI has recognized this problem and given me steps to fix it but I haven’t tested them.

If you would like to try, here is their email

In our older setup of RPI/NUC network (same as your QT) we had enabled port forwarding (8080 -> 80) on both machine to facilitate accessing the QTrobot wev config. In some cases and time-to-time (also depending on the router) this causes problem for RPI to reach the internet properly.

In this FAQ we explained it how to disable it: https://docs.luxai.com/FAQ/Network/

Some more comments on this issue : https://github.com/luxai-qtrobot/QA/issues/3

However all you need to do (as I imagine this can be the cause of the problem) is to disable the port forwarding on both machines QTPC and QTRP.

For QTRP (RPI), just follow the simple instruction in the FAQ link and comment the corresponding line in :code`start_qt_routes.sh`. Double check that your /etc/network/interfaces on RPI has the following config:

auto lo eth0
iface lo inet loopback
auto eth0
iface eth0 inet static
address 192.168.100.1
netmask 255.255.255.0
gateway 192.168.100.2
dns-nameservers 192.168.100.2 8.8.8.8

Regarding the QTPC, you can completely disable/remove the ‘start_qt_routes.sh’.