Baja - Part 2

Controlling a Robot

March 10, 2019

This is part 2 of the 3 part series about building a robot.

In the previous post, I hacked an RC car chassis to create a little robot. This post is to introduce the code I wrote to remote control the robot over wifi and 4g.

The robot control code has resulted in a Python library, RTCBot, which makes it super simple to use WebRTC to stream the robot’s video from an onboard webcam to the browser, and send back controls from the keyboard or an xbox controller.

The library is well-documented, with extensive tutorials guiding you in setting up the communications of your robot:

See the Tutorials

Example

This example code allows you to view the robot’s camera in the browser, and use an Xbox gamepad to control a robot. This code runs over WebRTC, so can be modified to work even when the robot is not on your local network.

The robot code is:

from aiohttp import web
routes = web.RouteTableDef()

from rtcbot import RTCConnection, CVCamera, getRTCBotJS
cam = CVCamera()

conn = RTCConnection()
conn.video.putSubscription(cam)

@conn.subscribe
def onMessage(m):
    print(m)

# Serve the RTCBot javascript library at /rtcbot.js
@routes.get("/rtcbot.js")
async def rtcbotjs(request):
    return web.Response(content_type="application/javascript", text=getRTCBotJS())

# This sets up the connection
@routes.post("/connect")
async def connect(request):
    clientOffer = await request.json()
    serverResponse = await conn.getLocalDescription(clientOffer)
    return web.json_response(serverResponse)

@routes.get("/")
async def index(request):
    with open("index.html", "r") as f:
        return web.Response(content_type="text/html", text=f.read())

async def cleanup(app):
    await conn.close()

app = web.Application()
app.add_routes(routes)
app.on_shutdown.append(cleanup)
web.run_app(app)

Browser code:

<html>
  <head>
    <title>RTCBot: Remote Control</title>
    <script src="/rtcbot.js"></script>
  </head>
  <body style="text-align: center;padding-top: 30px;">
    <video autoplay playsinline></video> <audio autoplay></audio>
    <p>
      Open the browser's developer tools to see console messages (CTRL+SHIFT+C)
    </p>
    <script>
      var conn = new rtcbot.RTCConnection();
      conn.video.subscribe(function(stream) {
        document.querySelector("video").srcObject = stream;
      });

      var gp = new rtcbot.Gamepad();

      async function connect() {
        let offer = await conn.getLocalDescription();

        // POST the information to /connect
        let response = await fetch("/connect", {
          method: "POST",
          cache: "no-cache",
          body: JSON.stringify(offer)
        });

        await conn.setRemoteDescription(await response.json());

        gp.subscribe(conn.put_nowait);

        console.log("Ready!");
      }
      connect();
    </script>
  </body>
</html>
See the Tutorials