Elgato Stream Deck

Has anyone tried using an Elgato Stream Deck for robot control? Not driving, but to let the co-driver activate robot functions or settings. It looks like a nice system, but implementation doesn’t seem to be straightforward.

Any ideas or experience with something similar?

I think this could be a really nice solution to build into a drivers station/board, I will definitely be looking into this in the up coming months.

1 Like

Hmmm… this sounds interesting. A lot of buttons and a screen you could possibly hook a camera up to.

2 Likes

I like the idea of mapping all the buttons/their states so the coach can easily watch what the driver/operator are doing…or thinks they’re doing.

For driver station, you’re looking at two things that rule out the Stream Deck in my book:

  1. The driver station software is looking for things that present as an HID gamepad/joystick/etc. Nothing in their documentation indicates the Stream Deck can do that.
  2. A small deck designed to live on the table with a tightly-packed set of buttons feels challenging to get the muscle memory on in the pressure of the moment.

For $150, I’d much sooner solder pushbuttons or toggles to a Raspberry Pi Pico or Adafruit QT Py, throw CircuitPython on it, and make it present as an HID device. Heck, you can even throw a little money at the problem and get a no-soldering solution.

3 Likes

It should be pretty easy to connect it to network tables to get/set the button data

I picture it used for for diagnostics, display for the coach, and overrides.

I own a Stream Deck, and I’m happy to answer any questions. I can bounce off of this some:

  1. Yep, it’s a proprietary device. It does not show up as a HID device, or register as keyboard or anything. It doesn’t register as an output source/monitor/screen either (so you can’t tie a camera onto it for viewing)
  2. For muscle memory, it’s not bad. There’s a bump on the middle center button (like there are ok keyboards F and J keys) to help you locate things. Also at 5 buttons on 3 rows (for the middle-sized SD), it’s not hard to really go off muscle memory. The fact they are buttons with labeled screens helps.

With the software installed, you can program actions into them to work for live streams (what it’s designed for) or for system-related things (what there’s a lot of plugins for). Like I keep some common programs I run on it to launch quickly, you see CPU usage, paste text somewhere, etc. While you probably could use it to do robot actions, I imagine writing code to interface with FRC software might not be worth the effort of using other solutions like arcade buttons and an Arduino that pretends to be a gamepad or something. Probably cheaper to go that route too.

3 Likes

There’s a project at work that uses one to control a prototype robotic arm in the testbed. AFAIK, it has a Python API, so they’ve scripted up some buttons to send canned sequences of commands to the spacecraft: “stop motion”, “stow the arm”, “take a picture”, etc. It seems to work well enough.

As mentioned above, it’s not a HID device, so not quite plug-and-play, but it wouldn’t be too hard to integrate it with Network Tables or something.

1 Like

Not being HID actually makes it easier IMO, since you don’t have to worry about fitting into the DS protocol, and can send the data however you see fit. That likely would be easier, and is something that makes using custom controllers with tons of buttons harder normally.

1 Like

I would think that sometime like the Stream Deck could be used for a lot of useful override/uncommonly used buttons. For example, with everyone switching to swerve, if your gyro gets messed up, you need to zero your gyro which could be placed as a button on the Stream Deck with a fancy icon instead of your controllers or SmartDashboard.

1 Like

I have a stream deck and the one issue I see with this is I find the software to have pretty high latency, but if you write your own low-latency program for it, it should work well. Very interesting idea.

There is no official SDK support for python plugins, but you can use this python library to control the stream deck directly, and then use pynetworktables or something like vgamepad to to emulate a controller in order to communicate with the robot.

While this would be pretty cool, I would be hesitant to use this for anything critical. Having to run your own script for it to work just adds a lot of risk. If you forget to run the script or find a new bug in finals match 3 you’re not going to have a good time. Displaying some robot information like a secondary dashboard could be pretty nice and not be a huge deal if you go without it for a match.

1 Like