21-08-2009 21:48

lynca

Re: paper: Semantic robotics for FRC competitions

Interesting ideas, thanks for sharing.

I'm curious about the Mechanical Turk prototype.

How does human processing integrate with computer processing ?

21-08-2009 23:06

Jared Russell

Re: paper: Semantic robotics for FRC competitions

Thank you for posting this!

This is definitely a novel way of looking at things in FRC. I work in the artificial intelligence field, and one constant source of frustration is when people want to fit the square peg of automation into the round hole of problems that are currently better solved by human reasoning. Yes, without at least trying we won't advance the state of the art in AI, but at the same time people seem to be keen on taking humans out of the loop for no convincing technical reason. Case in point: much current research seeks to make currently teleoperated UAVs fully autonomous. But each UAV needs a ground crew of probably 50 people to stay operational; is eliminating the one pilot really the lowest hanging fruit? Play to the strengths of each part of the team! But I digress...

Back to your paper, I think that some of your points could stand to be fleshed out just a bit more - I'm not sure that you've identified the best possible "square peg" and "round hole" in this case. You claim that pattern recognition tasks are better solved by humans than by computers, but I don't believe this to be true for a large class of problems. For example, in Aim High in 2006 I would argue that auto-targeting bots could outshoot even the best driver's aim. Tracking a known, invariant object has been done effectively using a variety of techniques in computer vision. Now, when you are talking about avoiding other teams' robots that vary widely in form and function, then I would agree with you that humans (currently) do it better.

You also talk about how computers can do a better job of decision making at a tactical level because they are undeterred by things like distractions, emotion, etc. This is a very interesting question that has significant repercussions in the defense community (as soldiers/pilots are exposed to the elements, at what point does their cognitive ability become inferior to a machine's?).

However, I think that there are some issues that will prevent your robot from ever being a more effective strategist than a person. In order to come up with an effective strategy, you need to know the full state of the field at a moment in time. Knowing the locations of goals and robots is NOT the exhaustive list of the elements of this state. You also need to know something about intent - yours, your teammates', and your opponents'. Even if you acknowledge that humans can estimate intent better than a machine, how are they going to convey their estimations to the robot in a reasonable amount of time? And how is the decided-upon strategy going to make it back to your teammates?

My comments here aren't meant to attack your paper; rather, I hope that you can use them to improve it! This is a really radical way of thinking about human-computer interactions in FRC that I don't think is far off from being quite useful in the right situations.

22-08-2009 19:49

nathanww

Re: paper: Semantic robotics for FRC competitions

Mechanical Turk has two redundant general-obstacle-avoidance systems. One is similar to the initial version of this system that was used in the first DARPA challenge. The second sends a camera feed back to a labVIEW dashboard which superimposes a grid on top. A human operator has a keypad, with each button corresponding to a square on the grid, and they push that button whenever an obstacle occupies the square.

The automatic system is designed to have very high specificity and moderate sensitivity--in other words, it is very unlikely to generate false-positives, but during normal use, the human operator will be responsible for detecting most of the obstacles. The automatic system serves as a backup in case the operator is distracted, unclear on instructions, etc.
The result, of course, from both of these systems is an approximate geometric position of all obstacles in the field of view, which the robot can use to modify its route plan.

While this is certainly true(we did it with our 2009 bot) , there are a couple of things:

1. Automatic tracking, even of a known invariant object, takes a significant amount of time for development, calibration, and debugging.
2. Automatic tracking systems are more likely to fail than human vision processing. Case in point:the number of teams that needed to recalibrate cameras for field conditions
3. Automatic and manual tracking systems are not mutally exclusive. For example, a system could be designed that uses automatic,high precision tracking normally, and switches over to operator-supplied data if its confidence rating drops below an arbitrary level.

Do you? I think this is something to investigate further--what I observed in Lunacy at least was that there were a small number of "global variables"(real-time scores, posession of super cells, etc) that affected tactics, but most of the action at any given time seemed to cluster in smaller "cells" of 2-4 robots in a particular area. If it turns out that the amount of field state information is larger/needs to be updated more frequently than is feasible for the operators, it's possible that FIRST could tweak the FMS system to make data that it already collects(i.e. real time scoring) availible to robots, especially if there were many teams interested in implementing this sort of system.

Regarding intent, this is a very tricky issue for AI systems to figure out, because fully understanding intent in a general sense requires a general theory of mind, which is an extremely hard problem. However, I think that in the restricted domain of an FRC game, judging intent is feasible. For example, to build predictive tracking into our Lunacy bot, we generated a Markov model that the robot could use when it did not have a line of sight to the trailer. The model was able to predict the basic physics, but also common evasive strategies. . In addition, if one can find ways to decrease the domain even more(i.e.concentrating on a game "cell" instead of the entire match), the problem becomes easier because the number of basic actions and combinations that can occur is decreased(which lends itself better to an expert system)

Okay, that was a ridiculously long post. But basically, this isn't intended to be a guide on "how to build a semantic robot"--it's really more intended to stimulate people thinking about it, since there are still so many unanswered questions. And I think that ultimately the only way to test some of these things will be to build a bot using them, and enter it in a competition.