Neural Network on RIO for Control Panel Color Detection

Identifying the colors on the control panel in this year’s game presented a unique set of challenges. To more reliably solve this classification problem under various conditions, The Zebracorns have employed a neural network running on the RoboRIO.



You may not like it, but this is what peak robot performance looks like.

Well done guys, this brought great joy to me today!


In my past, the issue that I have seen with Neural Networks is conditions where the input sensor data was not captured in the training set. Did you add additional training data to capture unexpected conditions?

1 Like

A monochrome-themed team claims they can detect color, and on April Fool’s day no less :thinking:.

This is a much better read than most of the papers on basic NN’s. I love that the entire data pipeline is described, since that is usually where the majority of real NN work happens.


Can you be more specific about what you mean? We want to try and get an answer but we’re unsure. Are you asking if we captured data for colors that weren’t on the spinning wait cursor?

I am thinking more about times that you might be on the edge of the wheel - perhaps not actually staring at the wheel. Perhaps occasions where lighting produces a slight change in the observed spectrum. If the NN was trained for Yellowish, instead of Yellow. Make sense?

We handle this in a few ways

First off, the data we collected is from a variety of positions and lighting conditions. That includes weird angles, in and out of shadows from the sensors (and students) and so on. That variety of training data should help the network handle a range of inputs and correlate it with the correct sensed color.

The other thing which helps is that the output of the net is a confidence value for each color. 0 meaning perfectly confident it is not a given output color, and 1 being confident it is. We can use thresholds on this confidence to figure out what to do. If the confidence is high for a single output and basically 0 for the rest, we can be comfortable that the net is predicting correctly. If the confidence is low for all colors, or perhaps moderate for a few, we can use that to mean we don’t have a good detection.

Another option which we didn’t explore yet is to collect data when the sensor is pointing at random non-color-wheel colors, and then marking that data with all the output classes set to 0. This would perhaps help train the net to understand what the sensed colors aren’t, if that makes sense. Care would need to be taken to not overwhelm the training data with negative examples like this, but it might help in some cases.

Hopefully that helps?

1 Like

Yes - that final set of training is to sort of thing that is essential for a NN approach!
What I have seen is that NNs are well behaved given that the input signal is contained by the training set. It is the cases where the input signal was not part of the training set that are interesting!

This is wonderful! Thank you for sharing it with us.

1 Like