Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

Per a few requests, and my personal objective to have FRC as a whole have more focus trying new things when it comes to software, I decided to make a generic Neural network code that can be easily adjusted to suit whatever control needs you desire.

How does it work?

1. Simply define your network topology as a vector of integers. For example, <2, 4, 3, 2> will create a network with 2 input nodes, 2 layers of hidden nodes with 4 and 3 nodes respectively, and 2 output nodes.

2. Get your data. I set it up to read data from a file, as you generally train a network off data after the fact instead of in real time. An example of a compatible data file is included in the git linked at the end of this post.

3. Send the data through the neural network until the weights of the nodes converge.

4. ????

5. Profit.

Let's use an example: You have an omni-wheeled robot that wants to center itself in a hallway such that it is the same distance to each wall. You put 2 range finders on each side of the robot to arrange for this.

What should your architecture be? This is a little bit of a tricky problem. You can have either 1 or 2 inputs. Let's stick with the 1 input route. The input would be the difference of the two range finders. The output you care about is the 4 motor values, but the output that the neural network is trying to achieve is a single value: zero. A possible architecture could be <1, 4, 4, 1>. Your second hidden layer is actually your motor values.

So how do you use this computational engine you just created? You can do it two ways, in real time, or gather random data and train it on that. I personally find it much more enjoyable to watch the robot learn to center itself. What you'd need to do is set up your robot anywhere (even in the center of the hallway!). Get the sensor data, feed it forward into your neural network, get your motor values, apply them for a duration of x ms, get the new sensor data (which is the output of the network), then train it on that value. Repeat until your weights converge. Once the network converges, save the state of it (as in the node weights). Now you have a robot that taught itself how to center itself in a hallway.

I'm not going to go under the hood of neural networks as this post is already rather lengthy.

Code: https://github.com/faust1706/Neural-Network

I accidentally pushed built code to the repo, my mistake.

Something really cool that can be done with this is have your battery voltage as an input. The network will generalize to work at any functioning voltage.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

Would it be possible to take a video of the realtime learning in action?

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

I'm don't have access to a simple navigation robot anymore...it got scrapped for parts. I'll see if I can put something together one day in the lab.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

Have you tried learning parameters for a PID controller?

Any thoughts on using recurrent neural networks to add some memory to your controller?

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

The goal of releasing this code was that teams would have an easier time with machine control as well as show them that there is more than one way to do things, and a lot of times the first solution you have (the PID controller given to each team), isn't always the best.

By learning parameters, are you talking about setting them yourself, or a PID controller that finds parameters itself? The latter already exists, but it is more complex than a neural network in my opinion.

RNNs do wonders, but I wanted to keep things simple for an introductory post. This would be a great topic for a team to explore for themselves, as well as Elman Networks and others of the like.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

Saw you mention this on r/FRC, I'm definitely eager to check this out!

EDIT: could you post a more detailed example implementation?

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

Of course. It'd be easier for me to do so if you give me an example that you want me to go through, do you have any in mind?

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

An example of this in actual robot code (e.g. teaching it how to center itself like you mentioned) would help a lot. Also if there's any way I can help contribute to this I would be glad to help.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

I added a case for real time training called realtime. It is not integrated with anything, but it is laid out to be embedded easily. There are plenty of comments which should allow for fast integration of any system, so long as you can give this code the sensor data and send the motor values to wherever they are needed. I feel it'd be easier to add this into whatever that you want to train instead of sending values over serial or something, but that's just me.

The code is a little messy, I apologize. The messy part is the training from a file, I'll work on it when I have more time.

If you need help integrating it into FRC code, don't hesitate to ask me.

Edit* also worked on the readme a bit. Still unsure of how specific I want it, but it's a start.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

It looks very nice, I forked it and I'm trying to make an implementation of a basic neural network for use with encoders.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

This looks really cool, and I'm planning to try to hack on it over April vacation (when I finally have some time). Since our team uses Java, I'm going to try to port it, which will hopefully help me understand the code better.

I have no previous experience with neural networks, so I'll probably have a lot of questions in the future, but for now I was wondering if there is some way to decide how to structure the hidden layers. In your example, you use 4 nodes, one for each wheel and two layers. What is the reason for this?

Thanks.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

Here is my implementation of your library for use with encoders on a robot.

https://github.com/DannyMoses/Neural..._implementaion

It's definitely not very polished(I basically copied your example). Am I doing it right? I don't have a robot to test this on right now so I would value any feedback. I also just want to say that I think this is probably the future of FRC programming right here.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

Extremely long post incoming, but I feel others would benefit from reading this as it is an actual implementation of it.

You are somewhat there.

Here is something that I found immediately:

//clear inputs and results to ensure no lurking data
inputVals.clear();
resultValues.clear();

//get your data, populate it in inputVals
//with inputVals[trainingPass].push_back(int)

// Get new input data and feed it forward:
net.feedForward(inputVals);

can you see what's wrong? Two things actually. One is that your input data vectors are empty (of size zero). The other thing is that you aren't sending what you want to train on (your inputs). By doing the latter you willing fix the first issue of the empty vector.

Another thing that is wrong: your architecture is 1-4-4-1, yet you have 2 target values. This is another dimensional error. This time instead of not having dimensional, you have too many!

Alright, moving on. So you want the robot to move 10 feet forward and 10 feet back, correct? What should be your input? What should your output be?

Let's redefine the problem for an easier first time implementation: let's move x feet forward, with the case of x being 10. You might want to decrease it to maybe 3 so you don't have to go chasing a robot around, just a thought.

So your input can be however many feet you want to move forward. In code that is

inputVals.push_back(10); //feet

what is your targetValue? Two approaches come to mind immediately. The first is your target is to travel 10 feet, so

targetVals.push_back(10); //feet.

There is a lot of room to expand on this. For instance, you could input your robot's position in x and y as two inputs, and your desired position be your target values x and y.

Moving on.

float outputVal = (float) motor_values.back();
myRobot.Drive(outputVal, 0.0f);

the .back member function of a vector calls the last value stored in the vector. What is the drive system of this robot? This assumes you only have one motor controlling your entire robot, or that you are only going to use one value of your motor hidden layer. Either way, it is sub optimal for two reasons. Your motor hidden layer has 4 nodes. If you want to go this route, you need to make you NN architecture account for it. That is, 1-n-1-1, where n is any number of nodes you desire.

Now you wait 50 (ms?). This may not be what you want, either. The neural network will try to find the motor value to travel x feet in 50 ms, and in your case that is 10 feet. You need to your robot time to try to get there based off the values you got from the network, maybe 5 seconds? Be careful though. When you first declare a NN, the weights are random, meaning it has a 50% chance of going backwards, so give the robot some room. I'd suggest doing this in an open area like a gym. A cool thing about this is that you don't have to recent the robot per iteration (unless you are getting close to a wall), because each iteration assume x feet from your current location.

I'm not familiar with FRC code anymore, but you may need to set motor values to 0 after the wait.

Now you get your updated sensor data:

results.push_back(leftEncoder.GetDistance() + rightEncoder.GetDistance() / 2);

What you did was right, however your units might not match up. I don't unit of length .GetDistance() returns. You may need to do a simple conversion.

You set this as your output layer, then you train through backpropagation, which is correct.

Then you wait until your network converges and exit.

Yikes. I just realized I forgot to save the state of the network.....that's rather important. I'll work on that asap.

Now your NN is trained properly to go x feet. But it isn't a general solution just yet. You need to test multiple inputs until each one gives the correct output, then send them all through again to ensure that your network is good.

Lastly, you'd want to save the state of this network (which I also haven't written code for yet) so you can load it and go from there.

When you load in a network to give you proper motor values, you do not need to apply backprop to it as it is already trained how you like it. So the code you need when you're done is:

Net net(NN.txt); //file that contains the state of the good NN
vector<double> targetValues(int x);
vector<double> inputValues(int y);
net.feedForward(inputVals);
vector<double> motor_values;
motor_values = net.getLayerValues(int z);
//rest of code that uses motor values

So great, you now have code that makes the robot go x feed forward in n time. As I mentioned earlier, you can get much more fancy with it. Let's bump it up to swerve drive. Your input could be your x and y position (could be assumed to be 0 0), the data you you get from the network could be heading and motor value, and your result would be your new robot position. Another cool addition would be to use the time you want the robot to complete it as an input. This way each distance is not treated equally in terms of time you give it to travel, which will allow for more robust autonomous routines. Yet another cool addition to the inputs would be battery voltage. This way if you have a bad battery in but trained on a good one, it won't matter because the network will generalize to work at every functioning voltage.

Hope this helped! Let me know if you need more assistance or if something doesn't make sense. There is a decent chance I messed up explaining something. It's been a long day.

Thanks for the last part, but the Poofs already have this beat by a lightyear with their 2014 autonomous.

Implementation note: you may want to only run through the network while training when you push a key, as well as add an emergency stop key stroke.

Adding this to the post as well because others could benefit from reading it.

Deciding the architecture of neural networks is a bit of a dark art. It is something that people just accept. As a side project in the lab, I am testing how different architectures of networks affects performance and rate of convergence (in terms of speed and iterations required). Hopefully I'll have some form of a paper by the end of summer to share with whomever is interested.

The reasoning behind by hidden layer architecture is that I am assuming a nonlinear relationship between the inputs and the proper motor values. If I don't have that second hidden layer then each motor values can be calculated by summing the inputs * weight_ij. With the addition of a hidden layer, it still allows for the there to be a linear relationship between the inputs and motor values, but it is also able to account for if the relationship is say, logarithmic.

I highly recommend to play around with different architectures.

I spoke with a buddy of mine at college and he said he'd be willing to help me include a gui so the user can visually see the network adapting, which would be really cool.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

Thanks I was sure something was off lol. I'm going to work on this more later when I have the time.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

I made a preliminary port of the core neural network code to Java, available here. It includes only the core logic and math, not the training data structure or example code. I have not been able to test it yet, but as far as I know it should behave identically to the C++ code, although there are probably a few bugs.

I now have a much better understanding of how the network works but I am still confused by a couple of things.

The simplest use case I have to test the use of a neural network is to replace the PID controller that controls the heading of the robot. From what I understand, the input data could be the difference in the current heading from the desired heading, and the target data value would be 0 in that case. The output value would be the rate of rotation of the robot and I would train the network by making the robot rotate to different headings. Does this sound right?

Also, is there any technical/mathematical reason why the network cannot use other transfer functions, or have you just not written that part yet? I was wondering because I am thinking about adding that functionality to my port.

This is probably a little irrelevant, but why do you only set the output value for the bias node of the output layer, not all of the layers?

Sorry for all the questions, but I am really interested in this.