Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

Per a few requests, and my personal objective to have FRC as a whole have more focus trying new things when it comes to software, I decided to make a generic Neural network code that can be easily adjusted to suit whatever control needs you desire.

How does it work?

1. Simply define your network topology as a vector of integers. For example, <2, 4, 3, 2> will create a network with 2 input nodes, 2 layers of hidden nodes with 4 and 3 nodes respectively, and 2 output nodes.

2. Get your data. I set it up to read data from a file, as you generally train a network off data after the fact instead of in real time. An example of a compatible data file is included in the git linked at the end of this post.

3. Send the data through the neural network until the weights of the nodes converge.

4. ????

5. Profit.

Let's use an example: You have an omni-wheeled robot that wants to center itself in a hallway such that it is the same distance to each wall. You put 2 range finders on each side of the robot to arrange for this.

What should your architecture be? This is a little bit of a tricky problem. You can have either 1 or 2 inputs. Let's stick with the 1 input route. The input would be the difference of the two range finders. The output you care about is the 4 motor values, but the output that the neural network is trying to achieve is a single value: zero. A possible architecture could be <1, 4, 4, 1>. Your second hidden layer is actually your motor values.

So how do you use this computational engine you just created? You can do it two ways, in real time, or gather random data and train it on that. I personally find it much more enjoyable to watch the robot learn to center itself. What you'd need to do is set up your robot anywhere (even in the center of the hallway!). Get the sensor data, feed it forward into your neural network, get your motor values, apply them for a duration of x ms, get the new sensor data (which is the output of the network), then train it on that value. Repeat until your weights converge. Once the network converges, save the state of it (as in the node weights). Now you have a robot that taught itself how to center itself in a hallway.

I'm not going to go under the hood of neural networks as this post is already rather lengthy.

Code: https://github.com/faust1706/Neural-Network

I accidentally pushed built code to the repo, my mistake.

Something really cool that can be done with this is have your battery voltage as an input. The network will generalize to work at any functioning voltage.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

Would it be possible to take a video of the realtime learning in action?

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

I'm don't have access to a simple navigation robot anymore...it got scrapped for parts. I'll see if I can put something together one day in the lab.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

Have you tried learning parameters for a PID controller?

Any thoughts on using recurrent neural networks to add some memory to your controller?

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

The goal of releasing this code was that teams would have an easier time with machine control as well as show them that there is more than one way to do things, and a lot of times the first solution you have (the PID controller given to each team), isn't always the best.

By learning parameters, are you talking about setting them yourself, or a PID controller that finds parameters itself? The latter already exists, but it is more complex than a neural network in my opinion.

RNNs do wonders, but I wanted to keep things simple for an introductory post. This would be a great topic for a team to explore for themselves, as well as Elman Networks and others of the like.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

Saw you mention this on r/FRC, I'm definitely eager to check this out!

EDIT: could you post a more detailed example implementation?

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

Of course. It'd be easier for me to do so if you give me an example that you want me to go through, do you have any in mind?

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

An example of this in actual robot code (e.g. teaching it how to center itself like you mentioned) would help a lot. Also if there's any way I can help contribute to this I would be glad to help.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

I added a case for real time training called realtime. It is not integrated with anything, but it is laid out to be embedded easily. There are plenty of comments which should allow for fast integration of any system, so long as you can give this code the sensor data and send the motor values to wherever they are needed. I feel it'd be easier to add this into whatever that you want to train instead of sending values over serial or something, but that's just me.

The code is a little messy, I apologize. The messy part is the training from a file, I'll work on it when I have more time.

If you need help integrating it into FRC code, don't hesitate to ask me.

Edit* also worked on the readme a bit. Still unsure of how specific I want it, but it's a start.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

It looks very nice, I forked it and I'm trying to make an implementation of a basic neural network for use with encoders.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

This looks really cool, and I'm planning to try to hack on it over April vacation (when I finally have some time). Since our team uses Java, I'm going to try to port it, which will hopefully help me understand the code better.

I have no previous experience with neural networks, so I'll probably have a lot of questions in the future, but for now I was wondering if there is some way to decide how to structure the hidden layers. In your example, you use 4 nodes, one for each wheel and two layers. What is the reason for this?

Thanks.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

Here is my implementation of your library for use with encoders on a robot.

https://github.com/DannyMoses/Neural..._implementaion

It's definitely not very polished(I basically copied your example). Am I doing it right? I don't have a robot to test this on right now so I would value any feedback. I also just want to say that I think this is probably the future of FRC programming right here.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

Extremely long post incoming, but I feel others would benefit from reading this as it is an actual implementation of it.

You are somewhat there.

Here is something that I found immediately:

//clear inputs and results to ensure no lurking data
inputVals.clear();
resultValues.clear();

//get your data, populate it in inputVals
//with inputVals[trainingPass].push_back(int)

// Get new input data and feed it forward:
net.feedForward(inputVals);

can you see what's wrong? Two things actually. One is that your input data vectors are empty (of size zero). The other thing is that you aren't sending what you want to train on (your inputs). By doing the latter you willing fix the first issue of the empty vector.

Another thing that is wrong: your architecture is 1-4-4-1, yet you have 2 target values. This is another dimensional error. This time instead of not having dimensional, you have too many!

Alright, moving on. So you want the robot to move 10 feet forward and 10 feet back, correct? What should be your input? What should your output be?

Let's redefine the problem for an easier first time implementation: let's move x feet forward, with the case of x being 10. You might want to decrease it to maybe 3 so you don't have to go chasing a robot around, just a thought.

So your input can be however many feet you want to move forward. In code that is

inputVals.push_back(10); //feet

what is your targetValue? Two approaches come to mind immediately. The first is your target is to travel 10 feet, so

targetVals.push_back(10); //feet.

There is a lot of room to expand on this. For instance, you could input your robot's position in x and y as two inputs, and your desired position be your target values x and y.

Moving on.

float outputVal = (float) motor_values.back();
myRobot.Drive(outputVal, 0.0f);

the .back member function of a vector calls the last value stored in the vector. What is the drive system of this robot? This assumes you only have one motor controlling your entire robot, or that you are only going to use one value of your motor hidden layer. Either way, it is sub optimal for two reasons. Your motor hidden layer has 4 nodes. If you want to go this route, you need to make you NN architecture account for it. That is, 1-n-1-1, where n is any number of nodes you desire.

Now you wait 50 (ms?). This may not be what you want, either. The neural network will try to find the motor value to travel x feet in 50 ms, and in your case that is 10 feet. You need to your robot time to try to get there based off the values you got from the network, maybe 5 seconds? Be careful though. When you first declare a NN, the weights are random, meaning it has a 50% chance of going backwards, so give the robot some room. I'd suggest doing this in an open area like a gym. A cool thing about this is that you don't have to recent the robot per iteration (unless you are getting close to a wall), because each iteration assume x feet from your current location.

I'm not familiar with FRC code anymore, but you may need to set motor values to 0 after the wait.

Now you get your updated sensor data:

results.push_back(leftEncoder.GetDistance() + rightEncoder.GetDistance() / 2);

What you did was right, however your units might not match up. I don't unit of length .GetDistance() returns. You may need to do a simple conversion.

You set this as your output layer, then you train through backpropagation, which is correct.

Then you wait until your network converges and exit.

Yikes. I just realized I forgot to save the state of the network.....that's rather important. I'll work on that asap.

Now your NN is trained properly to go x feet. But it isn't a general solution just yet. You need to test multiple inputs until each one gives the correct output, then send them all through again to ensure that your network is good.

Lastly, you'd want to save the state of this network (which I also haven't written code for yet) so you can load it and go from there.

When you load in a network to give you proper motor values, you do not need to apply backprop to it as it is already trained how you like it. So the code you need when you're done is:

Net net(NN.txt); //file that contains the state of the good NN
vector<double> targetValues(int x);
vector<double> inputValues(int y);
net.feedForward(inputVals);
vector<double> motor_values;
motor_values = net.getLayerValues(int z);
//rest of code that uses motor values

So great, you now have code that makes the robot go x feed forward in n time. As I mentioned earlier, you can get much more fancy with it. Let's bump it up to swerve drive. Your input could be your x and y position (could be assumed to be 0 0), the data you you get from the network could be heading and motor value, and your result would be your new robot position. Another cool addition would be to use the time you want the robot to complete it as an input. This way each distance is not treated equally in terms of time you give it to travel, which will allow for more robust autonomous routines. Yet another cool addition to the inputs would be battery voltage. This way if you have a bad battery in but trained on a good one, it won't matter because the network will generalize to work at every functioning voltage.

Hope this helped! Let me know if you need more assistance or if something doesn't make sense. There is a decent chance I messed up explaining something. It's been a long day.

Thanks for the last part, but the Poofs already have this beat by a lightyear with their 2014 autonomous.

Implementation note: you may want to only run through the network while training when you push a key, as well as add an emergency stop key stroke.

Adding this to the post as well because others could benefit from reading it.

Deciding the architecture of neural networks is a bit of a dark art. It is something that people just accept. As a side project in the lab, I am testing how different architectures of networks affects performance and rate of convergence (in terms of speed and iterations required). Hopefully I'll have some form of a paper by the end of summer to share with whomever is interested.

The reasoning behind by hidden layer architecture is that I am assuming a nonlinear relationship between the inputs and the proper motor values. If I don't have that second hidden layer then each motor values can be calculated by summing the inputs * weight_ij. With the addition of a hidden layer, it still allows for the there to be a linear relationship between the inputs and motor values, but it is also able to account for if the relationship is say, logarithmic.

I highly recommend to play around with different architectures.

I spoke with a buddy of mine at college and he said he'd be willing to help me include a gui so the user can visually see the network adapting, which would be really cool.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

Thanks I was sure something was off lol. I'm going to work on this more later when I have the time.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

I made a preliminary port of the core neural network code to Java, available here. It includes only the core logic and math, not the training data structure or example code. I have not been able to test it yet, but as far as I know it should behave identically to the C++ code, although there are probably a few bugs.

I now have a much better understanding of how the network works but I am still confused by a couple of things.

The simplest use case I have to test the use of a neural network is to replace the PID controller that controls the heading of the robot. From what I understand, the input data could be the difference in the current heading from the desired heading, and the target data value would be 0 in that case. The output value would be the rate of rotation of the robot and I would train the network by making the robot rotate to different headings. Does this sound right?

Also, is there any technical/mathematical reason why the network cannot use other transfer functions, or have you just not written that part yet? I was wondering because I am thinking about adding that functionality to my port.

This is probably a little irrelevant, but why do you only set the output value for the bias node of the output layer, not all of the layers?

Sorry for all the questions, but I am really interested in this.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

That is exactly what 1706 has done since 2012. So long as everything is set up properly, it will work. You might want to add in a feature to save a network state before you do serious testing, otherwise you'll have to retrain every time you run the program as the weights will be set to random and not saved. (The thing you need to save is the architecture as well as the weights. You could get by with just saving the weights if you use the same architecture, but why limit yourself?)

If you look at the readme that is nowhere near complete, I talk about different transfer functions. You can use any function you want that will map an input x, where x ∈ ℝ such that f(x) has a range of [-1, 1] (or [0,1]) and f is strictly increasing on any interval [a,b] (for any b > a, then f(b) > f(a)) Or in other words, f'(x) ≥ 0, where f'(x) is the first derivative of f with respect to x. There are a few transfer functions that I intend to code in but I haven't gotten to it yet. One transfer function, called the step function, only has two outputs: -1 or 1. This is used for binary classification and can be adjusted to multiclass classification if you give it more discrete outputs, such as -2 -1 0 1 2. To implement another transfer function, you must calculate the derivative of it with respect to your input. If you haven't taken calculus yet, you can look it up online or ask a mentor (or fellow student who has taken calculus) to find the derivative for you.

As for the bias node question. That was a mistake, that is why. That line of code:
layers.back().back().setOutputValue(1.0);
was suppose to be in the for loop right above it....thanks for pointing this out to me. I'm sure there are other things that are wrong in this code base as I haven't rigorously looked through it, but it works for the training data I gave it (the xor classifier), so it shouldn't be too devastatingly wrong.

Asking questions is the best way of learning in my opinion. This was all rather new to me as well when I was a student in FRC. Then I went through a textbook on neural networks my last semester of high school (Neural Networks and Learning Machines by Simon Haykin). If you have the math background (that is, calculus III) then go for it, it could be a great way to fill your free time. Even if you don't have quite that extensive of a math background, you can still learn a lot by trying to go through it. If you never expose yourself to new material, you will never learn.

A great resource for me several years ago was a online class over machine learning put on by Andrew Ng (https://www.coursera.org/course/ml). He founded google brain, which in the end resulted in android's speech recognition program. The guy know's what he's talking about.

Edit the following morning: I added the capacity to save the state of a network. It hasn't been tested and I wrote it in one continuous go. There is most likely a bug in it. I'll look over it tonight and fix whatever I find and then test it.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

One thing I noticed and is confusing me, is that you seem to be treating the bias node just like any other neuron in the feed-forward operation. It seems to me that you would never want to feed the previous layer into the bias neuron, because its output is supposed to remain constant.

Also, this is causing the feedForward() method of neuron.cpp to loop out of bounds on the outputWeights vector, because it gets called on the bias neuron, which always has an index that is greater than the max index of outputWeights. This does not cause a crash in the C++ version because the [] operator does not do range checking, but if it is replaced with at() (which does do range checking), your code crashes.

I'm not sure how well I explained this, but I hope you can understand it.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

This happened because I had a thought, partially went through with it, then forgot all about it later on. My thought was to feedforward into a bias node, but just force the output to be one. It seemed like a good idea at the time.

I changed how a network is constructed, which is in net.cpp. If I am not at the output layer, then the first node is the bias node. The bias nodes are added ontop of the architecture given. Meaning that if you give a topology of 2-4-4-2, the real topology will be 3-5-5-2. It still feels like I am missing something, however. I'll investigate further when I have less homework....

Thank you so much for pointing this out.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

I found something which greatly improves the performance of the learning process. In neuron.cpp, you calculate the derivative of the transfer function at output value, when I am pretty sure it should be calculated using the sum of the inputs. Before I made that change, both your C++ version and my Java version would only find a good solution to xordata.txt maybe 10% of the time. I'm not sure if you had more success. It now works nearly 100% of the time after making that change. Also, I think you meant to call "transferFunctionTanHDerivative" instead of "transferFunctionTanH" on line 129 of neuron.cpp.

On an unrelated note, you should probably include a .gitignore file in your repository so you don't accidentally commit backup and binary files.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

It was correct at one point...then I tried to add support for other logistical functions and messed up on it... A large part of why I shared this code was so I could have more pairs of eyes look at it, I really only work on it when I have downtime in the lab, which isn't very often. It is slowly coming along. I'll try to think of more data to train on and make a file for it. (The logical 'and' wouldn't be hard to make. Once I add a visualizer I could do some classification stuff or regression, but that's a little ways away.)

Yeah, I'm a git amateur slowly learning.

Edit* I went ahead and added a anddata.txt file to give something else to train on.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

I also made some other test files, which can be found here.

I don't really understand why you moved the bias neuron to be the first one in each layer. The code does not work right anymore (lots of out of bounds on the vectors and a weird crash in the destructor of Net). If you put the bias as the first neuron, you have to subtract 1 from the neuron index when getting its input weights from the previous layer, but only if you are not dealing with the output layer (because it does not have a bias). This creates (I think) unnecessary complexity.

Also, how do you get your test file (neural-net.cpp) to work? I had to make my own to be able to test your code.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

Sorry about the late response. I had my last round of tests before finals these past two weeks. I was having corruption problems with my git repo so I deleted it and made a new one of the same name.

I got rid of neural-net.cpp and added a tester.cpp for easy testing. I also appear to have fixed the training issue, kind of. It worked 5 out of 5 times for me using th (tanh) on the xor data. I'll have to look into sigmoid.

It isn't optimized, but it will work. It's not intended to be used for deep learning or anything. The most I could see an FRC team use is 4 inputs, a couple hidden layers and 3 outputs. That is, current x, y, heading, and time as inputs and a desired x y heading as outputs.

I also finished saving an entire network. That is, the architecture, transfer funciton, and weights. I have code to load a network that compiles, but I haven't tested it yet.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

Just to let you know that I haven't given up on this, I tested my code on a real robot yesterday and ... well it seems I must have written it too late at night.

I ended up getting the robot to oscillate around a setpoint, but then I changed something and the robot began to spin endlessly. After AP tests are over I'll hopefully be able to debug it some more (and get some video).

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

This^^ AP Physics and USH FTW

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

Something you can do to see what is happening in terms of the NN is save the error of the network after every iteration.

The code has a rather unfortunate bug right now that i don't have time to fix because of finals, it crashes when you try to save the state of the network (the weights). I'll try to find time to fix it as soon as I can.

Let me know if you run into more problems / the same problem and I'll see if I can help further.

I have gotten the saving and loading functionality working on my Java port. I modified your file format a little to make my parser simpler. I think most of my problems are caused by the robot code.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

That's good to hear for me at least...It might be that there is too much influence from new data (alpha), something to look into, but it is unlikely if it is <.20. A simple cheat would be to dampen the motor values outputted by the net if it comes down to that.

1706 is investigating into swerve this summer. I can't wait to see what @cmastudios is able to do with it along with our swerve trajectory planner code.

Re: Tired of tuning pesty PID loops for hours? Try Neural Networks!
 

Wow, I really want to have a chance to mess with all of this.