NI releasing/designing new controller for FRC

[apples000]

Does anybody know where the extra time is actually coming from?

[JamesTerm]

Originally Posted by apalrd

The only reason it's so darn hard to do stuff now is because of inefficiency in the current system - The WPIlib in LabVIEW is so unoptimized that it's nearly impossible to run code that uses IO calls faster than 20ms task time without saturating the CPU.

I'm wondering if any c++ teams had any performance issues as stated here... From my wind-river experience, I have no complaints with WPIlib performance, and we do some complex code. We run using a 10ms sleep but could probably do 5ms quite easily... the entire loop clocks around 1 - 2 ms... I'll verify later today... but the cpu usage was under 30%.


P.S. it was amazing what could be done with 6502 assembly! C=

[MrRoboSteve]

I looked at a lot of C++ code at the regionals where I was CSA, and with the exception of vision processing didn't run across a team who had CPU utilization problems related to WPILib performance.

It would be interesting to see a sample project that demonstrates the issue.

[Radical Pi]

Originally Posted by JamesTerm

I'm wondering if any c++ teams had any performance issues as stated here... From my wind-river experience, I have no complaints with WPIlib performance, and we do some complex code. We run using a 10ms sleep but could probably do 5ms quite easily... the entire loop clocks around 1 - 2 ms... I'll verify later today... but the cpu usage was under 30%.

Same thing here. The only time we ever saw 100% CPU was due to a bug in SmartDashboard that I missed the patch for. Even then it didn't cause any significant issues. Same thing with Java from what I've heard.

[FrankJ]

Frank has done it again. He smuggled out a picture of the 2015 control board on his blog. It is chip-less!!

[Tom Line]

Originally Posted by Greg McKaskle

I have two years logs for 1718. Their typical usage for matches in 2013 St Louis was under 30%. At earlier 2013 competitions it is several points lower, and in 2012 at the end of the season it was 40%. I don't have logs for the other teams on my laptop.

To add information to this:

In 2012 the two biggest chunks of processor usage were our vision processing and our speed control on the shooter. The vision processing would increase CPU utilization 100% but only for a fraction of a second before we started shooting.

Our speed control ran in a timed loop and that was the single biggest contributor to our CPU usage that year. We scaled it back from 5 ms to 10 or 15 (I can't remember the final one) to improve CPU utlization.

In 2013, our speed control loop was the biggest user again. We didn't use vision at all. During the year, we stopped using the old '09 classmate because the lag was noticeably worse than using a new laptop with an I5 processor, even with the stock driverstation.

I wouldn't say that CPU usage has ever limited what we've done in competition. Teams need to understand how the changes they make affect CPU usage though. Perhaps it's time to flash a message in the diagnostic window that cpu usage is approaching 100%. That will at least let users know when something is wrong. Many new users don't know enough to look at the charting tab.

There is no getting around it though: an FRC control system is not simple. Personally, I don't think that it's too complicated for high school students to utilize.


The single biggest issue I have with the cRIO and other systems is compile time. In order to completely remove compile time issues from our robot, we now have every single constant or modifiable value stored in text files. Updating something is a matter of changing the text in the file and uploading to to the cRIO via FTP. It takes about 5 seconds after power on, since you don't have to wait for code init or anything else. The cRIO operating system boots very quickly. The change was necessitated by the time in 2011 at Worlds when we weren't able to finish tweaking our two tube auton because it was taking too long to compile. We can do it while the robot is live, too, and a single button press reloads all the constants from the text files.

The only time this year we actually reprogrammed anything was when we added a drive-to-mid-line and stop on the center discs in case we played against 469. On another note, this is the second time we specfically had to write an auto mode to try to stop them - the first time was in 2010.

[crake]

Originally Posted by FrankJ

Frank has done it again. He smuggled out a picture of the 2015 control board on his blog. It is chip-less!!

This is now part of the 6 week design process. Some assembly required.

[JamesTerm]

Originally Posted by Tom Line

To add information to this:
Our speed control ran in a timed loop and that was the single biggest contributor to our CPU usage that year. We scaled it back from 5 ms to 10 or 15 (I can't remember the final one) to improve CPU utlization.

Which platform/language are you using?
If it is c++, Do you use the PID functionality included in the WPI Libraries?

[Tom Line]

Originally Posted by apples000

The CMUcam wasn't as great as the current axis cam, but the only successful implementations of vision that I know of don't use the cRIO.

There were many successful vision implementations that used the cRIO. It is important to understand the difference between real-time-targetting and taking a single frame to aim.

[Tom Line]

Originally Posted by JamesTerm

Which platform/language are you using?
If it is c++, Do you use the PID functionality included in the WPI Libraries?

We use LabVIEW exclusively, and use Velocity PID loops that we wrote. We were using the 2011 banner light sensor with reflective tape on half the wheel just like Jared's implementation on 341 Miss Daisy.

The PID is not what eats the CPU. The timed loop in LabVIEW actually forces the loop to a certain timing, rather than just waiting or sleeping it. To simplify it somewhat, if you set a timed loop to 5ms, all the other tasks will take a back seat to that single loop running every 5ms. It really hurts CPU.

I don't know if c++ has an equivalent.

[JamesTerm]

Originally Posted by Tom Line

The PID is not what eats the CPU. The timed loop in LabVIEW actually forces the loop to a certain timing, rather than just waiting or sleeping it. To simplify it somewhat, if you set a timed loop to 5ms, all the other tasks will take a back seat to that single loop running every 5ms. It really hurts CPU.

I don't know if c++ has an equivalent.

In the WPI Lib PID code in PIDController.cpp (as of version 3487) opens a Notifier object with a default parameter of 50ms. It does like you describe forces the loop to a certain timing, and the reason for this is that this design does not wish to calculate uneven time slices within the PID algorithm. The result is good results from a functionality standpoint, but will have the same issue of context switching especially if one wishes to use a lower period. In c++ the programmer does not have to go in this direction, they can write their own PID http://www.termstech.com/files/Archi...Controller.zip and avoid context switching all together. I had proposed this solution for the WPI library but there was not enough expressed interest in going in this direction. Really when you think of it... a machine can only execute one instruction at a time... when you have tasks/threads... the context switch overhead can really eat up cpu processing, and it doesn't have to be this way. I have spent several years on this issue documented here http://www.termstech.com/articles/PID_Kalman.html

[apalrd]

Interesting,

The last time I ran the VI profiler with the full WPIlib the highest time VI's were the Relay Set and Motor Set, and the inefficiencies of them stacked up.

One of the real reasons LV is so inefficient even with seemingly simple code is because of the way it deals with with execution. In C and most other languages a function is an almost logical construct which just segments code, LV does not separate VI's this way.

In LV, each VI is a node in the execution system. LV then manages a smaller set of VxWorks tasks and an execution state of each VI. For this reason, by default each VI has a single instance and any local memory is retained between calls (you can use a VI for data storage by having a get/set input plus a data input/output and a shift register, the WPIlib does this a lot if you look). Any single VI can also only execute once at a given time, so an execution in another thread blocks the same VI from executing in a different thread. When all of the inputs necessary for a VI to execute is ready, the VI will be scheduled into a task to be run at the earliest opportunity, and then the data output will be set and the dependent nodes can execute.

This execution system is significantly more inefficient than a C function call, but virtually nobody realizes this. For this reason, a VI call is considered an expensive entity, not as expensive as another task entirely but not as cheap as a C function call. The ways around it are to set the VI as subroutined (this will not work if any contents are non-subroutined or blocking nodes) which cheapens it to near a C function call, or set the VI as inlined (this does not require non-subroutined subVI's but does require the VI to be re-entrant) which is inlined at compile time (this can reduce compile efficiency if you change a VI which is inlined as it has to rebuild all VI's which include it). Both subroutined and inlined VI's cannot display front panel data or probes in realtime when debugging, but you can still pass data through the connector as usual.

In a lot of ways the LV execution system helps a lot when you want to do multitasking (which is trivial in LV) and the single local variable set helps with data storage in quite a few cases, but if you don't understand it and set the subroutine and inlined properties rigorously for every VI in a project, the inefficiencies of the execution system stack up really fast, especially for a library the size of WPIlib plus a team project with over a hundred VI's. In 2012 I thoroughly went through my WPIlib copy to subroutine and inline VI's where possible, I believe some of this was later integrated to WPIlib.


I personally think it's quite reasonable in RT embedded systems to essentially cheat the OS. Every other RT embedded system I've worked on runs purely statically allocated RAM and uses processor ISR's to deal with tasks, so the OS kernel is a single function (timed ISR) and there are no context switches. There is then no penalty for doing context switches frequently, but we still try to optimize it.

In C++ the PID only runs at 50ms (20hz)? That seems insanely slow! I would expect at least 20ms to be considered a realtime control loop.

[Tom Line]

Originally Posted by JamesTerm

In the WPI Lib PID code in PIDController.cpp (as of version 3487) opens a Notifier object with a default parameter of 50ms. It does like you describe forces the loop to a certain timing, and the reason for this is that this design does not wish to calculate uneven time slices within the PID algorithm. The result is good results from a functionality standpoint, but will have the same issue of context switching especially if one wishes to use a lower period. In c++ the programmer does not have to go in this direction, they can write their own PID http://www.termstech.com/files/Archi...Controller.zip and avoid context switching all together. I had proposed this solution for the WPI library but there was not enough expressed interest in going in this direction. Really when you think of it... a machine can only execute one instruction at a time... when you have tasks/threads... the context switch overhead can really eat up cpu processing, and it doesn't have to be this way. I have spent several years on this issue documented here http://www.termstech.com/articles/PID_Kalman.html

Our homebrew PID normalizes for time, so the calculation itself does not need to be inside a timed loop. We place it in a timed loop so that we can attempt to match the speed controller update rate for the best 'idealized' performance. I'm not sure how you'd go about this without a timed loop.

[JamesTerm]

Originally Posted by apalrd

I personally think it's quite reasonable in RT embedded systems to essentially cheat the OS. Every other RT embedded system I've worked on runs purely statically allocated RAM and uses processor ISR's to deal with tasks, so the OS kernel is a single function (timed ISR) and there are no context switches. There is then no penalty for doing context switches frequently, but we still try to optimize it.

That is very cool in regards to the ISR's dealing with tasks... I'll want to research this a little tomorrow. What I have found (using Intel processor I7). Is to treat threads (aka tasks) as sparingly as possible to only be reserved for timing and I/O tasks. We tried some parallelization techniques, and found unless there is NO convergence (which is usually not the case) we don't come ahead in saving time except for very rare cases which only benefit on some machines. Doing less work that is simplified in one task usually wins. I'm not saying that this is not possible, but to successfully achieve optimal parallelization infringes on making the code less readable and maintainable than what we cared to pursue.

Originally Posted by apalrd

In C++ the PID only runs at 50ms (20hz)? That seems insanely slow! I would expect at least 20ms to be considered a realtime control loop.

This is what this looks like:

PIDController(float p, float i, float d,
					PIDSource *source, PIDOutput *output,
					float period = 0.05);

I am not sure why this was chosen as the default parameter, but as I recall from Brad, this was tested by several teams to work well for many general cases. I think I may have an email on that somewhere. One can easily construct this class to use a different rate, but I know some teams may have used the robot builder for this season and they probably could have over-looked this in the auto generated c++ code.

[JamesTerm]

Originally Posted by Tom Line

Our homebrew PID normalizes for time, so the calculation itself does not need to be inside a timed loop. We place it in a timed loop so that we can attempt to match the speed controller update rate for the best 'idealized' performance. I'm not sure how you'd go about this without a timed loop.

There are a few questions I have on this:
1. How well would this work mechanically if you used a 10ms timed loop?
2. How much improvement would the cpu usage be for this?

Unfortunately, I do not know much about LabView at all, but in c++ you call GetTime() in your main OperatorControl() loop and pass this time as a parameter. For our code the entire loop consists of "void TimeChange(double dTime_s)" call delegated out to various classes including the PID controller class (per rotary system).

Here is our main loop

	double LastTime = GetTime();

	while (IsOperatorControl() && !IsDisabled())
	{
		const double CurrentTime=GetTime();
		//I'll keep this around as a synthetic time option for debug purposes
		//const double DeltaTime=0.020;
		const double DeltaTime=CurrentTime - LastTime;
		LastTime=CurrentTime;
		//Framework::Base::DebugOutput("%f\n",time),
		m_Manager.TimeChange(DeltaTime);
		Wait(0.010);
	}