Can someone please explain the issue with brownouts for 6 CIM motors with Robo-Rio?
Check out these two threads:
Brownout behavior - alternative design goals
How do you design a robot that doesn’t brownout?
The issue isn’t strictly limited to 6 CIM drivetrains, but to any system which draws enough current from the battery to bring the battery supply voltage (momentarily) below 6.3V. This is most likely to happen in a pushing match, and many (most?) 6 CIM drive trains designs are intended for pushing matches.
Edit: Another common cause for the brownout is running the robot for far more than three minutes without changing the battery. We haven’t had this happen to us during competition, but it has happened several times during drive practice and at demos.
The high electrical draw of the CIMs such as when turning or accelerating or pushing can lower the overall system voltage, when it goes lower than a 6.8v the roborios inbuilt brownout protection system starts to cutoff things, until it finally turns off itself, meaning that the robot disconnects from the FMS and the driver station.
It is explained in detail here
http://wpilib.screenstepslive.com/s/4485/m/24166/l/289498-roborio-brownout-and-understanding-current-draw
The RoboRIO brown-outs at 6.8V. What does that translate to in Amps? Based on my limited high school physics knowledge, I think it depends on the battery’s internal resistance, but is there a set number for that?
If not current, how else would we know the voltage drop caused by the current draw of a design? I see warnings about 3 CIM drivetrains everywhere, but is there a way to calculate/estimate the voltage drop and mathematically prove that 3 CIM drivetrains will brown-out? It would be interesting to see whether a 3 CIM drivetrain would work under certain situations, or if 3 CIM drivetrains are just not possible.
I can’t speak to the battery resistance part. I can say, however, that there’s more to it than just 3 CIM motors. It also has to do with robot weight, track CoF, gear ratios in the drive gearboxes, lengths and size of wire between the battery and motors, and a bunch of other stuff.
There’s lots of good information in those threads and screensteps. Here’s some more information.
In most cases, a brownout condition will turn off your motors for a fraction of a second, and then everything will recover. You may not even notice most brownouts.
The times that you’re most likely to brownout is when accelerating from a stop, when turning, or when in a pushing match. Here’s ways to reduce or eliminate brownouts.
- Verify all electrical connections are tight. Loose electrical connections will increase current draw.
- Verify smooth mechanical function. Binding or rubbing will cause more current draw.
- Reduce long wire runs. Wire electrical resistance is proportional with length, and reducing wire length will reduce current draw.
- Shift to low gear when pushing or turning
- Reduce aggressiveness of commands to the drivetrain. This can be through driver practice, or through a filter or rate limiter implemented in software.
- Increase wire size. Think about 4awg instead of 6awg for batteries, and 10awg instead of 12awg for your drive motors. If you have a drivetrain that draws 240amps, changing 2 feet of 6awg wire for 4awg wire will reduce voltage drop by about .2 volts, and 2 feet of 12 awg wire with 10awg will reduce voltage drop by about .1 volts.
- Consider reducing power use in other areas. For example, adding air tanks might make it unnecessary to run the compressor.
- Change gear ratios. Using a larger gear reduction will reduce current draw in all the scenarios above.
- Reduce lateral traction (put wheels with less traction, or omni wheels on the corners). This will help with the turning case.
- Reduce forward traction. This will help with the pushing case.
I’m sure I’m missing a few tips.
I’ve seen people suggest that removing 2 CIMs from a 6 CIM drive train will fix things. My guess is that a 6 CIM drivetrain that is consistently browning out will not be measurably helped by removing 2 CIMs. It will respond much more sluggishly. It will also significantly increase the chance of poping the 40 amp breakers, which will take seconds to recover from, rather then tenths of seconds from a brownout.
I did say that in most cases brownouts won’t be noticeable. If a longer brownout occurs, and the power drops to about 6.2 volts, the roboRIO 5v rail will turn off. This removes power from sensors. Here are some specific cases where they will cause negative effects, and how to design around them:
- A mechanism that uses a motor providing constant power to hold something. This could be pinching a ball, holding up an arm, or pulling back a spring for a launcher system. Adding a ratchet or a counterbalance will reduce the effect, if the motor turns off for a little bit.
- For ratio-metric sensors, such as potentiometers, implement ratio-metric measurements using the voltage from the 5v power rail. The voltage is available in all three programming languages, and ratio-metric measurement is implemented in potentiometer class in all three languages. We did this with our 2014 robot and noticed very little disturbance in potentiometer readings when in brownout.
- Replace incremental sensors with absolute sensors. For example, instead of an incremental encoder that you zero at the beginning of the match, use an absolute encoder such as the MA3.
- For sensors that have some type of calibration routine on startup, power them from either the roboRIO USB port or VRM. Some examples are ultrasonic sensors and the NavX IMU. The NavX has additional information here: http://www.pdocs.kauailabs.com/navx-mxp/guidance/best-practices/#2)_Plan_for_Brownouts
In all but the last case, if you’re doing closed loop control, read the state of the 5v rail to determine if you can trust the value from sensor.
Here’s some real world experience:
In 2014, we ran a 6 CIM drive train, 2 speed, with theoretical speeds of ~18 ft/s and 8 ft/s. We used 8 wheel drive with versawheels. We never blew the main breaker. We played plenty of defense towards the end of the season.
We used 10 awg wire on each of the drive motors, as well as kept the wires short (PDB was mounted very near the motors). We used 4 awg wire from the battery to the breaker and PDB, but used standard 6 awg on the battery. We did not implement any software magic, but the driver knew to shift to low when pushing.
We beta tested the roboRIO, and installed it on our practice robot in the fall of 2014. We competed with that robot at the 2014 SCRRF Fall Classic. We never noticed any problems due to brownout. We collected current during a match. See http://www.chiefdelphi.com/forums/showthread.php?p=1403002&postcount=153 The battery voltage shows that we browned out several times, but our drivers never reported any problems.
Several other teams with 6 CIM drivetrains also ran with the roboRIO during the beta. Several teams reported seeing brownouts, but only a few had real problems.
The important thing is to test your robot and see how it responds to a brownout. Like GeeTwo said, just run your robot on the same battery for long enough.
That’s pretty much the high-level story. 3-CIM-per-side drive trains are not, if managed properly, the kiss of death. With proper current and voltage monitoring, it is certainly possible for software to truly “brown” out the motors (by reducing the supply percentage or cutting off a motor) rather than “black” them all out at the same time as the RoboRIO shall do. This is a “black” out in that all of the PWM and CAN motor controllers are disabled at the same time; a brown out is when voltage is limited to prevent a black out. The “black out” prevented by the built-in “RoboRIO brown out” is only enough to prevent a RoboRIO shutdown and reboot. While I cannot tell you that it hs never happened, I do not know of this happening to a cRIO starting with a fresh, healthy battery in a 3-minute match.
The real question is whether, after writing “true brown out software” there is enough of an advantage to justify the weight, expense, and gears of 3-CIMs vs 2-CIMs per side.
Edit (ninja’d):
In our experience, you will **definitely **notice. This is what happens:
- The current draw causes the RIO to “brown out” the motors.
- The voltage recovers.
- The system restores the motors.
- The motors draw inrush current.
- Return to step 1.
This results in a very distinctive “stutter” at about 15-20 jitters per second that gets the robot nowhere fast.
Please don’t construe this as disagreeing with any of Joe’s mitigation strategies! (Well, in #7, I’d say tanks rather than cylinders. Are you reading, IndySam, Lil Lavery, and BBray_T1296?)
I don’t disagree that this is what happens in the really badly wired / really badly geared / really old battery cases, but I don’t think this is the normal case. Certainly, our robot didn’t experience that in real competition (although did in extended pratice). Rather the robot gets started moving, browns out, and starts again, however since there is already motion, the inrush current is less and it breaks the loop.
In the interest of full disclosure, we encountered the “stutter” we encountered was experienced when running longer than 3 minutes, either in driver practice or at a demo. To the assertion that the behavior in the first 3 minutes is “it browns out, but by the time it comes back, its OK”, I have to express a healthy dose of skepticism. If that’s what actually happens, then the old truism “A difference that makes no difference is no difference” applies. I suspect that we’ll end up simulating some pushing matches this year to find out for ourselves.
What would be a safe fps for a 120 pound robot with a 6 CIM drive train?
It depends on your driving conditions and what you mean by safe. If safe means that the chassis is traction limited so that it is all but impossible to draw more than 150A, total, you can use JVN’s latest calculator to get the answer. I leave that as an exercise, but the answer will surely be “too slow to be competitive for most FRC games and strategies.”
The real question with a 6 CIM vs a 4 CIM drivetrain is “How much more torque/power can I get when running current limited?” For sake of calculation, let’s assume you have a 180A current budget for your CIMs in a relatively short battle. During this time period, each CIM in a 6 CIM drivetrain will draw 30A, and each in a 4 CIM dirivetrain will draw 45A.
Checking the CIM power curve, at 12V and 45A each CIM generates about .79 Nm of torque, and at 30A it’s about .51 Nm, meaning that the 4 CIM drive train can actually generate about 0.1Nm more torque. The advantage is in power: the 2160W of electrical power consumed by each drive train is converted to mechanical energy about 7% better by 6 CIMs than 4, yielding another 160W of mechanical power.
If your mechanisms are fast enough and reliable enough and your driver good enough to make use of that power, go for it. If not, 6 CIMs just makes it a bit easier to draw too much current. The bottom line is that a 6 CIM drive train is inherently more dangerous than a 4 CIM drive train. You need to evaluate whether the danger you’re causing is worth the danger you’re accepting.
With 4 CIMs it seems easy enough to trip a 40 Amp breaker if the robot is geared high (fast) and pushing. But what if you ran 6 Mini-CIMs? Less chance of tripping a 40A breaker, and no concern about browning out. In theory, you could gear a robot with 6 Mini-CIMs higher (faster) than a robot with 4 CIMs. Thoughts?
Would pushing with a high gear reduction (slow) do you still have a chance of browning out?
The big snap action breakers are rated 40A. The main breaker is rated 120A. If you have four or more snap action breakers in roughly equal use, the main breaker and brown out are more likely to be an issue than an individual breaker. The advantage to having more small motors is that you are more likely running them nearer peak efficiency, and also spreading the waste heat around better.
You would think this is true, but take a look at the breaker curves. Let’s say you have 4 CIMs going at 80 amps each. That’s 200% rated current for the 40A breakers, so that’s 1.5 to 3.9 seconds. The total is 320 amps, or 267% rated current for the 120A breaker, which is about 7 to 17 seconds. The 40A breakers should go first, assuming your drivetrain is the only thing running.
Does that match teams’ real world observations?
Datasheets*:
http://files.andymark.com/PDFs/am-0282_data_sheet.pdf
http://files.andymark.com/MX5SpecSheet.pdf
*Presumably the datasheets for the 40 and 120A breakers on the Andymark website are accurate and up-to-date
We have gotten better performance on the individual snap actions than advertised. We didn’t do a proper analysis, but it seems we’ve gone about 30-50% beyond those specs (in time) without tripping. We have never really tested the main breaker numbers, because it has never been worth any significant chance of losing EVERYTHING for the rest of the match. As far as I am aware, we’ve only tripped a main breaker one time in 4-1/2 years. In that one case, we found a possibility of a dead short due to missing insulation, so we fixed that, moved on, and never had it again.
Are you sure of this?
Yes, if viewed from the standpoint of accomplishing a given task, which is how we usually consider the robot design. That is, with loose connections or long runs, you will need to draw more Amp-seconds from the battery to (for example) lift a stack of four totes by fourteen inches.
Someone asked for real world testing with a 6 cim drive. I have some things to report.
The robot tested uses the cRio based control system, but we logged voltage to look from situations where we saw voltage drop below 7.5V. We consider this to be an adequate safety factor to ensure we would be in the clear.
Testing completed on slightly lower pile carpet than normal field carpet, but similar in construction.
The drive train is configured as follows:
Mechanical
2 VexPro 3 Cim Ball Shifters at 18.75:1 and 7.08:1 ratios.
6 VexPro traction wheels 6" diameter 2" width with blue nitrile rough top tread
1/8" center drop
Center wheel direct driven.
#35 roller chain to drive front and rear wheel on each side
Outer axles are bolt-through-tube style dead shafts with bearings in the wheel/sprocket
No tensioners used, but chain run has worn in and is not putting un-due load on bearings/gear boxes.
Plate and standoff style construction bolted to a box channel core frame
Robot weight: Roughly 100 lbs
Electrical
6 Standard Cim motors, all new at the beginning of 2014 competition season
10 awg wire between PD Board – motor controller – motors
Power pole connector between PD Board and each speed controller as well as between each speed controller and motor
6 Black Jaguar speed controllers on CanBus through a 2Can driving motors
Battery was changed 2 times during testing.
Programming
Smoothing on inputs was implemented in code for this drive train, but still responds sharply and operates precisely
Automated shifting used in competition.
I need to verify the operational mode used, but I believe 1 speed controller on each side was implemented in speed control mode and the other 2 controllers were slaved to that controller based on current
In testing condition shifters were set manually to high or low gear and shifting code was disabled.
Results
High gear, No compressor, from standing, maximum achievable acceleration
– Would have browned out; voltage dropped significantly below 7.5V
Low gear, No compressor, from standing, maximum achievable acceleration
– Would not have browned out; voltage did not drop below 7.5V
High gear, Compressor running, from standing, maximum achievable acceleration
– Would have browned out; As expected based on first indicated result
Low gear, Compressor running, from standing, maximum achievable acceleration
–Not conclusive; Voltage drop below 7.5V, but not below stated brown out condition voltages
High gear, no compressor, from standing against wall, ramp to traction limit
–Would have browned out; Motor stall achieved critical voltage drop
Low gear, no compressor, from standing against wall, ramp to traction limit
–Would not have browned out; traction broke before 7.5V limit was reached
****driver modulation to stall condition could force brown out.
As much as it pains me to say it, I don’t have actual data to share. This was a quick and dirty test and I thought our experience might be useful.
I’m convinced automated shifting is of critical importance with the consideration of brown out. I recall a number of teams losing 120amp breakers with 6 cim shifting drives at competition in 2014, which is where this drive base originated, and we never had that problem.
Please let me know if you have questions. I’m going to try and set this up and collect actual data early in build season if it appears it might be useful.
My team last year definitely noticed this in our practice sessions, especially when trying to turn (scrub friction increasing the load on the motors). It happened after a few minutes of driving as the battery wore down, and of course it was worse when the robot was trying to move a stack of totes because of the added weight load. Our forklift design caused much of the weight of the totes to press down on the front two wheels.
We would start with a fresh battery, get in a few minutes of driving and stacking, then at some point try to turn the robot while carrying a stack and experience the CLACK-CLACK-CLACK-CLACK stutter which told us it was time to change the battery.
With a fresh battery for each match in competition we didn’t notice it as much. We also did a MacGyver fix by applying duct tape to the treads of the two front wheels to reduce the scrub friction against the carpet.