Achieving Consistency
 

The biggest difference between good and great robots is consistency. My question is how do you build a consistent robot? Are some mechanisms always superior to others? Taking last year's game as an example, Texas had a bunch of amazing catapults while California and Canada had a bunch of incredible wheeled shooters.

Does it just come down the number of iterations of a design? The more tweaking, and fine tuning, the more consistency... What is the best way to go about this fine tuning? Is it possible to make an inferior mechanical design superior with enough programming loops?

During your game analysis/initial design brainstorming, how does consistency factor vs. other design elements?

My team has lacked consistency in scoring game elements in every year of its existence. We've managed to do pretty well in spite of that through strategic creativity, but it's definitely time to take the next step as a program. How do you achieve the consistency needed to be the number 1 alliance captain or first pick at an event?

Re: Achieving Consistency
 

Robustness. One breakdown can and will cost you an event. Which means you need to drive your robot before you bag it. Teams that consistently do well in early season events are the ones that practice driving and find the large bugs in their robot before they get to the first event.

Focus. If you're fighting with some mechanism at the competition and can't seem to get it working - stop. Just don't use it. For instance, we fought with our climber for most of states. As a result, we were always rushing and small things were missed. This hurt our overall performance significantly.

Routine. In everything you do. Have one or two people who always pack the trailer. One or two people who always handle the driver station. One or two people who always do the batteries. When other folks get 'helpful' and try to do things, they get missed or messed up. Stick to your jobs. When everyone performs you'll do well.

Checklists. What must be done before the robot leaves the pits. What must be done when the robot it placed on the field. When should the robot be powered up.

Delegate. Everyone should have their job before they get to the competition. Sorting things out once you get there is difficult.

Verify. One person should be making sure that everyone else gets their responsibilities done.

To summarize: Plan.

Re: Achieving Consistency
 

GOFIRST regularly has engineers or other professionals come in and speak to our group for our general meetings, and today we had an engineer from Boston Scientific who gave an excellent presentation focusing on this exactly (his day job is designing pacemakers and other implanted medical devices). He had a few things which I think are particularly applicable to competitive robots:

1: Understand when reliability is needed, and what the acceptable threshold for reliability is.

He related this back to developing consumer versus "mission critical" equipment, but I think it's particularly relevant in the context of robotics where 100% often isn't actually feasible for a variety of reasons. If failure isn't a huge time sink, designing something that can do the thing multiple times very quickly might actually be a better option than trying to design something that is 100% reliable (and inevitably failing).

2: Fundamental simplicity is better than any level of complexity.

His example for this was a manufacturer who had a laser device for which the lens changed its focal length between operating temperatures (-20 to 100 degrees celsius). The solution that worked best wasn't adding thermal sensors and a control loop, but making the stand out of a material with similar thermal expansion properties as the lens, which automatically kept it at the right length. On robots, this is sometimes more difficult to catch, but think of some of the most famous/well designed robots out there-- like 1114's Simbot SS, which have a fundamental simplicity and functionality to them.

3: Understand modes of failure and alleviate or plan for them.

Often in FRC this means building a part more robustly or more intelligently, or bringing back ups. It also might mean using COTS parts and building in the design to easily remove and replace critical subsystems. I believe Nemesis (2590) has designed robots for which the drive rails are replaceable, which may be an example of this.

There's my small digression, but next up is my personal opinion from someone who has only designed one subsystem that I consider to have been 100% reliable during competition (2220's 2014 drive train). So take it with a grain of salt since I am definitely not the most qualified person on here to be speaking on this subject.

I think Tom hit on the core of it with his last point: consistency is about planning. Not just planning in the general sense, but knowing what can and should be planned for. Sure, a complete failure to plan can ruin your day, but so can putting too much time and effort into planning/designing for things to go wrong when the most effective solution might be to cut your time and make an extra plate or part. And ultimately that knowledge comes from being willing to learn from your experiences.

Re: Achieving Consistency
 

The robot is never done, there is always something to improve. Keep tweaking until it's better and understand why it is better, then redesign and implement. Simplify your systems as far as you can. Accept nothing less than 100% reliability, then making it faster.

I don't even want to talk about how many can grabbers we built in 2015. /shudder But that is the reason not a single team ever beat us in a can race at champs, we made the fastest triggering, highest accelerating, reliable grabbers by spending MONTHS reworking them. We could have gone faster but decided to stick with building a robot and not a cleverly disguised lethal booby trap.

Re: Achieving Consistency
 

Pneumatics. In all seriousness the repeatable force and defined retracted and extended positions allow your robots mechanisms to be where you want them. Giving up full range of motion simplifies not only design, but software control as well.

Additionally, using the field to your advantage to score can eliminate variables and increase consistency and accuracy. See ramp camping in 2006, fender shooting in 2012, shooting from contacting the pyramid or wall in 2013, team 254 2014 etc...

Re: Achieving Consistency
 

A lot of the things I have to say have been said but I can add a few. Most people overlook a couple of things such as combining designs to simplify your robot design. A lot of people this year combined their shooter and intake mechanism into one assembly which I thought was one of the better designs.

Simple machines have less failure points and better durability. I tried to convince my students to redesign our shooter/ intake into a single pivoting mechanism but they were married to what we had for whatever reason. But hey, we are a student led team so that is their machine and their decision.

One thing I try to push out to my students is that the robot is never complete and it can always be better. We don't have a 6 week build season. If you attend more than one event and you didn't win your first, change your machine using your allowable part limit of 30 lbs. That is 1/4 of your machine!

Design new parts for your machine in CAD (if you don't use cad, start) and plan to replace anything that does not work. If something does not work, don't try to use it during competition and remove it from your machine as useless weight. Use a practice robot chassis running or not to test these new designs since your robot will be in the bag. For regional participants this is more challenging but district model teams have their 6 hour time between events to implement.

Just like Karthik's presentation stated and one of the cleanest ways to state it I think: Chase perfection and achieve excellence along the way.

Re: Achieving Consistency
 

I've seen a lot of great points about the robot, but another important point is that you need as much driver practice as you can get. The driving and scoring motions should become natural to your drivers so they almost don't have to think about what buttons they're pushing.

Re: Achieving Consistency
 

Great thread! Achieving consistency is going to be a huge topic for our team this season.

I love all the terrific responses. My question is then, how did you implement these processes? Did you build a second robot? How big is your CAD sub-team? How did you decide to allocate time for the design, build, test iteration?

Thank you!

Re: Achieving Consistency
 

For us, we had two students using CAD but there was a huge choking point in our progress last season with only two people creating the content for the entire mechanical team. This season we trained the majority of our mechanical students in CAD making them more responsible for their own ideas and iterative designs. I hope we turn over designs and ideas more quickly this way because we can fail things out that will not work much faster.

Re: Achieving Consistency
 

Tolerances: Don't design a Jaguar, design a Ford Pickup. You don't want your robot to only work when perfectly tuned and everything is just so. You want it to be able to tolerate as much variation as possible. This variation can come from game elements, field location, or degradation of the robot over the season.

Margin: Don't build to peak performance, always have margin for when things degrade. Electrically, your battery will drop below 12V and this will impact your motors. Pneumatics wise, you may get something in your air flow that restrict it slightly.

Parts that Break: The earlier you find out a part is going to break, the better. This year the programming team was shearing rivets on our intake (we WERE being careful). We got it changed to bolts for the season (drive team is not as gentle as we are). Related item, make sure you know what will break and use this to your advantage. If you have the choice in your design between an easy to replace part breaking or a hard to replace part, ensure the easy to replace part breaks first (sacrificial component) and just build several of them. We had some small pieces of metal used for lifting the portcullis and they broke ~10 times during the season due to impacts with the walls.

Automation: Some designs you can control with PID (or similar). Some designs you can't. If you can have it controlled by software, you'll get more repeatability than by a player. In 2014 we had two different ways of pneumatic catapulting a ball (complaints on pneumatics aside). One pulsed the pneumatics for ~125ms (milliseconds) and the other fired into a hard stop. Both used software and the timing made it so our driver hit one of two buttons based on where he was on the field to score. He couldn't have done these reliably under manual control.

Re: Achieving Consistency
 

Not really. The montra I have always heard is "Never try to fix hardware with software".

Utilizing feedback to enable the software to make better decisions is one way to increase robustness. For example:

A wheeled shooter using a PID or Bang-Bang controller will be more consistent than one which just sets full power all the time (since "Full Power" decreases as battery voltage decreases).

An autonomous routine that uses encoders to travel a fixed distance will work better than a routine that simply turns the motors on for a fixed amount of time (battery voltage, obstacles, differing carpet friction, etc.)

However, without the feedback mechanism (encoder or other sensor), there is very little help that software will provide. Even with a feedback mechanism, there will still be mechanical limits which software cannot get around.

As stated, planning up front is crucial to this process. Know what software techniques are available from your team's knowledge and experience. Part of every technique is knowing its limitations and respecting them during the design process.

Re: Achieving Consistency
 

The iteration of the design must be driven by concrete measurements to be of real benefit. In mass manufacturing, the concept of DMAIC is used for statistical process control. It means that changes are made and the effects of those changes measured, recorded and compared with the results of other experiments. An integral part of this is keeping an Engineering Notebook. I have heard the following many times "we tried a bunch of different settings for X and it worked great at one of them but we don't remember what that setting was and we can't get it back".

https://en.wikipedia.org/wiki/DMAIC

From my past experience in robotics and my day job designing industrial electronics, one has to be willing to go through at least 3-4 iterations to get a robot that performs consistently in competition or a design good enough to ship to a customer. This means you have to commit, up front, to having enough time in your schedule and have enough resources and materials to actually do those iterations. Prior experience applied to better planning and design work up front can reduce the number of iterations that you will actually need. You can then use that time for further design improvements or practice.


Yes, if you mean iterating an inferior mechanical design to make it a good one. It may take a lot of work. It would be best to study the state of the art, do better planning and up-front design (including some calculations for some STEM content) so that you have a better initial design. Aim to throw out the inferior designs in the time between brainstorming and committing to a final design. Once one has committed to a design, it becomes increasing difficult to make improvements (see Figure 1 in the link below).

http://enfinio.com/new-product-development/


Consistency of performance should be one of the top priorities throughout your design process, especially when brainstorming and doing the detail design work.

At the 2015 Championship I visited the teams that were doing well in our division, Tesla, and the division with pits across the aisle, Archimedes. I asked them what made their teams successful. Later, I took a group of our team members and visited those teams again and asked them to repeat what they had told me (thank you 234, 1538, 2481 and 2122). The common thread was best expressed by one of the mentors for the Roboteers, "obsessive control of the game piece". Look where it got them :)

I chose these teams because I felt that our team members could relate to what they were doing and feel that they could emulating them. I specifically avoided taking our team members to visit teams such as 254, 1114, 67, 16 since our students were likely to view them as "Gods who can do anything".

Re: Achieving Consistency
 

This. DO NOT BABY YOUR ROBOT! Do not drive it gently in practice. Do not place game pieces perfectly for your robot to intake in practice. Practice with everything in the worst possible scenarios you can throw at your drivers, make them chase those game pieces because that is what the game will be like. Take bigger risks in practice to find the limits of your performance and durability, then try to improve the robot to make it easier for your drivers to do their job. Remember that every single second counts and needs to be spent wisely.

Re: Achieving Consistency
 

General advice: Fail faster. Iterate like crazy, identify design weak points and fix them (either literally weak as in structure, or as in "the speed of the intake is the limiting factor in our scoring speed"). Spend an extensive amount of time prototyping and tuning variables to match desired performance goals rather than just proving the concept. Strategize well. Compromise on the right part of the design, not the wrong parts that make it easy.

Re: Achieving Consistency
 

We build a second robot. Our cad team is usually 4 or 5 students and a mentor. We have a build calendar that we've honed over time, but in reality there has never been a year that we follow it to a T. Every team needs a mentor who has a grasp of everything that is going on with the robot and knows when to break process and take a shortcut, or when to call it quits on a mechanism that showed promise but is proving problematic. That program lead mentor is the difference in a lot of good teams.

Re: Achieving Consistency
 

Can you speak more on which responsibilities are given to the program lead mentor other than watching development and calling off progress on a mechanism? This is an interesting role that I never really see being talked about and would like to know more in depth about its proper usage.

Re: Achieving Consistency
 

Great tips in this thread. I'm a fan of "fail early, fail often". It's definitely something we've been practicing this offseason.

At the event: Do a FULL systems check and inspection of your robot. This helps us identify potential issues early enough to fix them for our next match. Also, avoid fielding untested changes by doing a full systems check after ANY change, software or hardware. Even the smallest change may have unexpected effects on robot functionality.

Minimize(ideally eliminate) the number of changes to driver-facing controls at the event. This ensures that your drivers' inputs will produce the same outputs match to match.

Re: Achieving Consistency
 

Be careful what you wish for, because you may get it.

I recall an FLL team that won an award, with consistency being one of the judges' high comments. The kids were greatly pleased to win an award, but one mentor was heard to say, away from the kids, "What, consistently mediocre?" :rolleyes:

Granted, you want what you can do to be consistent, rather than haphazard. Doing one thing well consistently is better than doing 3 things inconsistently, only sometimes achieving the results you want. But I would say consistency is only one of the criteria needed to become a "great team".

Re: Achieving Consistency
 

I can agree consistency is only part of being a great team, but from an engineering design standpoint consistency is the first thing you need. Edward Deming taught manufacturers to drive variation out of the system. Only once you do that can you truly control where you are at and drive the system to where you want it to be. If you're inconsistent, you can't say if something worked because it works or if it was just a one time event.

Re: Achieving Consistency
 

1 - Full function prototypes.
You don't need to do this for everything, just the stuff you want to work.

2 - Hundreds of hours of consistency testing and tweaking.
The 148 unveil video from this season showcased our method of "us putting balls through the catapult until we didn't hate it".

Re: Achieving Consistency
 

Getting the process that can help breed consistency on your team is just like learning how to walk before you can run. You're almost never getting anywhere with an inconsistent robot. You're getting somewhere when you hit a consistent walking pace. Figure out how to add complexity without sacrificing your consistency? You're running laps around everyone else.

Build up some good, old-fashioned hate for your robot when you first start working on it. Try to break everything! We built our drivetrain in a week and spent the rest of the season running it over defenses trying to break it (we couldn't). Hold your core functions (driving, primary scoring acquisition, and primary scoring delivery are usually the core functions of a competitive robot) to incredibly high standards. Accept you will never get them perfect, but never hesitate at investigating any possible changes. As a team hones the ability to master these core functions, you can add complexity to these and other functions and create a stratospheric machine (I hope).

Re: Achieving Consistency
 

I would go even further and say always hate your robot. Keep breaking, iterating, and trying to figure out how to change the robot from week one of build season to champs.

Re: Achieving Consistency
 

Doing a little research on 4607, I’ll note that you aren’t in too different position than 3005 was ~1-2 years ago. Arguably, we have a ways to go before claiming consistency, but here are the things I think mattered most for our success this last season.

#1: Starting with a solid strategy. If you design a great robot to do the wrong thing, or commit to building “too much robot” and sacrifice time to tune, practice, iterate, you are taking a high risk / high reward approach. I’d argue that a team that is consistently successful is more likely to grow and improve. Define what consistent success looks like for your team, perhaps always make elims at a regional, or trying to be a “picker” vs. a “pickee”, or always making it to champs, or anything else.

#2: Put in the time, but make sure the time is meaningful. The best teams aren’t magic, their combination of hard work and design efficiency allows them to do in a few days what some teams need 6 weeks to do. This goes into later points, but you need to enter the build season with as many people as possible trained to contribute in some meaningful way. That doesn’t mean every student/mentor needs to be a CAD expert, or a machining expert, or scouting expert. Robot seasons are made up of a million important tasks, from designing the most complex mechanism to tapping the millionth hole. Try to delegate in a way that uses each person most efficiently. As an example, our students are still building confidence in their ability to prototype independently. A known bottleneck is having enough people in Week 1 & 2 to efficiently and effectively evaluate a breadth of options. Identify bottlenecks and address them.

#3: Have a plan and be willing to make tough decisions regarding schedule. There is likely not a single magic design I would trade to make me willing to go into the first regional with zero driving practice or runtime on a robot.

#4: Design in CAD / Practice Robot: This can be dependent on machining skills, CNC availability, finance, other resources. However, under the bag/tag rules (correct or not), there is a huge advantage to the additional iteration time that is afforded with a practice bot. For this to be effective though, your construction needs to be repeatable enough such that your iterated designs can be transferred to your bagged robot with a high probability of working.

#5: Sweat the small stuff. If you want to be a top 5 robot at a regional, it’s very hard to do if you lose more than 2-3 matches. Start by figuring out why you are losing matches. It’s easy to think, well if my robot could score 10 more points I could beat X, Y, Z team, it’s a robot design issue. If this was 2015 and every match you scored exactly the same amount of points, that could very well be the case. However, I can’t tell you how many matches we’ve lost in my history due to… faulty battery, leaking pneumatics, bolt working its way loose, part breaking without a spare, etc. All the points about pit checklists, getting students to take ownership of the details (battery management, spares creation, etc) can be considered low hanging fruit when compared to designing a better robot.

#6 Robot Mechanical Design: Yes, there are certain designs that are just more prone to consistency than others. I agree with much that is said about simplicity. As an additional preference, even at the sacrifice of simplicity, I’ll point to 2015 and our decision to not have an active intake. Our elevator/arms interface with the tote was only effective when we were lined up ~+/- 5 degrees with the tote, and we lacked any way to effectively center it. Any time your robot can actively engage a gamepiece and pull it into a known position before taking the next action, it is likely worth it. Same thing with methods to clamp a ball before shooting (in case you are bumped), as well as other game piece movement issues.

#7 Robot Electrical Design: Learn how to properly wire robots, use high quality connectors/wire, properly strain relieve wires, etc. Electrical issues are a pain to troubleshoot, stop them at the source.

Re: Achieving Consistency
 

Exactly. It's good to not get too invested at the beginning, but I would guarantee you that there are probably fewer than a dozen robots ever built at the powerhouse level that didn't have some obnoxious quirk or "what if" scenario attached to them by the teams that built them.

Re: Achieving Consistency
 

If memory serves, I've heard only one time in my life someone from a powerhouse team say "I don't think we'd do anything differently if we had to do it again". I heard someone from 2056 say this at IRI in 2012.

2056 is the most consistent team in the world. They're the definition of excellence. Any designer that doesn't hate SOME aspect of their robot, unless they are on 2056's level, is blinded by ego. Be constantly vigilant for flaws and imperfections, and be bold enough to iterate past them and fix them. If at some point you or your team burns out and can't justify continuing to improve the robot, it's okay to make the tradeoff between excellence and survival, but figure out how to avoid that scenario in the next year.

Re: Achieving Consistency
 

While 1764 had an issue with breaking down, I wasn't a big part of mechanical or the drive train, so I couldn't throw in my two cents on that.
What I CAN have a part in is the discussion on have consistency scoring, specifically with high goals, as we also had an issue with that, but to an extent that we're going to try to fix by our offseason event. ALWAYS USE SENSORS, this may seem like an obvious statement, but the failure to properly wire and mount ultrasonics, limit switches, and encoders drastically decreased our ability to shoot. the more sensors you can have (within feasible parameters), the better off you are, if you can do camera tracking AND an ultrasonic rangefinder along with a predictable arc for your boulder (meaning either a catapult or a wheeled shooter+encoder), you could probably wreck at comp.
If this is common knowledge to everyone else then sorry, but my team had an issue with it so I though I would bring it up for teams in similar situations.

Re: Achieving Consistency
 

This thread has many great recommendations. Underlying most of them is that good teams are careful planners. We have a saying (albeit a common one) - plan the work then work the plan. Map out time for driver practice, tuning, iterative design and inter-competition improvement. The planning does little good without enforcement. The enforcer can be a group of student leaders or the lead mentor, whatever works for your team. Watch your money, your weight, your progress on key mechanisms, progress towards awards planning milestones - watch it all and do it every day.

Re: Achieving Consistency
 

Know your comfort zone.

Don't get me wrong, trying something new is a great way to learn and this year we tried a lot of new things on our 2016 robot.

Something we agreed we wouldn't try was a wheeled shooter knowing we had had limited success with them in the past. While it eliminated a lot of winning robot concepts it allowed us to focus our efforts on mechanisms we knew we could make far more consistent than a wheeled shooter with foam balls.

Re: Achieving Consistency
 

This right here is EXACTLY why we always try to prototype some robot mechanisms in the offseason that we have never tried before. We remembered the 2012 game balls and knew that they could be shot well with a good wheeled shooter and we have had a lot of experience with them, so for stronghold we ran a wheeled shooter(the wheel you choose makes a huge difference btw, it depends on the game piece). However you should always expand your knowledge base whenever possible. Take design risk when they don't matter so much. In 2014 we did an elevator,which we used the last 2 robots. In 2015 we did an articulated arm, which we used this last year. For this year we are doing both a turret and a pneumatic catapult.

Build what you know!

Steal from the best, invent the rest! Then take notes, record your results and put it in the cloud.

Re: Achieving Consistency
 

A team will retain the lessons learned much more if they keep some sort of records of what worked, what didn't work and why. Students and mentors come and go, taking the experience they gain, personally, with them.

Re: Achieving Consistency
 

This is something that my team is currently focusing on improving. We are definitely not experts in this but we have had very good results with our prematch check lists. IMO The little things cost you more matches than the big things.

Re: Achieving Consistency
 

I've found two ideas which are useful in and out of FRC:

1. Be willing to give up your ideas: You are not perfect, and neither is anyone else. Therefore, every idea that you have is not going to be the solution to your current problem. Listen to the input of your peers, and be willing to work with others on their concepts. Your goal is for your team to win and do well, not to build your robot.

2. Don't fall victim to the Sunk-Cost Fallacy: Just because you have spent X amount of hours or Y amount of dollars on creating your idea/mechanism/robot doesn't mean the mechanism's faults are justified. If your, albeit expensive and costly, mechanism is faulty, get rid of it.