formal quality procedures in FRC?

[GeeTwo]

Originally Posted by sanddrag

While most of our team can recognize when something is not as good as it can be, we've yet to have a formalize QA process. We really ought to soon.

If it's in your culture, it doesn't need to be formalized - and formalization might just kill the goose that lays the golden eggs.

[wireties]

Originally Posted by GeeTwo

If it's in your culture, it doesn't need to be formalized - and formalization might just kill the goose that lays the golden eggs.

I can see that might be the case on a FRC team. But a company (that makes machines, not web-ish stuff) would never leave anything to chance, there would be a parallel formal QA effort.

[Brian Maher]

After being a victim of a variety of mishaps in previous years, Team 1257 has implemented a pre-match checklist that is completed before every match (practice, qualification, and playoff), listing vital maintenance and match switching procedures. One Pit team member is designated "Inspector", and it is his/her job to oversee the checklist's completion. It has made match turnarounds much more thorough and smooth, and reduces the chance of mistakes.

We also keep a log of technical failures that occur during competition. This helps us notice patterns of error and better address them.

[Qbot2640]

As a quality engineer by trade, I have to admit some embarrassment at not thinking harder about implementing a quality focus toward our FRC team. After reading through this thread, it occurs to me that the FMEA (Failure Mode Effects Analysis) could be particularly useful here. In the early days of build season, the DFMEA (design) to consider all the possible things that could fail with the design and what effect they would have. This could be an effective way to determine where a flaw in the design could lead to a problem you would not otherwise have discovered. As strategy is developed, and procedures are developed for event preparation and participation, the PFMEA (process) to figure out what problems could occur surrounding team performance - similarly useful to guide strategy, maintenance, and other competition considerations.

[IKE]

I was working with 469 this year, and they have a couple practices that are worth sharing:

#1: system Checks in the pits. This is a recommended set of checks for each subsystem to know that the robot is ready for its next match. I highly recommend making a set of checks and refining the list as your season continues.

#2: During build, they ahve a really cool system. The students design every part in CAD that must be fabricated. They then make a drawing for each part, and have a part number scheme that starts with 3 letters that represent the subsystem. IE Arm parts are: ARM001, ARM002.... Stacker parts were: STK001, STK002.... A drawing goes out with each student fabricating the part. They then use 4 colors of stickered dots to code each part. One for in process comp bot & 1 for in process practice. One for Approved comp bot, one for approved practice bot. As soon as the rough stock is cut for a component, it is labeled with the "in-process" sticker which has its PN on it (IE ARM001-yellow). This sticker stays on until the part is "complete" and it is reviewed by the student design lead who approves it by removing the in process, and replacing it with an approved sticker (IE ARM001-yellow).
They keep all the fab parts on a set of shelves with a label for each level corresponding to a subsystem.
They also keep a Fabrication BOM that has all the parts organized by subsystem, and the same color code with a 5th color for not started. They added a column for "milling required" in order to help prioritize/schedule parts through the mill which was one of the limited resources this year.

The Fabrication BOM was a great tool for helping the build coordinator(mentor) know what was ready and set daily priorities as well as guage readiness to plan and add shop hours as needed.

This was my first year dealing with such a slick system (33 utilized a different system that worked well, but differently when I was there). A key component was the team was pretty disciplined to utilize it. This supported a very fast assembly schedule which allowed for some additional design time compared to other teams I have worked with. It also ensured that practice robot was very very similar to competition robot. This allowed for tweaks to be tested on practice and then later implemented on comp bot during the season.

One drawback with this system was practice robot was not "done" (or more appropriately labeled "useful") nearly as early in the build season as some other teams I have talked to. I don't think practice robot was terribly useful until the last couple days of build vs. some teams have a useful "practice" robot around week 4.

************************************************** *********
"Formal Quality" is a pretty broad topic. At my work, it encompasses a lot of different areas, and items. Occasionally, it can be adversarial, though that is often due to "design expectation" not meeting design need. This can be especially true in a "protoype" environment.
************************************************** *********
To the original thread starter, some recommended reading is:

Checklist Manifesto. How to get things right.

Atwul Gwande was part of the World Health Organization responsible for the Surgery Checklist. The results of utilizing a checklist were amazing:

http://www.nejm.org/doi/full/10.1056/NEJMsa0810119

From the above link:
"Use of the WHO Surgery Checklist reduced the rate of deaths and surgical complications by more than one-third across all eight pilot hospitals. The rate of major inpatient complications dropped from 11% to 7%, and the inpatient death rate following major operations fell from 1.5% to 0.8%."

As a reminder, these are some of the most educated people in the world conducting these procedures, and reduction of in-hospital deaths between 30-50%

Because it is a small number, you might say, so what. There are a lot of oeprations in the world. Also from the link:

"Why is the Checklist important?

Surgery can be a life-saving or life-changing intervention in many conditions and the provision of surgical services is being increasingly recognized as a significant public health issue. A modeling study estimated that 234 million operations are carried out every year across the world. This translates to one operation for every 25 people and is more than the number of children born worldwide each year.
An estimation of the global volume of surgery: a modelling strategy based on available data
Lancet 2008

However, despite the positive impact the provision of surgical services can have on a population’s health, surgery itself carries risk. Current estimates of morbidity and mortality following surgery indicate that over 7 million people worldwide will suffer complications following surgery. One million of these people will die as a result. Around half of these complications are potentially preventable, so using the Checklist to improve the safety of surgery will save many thousands of lives each year."

Now, using the checklist and associated practices could/should in theory save: between 300,000 to 500,000 people per year! To put that into context. I believe there were around 40,000 people at the FIRST World Championship and Festival. Imagine saving 8X to 12X the number of participants... every year.

How does this apply to robotics?

A lot of robots end up having issues during matches that keep them from working. I am not sure the exact ratio, but I would guess somewhere around 10% or about 1 issue every other match. This seems a bit low, as I know a lot of very good teams tend to have at least 1 issue per event (with a 12 match qual schedule). Several of the Greats are able to keep a clean schedule at most events, but that is a delta from "really good" to "Great". Often 1 qual match victory can be the difference between #1 and #4/5 seed.
So, if you can reduce system failures from 3 per season to say 1 or 2... you may have a dramatically better season.

[wireties]

Lots of great info here! Thanks! I'm thinking we'll have formal in-process inspection by mentors, hopefully quickly handed over to students, and a rigorous checklist at competitions by students. We'll write up the plan in next few months.

[mathking]

I think there could be two different concepts lurking here. One is the physical quality of the robot. We have found (after years of competition and many unforced errors) that checklists are quite valuable. We have three basic ones we use.

One is the setting up the robot on the field checklist, which includes things like turn the robot on, make sure the robot is properly positioned, make sure the safety bar is removed (that last item came in handy a couple of times last year), and "make sure the proper auto mode is selected."

The next checklist is the one for when the robot arrives back in the pit after a match or practice. It includes take out and charge the battery, put in a practice battery, check the chains (if any), visually inspect the pwm and power connections, and checks for specific problem areas.

The final checklist is the before heading out for the next match list. It includes things like correct battery is in the robot, any prepositioning of components is done, in bumper years the correct color bumpers are on, this year the correct LED light color is selected, driver station is ready to go and the like.

These don't eliminate unforced errors, but they minimize them. (For example, in 2012 we did not initially have "balls loaded" on the checklist. We learned.)

The other concept revolves around how the match itself is actually played. Mistakes by the drive team, interference by allies, defense by opponents and other things can have an effect on how the match turns out. One important thing we try to do is to talk about each match immediately after the match. Video review can help with this. The view from the stands is generally far superior to the view from the driver station, so discussing each match immediately after can be really useful. Be sure to keep the tone of the discussion constructive and not accusatory. Constructive after action conferences at the conclusion of each match can be very helpful in identifying and implementing good practices in competition.

We generally try to divvy up the responsibilities of the drive team so that someone is thinking about what is going on now and someone is thinking about what to do next. Driving in a match is very different from driving on a practice field. There is far more going on in a match, and it is consequently much more difficult to focus on the task at hand. One thing we try to focus on is how to most quickly and effectively communicate so that the drivers know what to do. This reduces the incidence of "paralysis" on the field when the robot is sitting as the humans try to decide what to do. A good coach can help a lot with this. It is one of the reasons we have generally decided to have a mentor coach rather than a student coach. Our on field coach is very good, particularly at staying calm and focused. He generally finds that the more we play, the less he is instructing the drivers in what to do and the more he is looking around to keep ahead of events on the field.