If you don’t mind, how do you know? I don’t want the same thing to happen to my mind so being to spot the warning signs would be helpful.
While it is surprising in some ways, these observations match both my professional and robotics lives. Formality is the enemy of fast. It’s a fact of life I’ve seen proven out over and over.
I think a lot of teams who build quality robots have figured out the overall process to being successful relies on failing fast, failing fast again and being flexible, in that order. ~10 or so years ago, 125 had a fairly rigid process, with sponsor design reviews, gantt charts, etc. It wasn’t very effective and really bogged us down as we tried to hit each ‘checkpoint’ on our way to a good robot.
Think of the scrappy start-up versus the established defense contractor. They both have their places in the world - especially when consumer/user safety is on the line.
The reason the ‘scrappy start up’ process likely wins out in FRC is because the time constraint is so severe. Failing frequently helps you get to something workable faster. That being said, I think a ‘scrappy start up’ team needs a wealth of wisdom/guidance from a group of mentors to ultimately be highly successful - as it helps to use the rapid iteration/failure to be as efficient as possible (ie: not wasting failure cycles on less than stellar concepts or ideas).
By making mistakes as a mentor. In the past, I’ve “given design feedback” as a mentor without being careful of power dynamics. That “giving design feedback” became just taking control of the design away from the students working on the robot, and in the process I accidentally took a lot of the wind out of our sails on student engagement in that year’s robot.
(I can actually think of at least two episodes in this archetype (several years apart ::ouch:: ) that I’ve been involved in. Mostly happens when I get too caught up geeking out about robot details and forget my purpose as a mentor is to enable students.)
+1! Making sure the students pick up more lessons from fewer failures is a key mentor role.
Thanks for doing this Adam, the results are really surprising to me even though they shouldn’t be. I’ve always thought the 973/148/125/1678’s of the world had FRC down to a science with a rigorous and proven design process. Even when reading JVN’s Build Blog, I always thought that there must be some sort of magic structure/formula underneath all the “fake it 'till you make it” style that’s so eloquently described in the blog. I unabashedly parrot “fail faster” every chance I get when working with a team that is trying to get better…Basically I thought that there must be some magic formula that leads to consistent success at the highest levels, because how else would that level of sustained success be possible?
When I compare this line of thinking with my actual experiences within FRC, it doesn’t hold up. Arguably the best robots I’ve helped to design/build have come out of Ri3D… which I realize is kind of sad, but when I think about it, it makes a little sense. Ri3D is the ultimate when it comes to free flowing ideas and failing fast. There is no time for bureaucracy and there certainly isn’t time to implement a structured plan. I tried following a strict plan my first year of Ri3D and you can watch our videos from that year to see how that went… but actually please don’t, they’ll make your eyes bleed (it doesn’t help that the “game” was Recycle Rush).
Additionally, my personal experience with successful teams from my region seem to follow the same design/communication principles that have proven successful for the teams listed above. I can’t speak to internal team communications/structure specifically, but 4539 “finishes” (of course FRC robots are never actually done. I just mean done to the point where they could win a week 1 event) their robot by the 4th week of the build season. They fail so much faster than other teams in Minnesota and I mean that in the most complimentary way. I’m sure many other successful teams will have a similar story.
On 4607, the team has gone through many different phases, as I would call them, when it comes to design and team communication. The team has used Slack incredibly effectively for communication in recent years. The team Github is becoming more and more of an asset as documentation has gotten significantly better lately thanks in no small part to some amazing new mentors. When it comes to failure documentation, I haven’t seen a better implemented FMEA system from an FRC team, and that will only improve as historical data becomes more accurate and better tracked. The FMEA allows the team to make data-driven improvements to the robot without bogging down speed/creativity with a rigid design implementation structure. I think it strikes a fine balance between structure and speed which proves to be a net positive for the team.
The Outreach team on 4607 has significant amount of structure built into it, but unlike the design/robot side of an FRC team, I believe this is a very positive thing. The structure enables a more systematic approach to culture change, which is goal driven and well documented. Without this infrastructure, the team would not be nearly as effective in this area in my opinion.
An area where I think there will be significant growth for 2019 relates to the point this thread is making. 4607 does a lot of things right, but one thing that has eluded the team has been a well implemented design process. Some years, the design group has been too isolated with not enough new ideas. Some years there are too many ideas and not enough prototypes. Actually… there are never enough prototypes. A statement which probably rings true for most teams in most years (except 148 by the sounds of it). Last year for example (arguably 4607’s most successful year), a strict design review process prevented the team from completing the drivetrain design until the end of week 2 of the build season (this strict policy can largely be attributed to the fact that 2018 was a complete reset year for the team). Similar strict policies have held the team back in a number of other cases. The growth that I expect to see in 2019 will stem directly from a streamlined design review process, with more prototyping and a significantly more effective use of time. Gaining a few super-mentors and not graduating a single member of the driveteam, strategy/design team, and manufacturing team should also help the growth process…
Thanks again for this thread and the replies that have come in so far. It has been educational for me, and it makes me excited about the upcoming season.
Interesting timing, just heard a talk from Tom Wujec at my work that is highly relevant. One of the highlights is the Marshmallow Challenge, in which he has found after repeated exercises that Kindergarteners tend to outperform most educated adults. Teams who do not spend as much time on rigid organizational structures or creating management hierarchies and instead quickly iterate consistently perform the best. A great 6 minute watch if you have the time. FRC has similar time and resource constraints that would lead to better performance by this quick iteration culture, as others noted.
(Ironically my employer is a utility, one of the most conservative industries there is)
I think 1072 has had varying amounts of communication and coordination. We do manage to maintain part numbers and track progress of all the components of the robots, but we do not do much in regard to design reviews or any robot requirements. Do you (Adam/Mike/John) have basic design reviews / extensive requirements? I definitely see the pitfall in having too much structure and review in the design phase. How have you managed a balance?
I think it’s very interesting that the top 3 posters are all from teams that were recent captains or first picks from championship winning alliances and yet all share the same doubts. I can’t say I’m 100% surprised. It’s really a testament to the fact that there’s ALWAYS something you can improve on, and that FRC doesn’t come easy, not even to the best of teams out there.
That said, it really seems like the most common path to success lies at a structure just rigid enough that everyone knows roughly what they should be doing when, but not so rigid that inevitable setbacks (or occasional leaps in schedule) don’t throw a wrench in the entire operation.
I think Adam discovered design by committee doesn’t work. :] Actually if you want to get something stupid done, form a committee, but I digress. Some of the structures that Adam mentioned are absolutely required to large projects done, but FRC’s primary goal to to develop and inspire individuals. It seems ot be working.
I think I see this as traditional Waterfall project management vs Agile methodology.
Since Agile is focused around quick decisions, minimal viable products, and rapid iterations, I see it fitting into an elite team’s structure much more than a traditional Waterfall method of “Week 1 we MUST HAVE ALL CAD DONE!!!”
It makes sense to me, and I’m sure more teams would like to start working in a more Agile way, but team infrastructure and resources don’t allow for it (for example, lack of rapid prototyping through printing, laser cutting, and in house machining).
I can’t speak from an FRC perspective (as I haven’t been formally affiliated with a team in the last 7 years :yikes: ), but I can speak from an engineer’s perspective.
(Context: I’m a new projects engineer for a Toyota supplier. I develop equipment and run trials on such up until launch day. In a way it’s like a perpetual build season)
Here’s how projects go: Generally, Either I produce well running equipment that kicks butt but my project management suffers and I get yelled at/lousy review, or my project management is better but the resulting equipment kinda stinks (which also produces a lousy review and people yelling at me). Which is better? Don’t know. All I am told is “I want it all”. From a corporate perspective both are valuable and needed.
In a way, this is like with FRC. The former favors on-field awards/performance; the latter favors off-field performance. Which is better (or what blend is best) depends on what you want as a team and what you need as a team. People are different, and different people need differing amounts of structure to thrive.
I’ve recently been of the belief that process is for controlling the unpredictability of people.
If you have a group of individuals who are arbitrarily smart and arbitrarily good at communicating, there is not need for any established process. The team will simply “do its job” because everyone is exteremly capable of doing their jobs. No need to define a process that describes how to do the job.
When people left to their own devices tend to introduce chaos, that’s where a process might be able to help.
“Process” is often just a fancy term for “cat herding”.
The consensus seems to be that a rigid design process slows down the team, reduces flexibility, and is generally incompatible with the FRC challenge. Are there any specific practices that work well within FRC constraints (aside from flexibility to subteams, version control)? How do teams instead assess their progress and stay on schedule? Do similar considerations apply to build season planning more broadly?
Somewhat unrelated, but how do team track inventory, from purchasing to stocking the build space, to parts on the robot. Adam mentioned that Part number systems were a standard process among the elite teams, and my team is currently looking into revamping our inventory system.
Thanks everyone for sharing all this valuable information
I think he was mostly talking about numbering parts that they design (based on what he has talked about in the RAMP videos). If you have globally unique part numbers for each robot part you design you can better track what is moving through the pipeline.
We’ve developed what we call our robot indexing sheets - these are a system of google sheets with information about our stock, suppliers, and ordering systems that connect to our robot parts list to automatically keep track of each part from design conception to reception and completion. Here is an example of one for our current offseason robot: https://docs.google.com/spreadsheets/d/1uF9zjr3qXdZTNbHrPBlvzGT2d3ygHhuZCLU-qf8pEBY/edit?usp=sharing. It’s by no means perfect, but we’ve found it to be helpful so far. If there’s interest I can share a blank template without all of our parts on it.
EDIT: I’ve gotten enough people asking about a blank version that (https://docs.google.com/spreadsheets/d/1wUcsAZc_cofbk06fxqQ2XTBaOAWztatHOt6m8yq1Dpg/edit?usp=sharing). PM me with questions if you have any!
Thank you for the clarification!
Thanks for sharing this resource!
I’ll offer my experience with CAD:
Part number systems and some sort of tracker can be a godsend. We use a Google spreadsheet with a row for every part number, then columns with quantity per robot, expected manufacturing method, and then a series of Y/N columns such as approved for manufacturing, stock ordered, stock cut, and finished manufacturing. Typically it’s “owned” by one or two students who make sure that everything is on schedule.
With reviews, our rule is that at least one other person has to see a CAD for it to be shipped to fabrication. Typically, this review lasts 10-30 minutes and consists of:
- Drag the assembly through its range of motion
- Check for interferences
- Sanity check the numbers
- Find small, not-gonna-fly tolerance or strength issues
The design either passes (most of the time) or it fails, in which case you sit down for maybe 45 minutes and fix all the issues you can find. This helps with correctness and prevents stupid mistakes but can let you get through quickly.
I can’t say I’m surprised that the top teams are focused on individuals and key people making the important calls. While this isn’t the direction Adam was going, it really just leads me to ask:
(a) how are teams developing those individuals? (for teams that function how Corsetto described, where often students are those key people)
(b) does the focus on key personnel leave others out in the cold, or does the rising tide lift all boats?
Teamwork requires two things: common purpose and trust. If your trust is inherent in your relationships, you don’t need much formal structure or formal communications to hold it together. I thought I had something else to say, but now I think this covers it; formal structures are a means to build trust among “teams” which otherwise wouldn’t have it.
One thing that’s totally missing from the discussion above is the effect of the communication structure on the less skilled/exeperienced members of the team. A small core group that communicates effectively through informal means might come up with a good design. In fact, it might be more likely to come up with a great robot. But it totally fails the goal of “spreading STEM education” beyond the cohort that would have been engineers anyway. That’s not to say that formal design reviews are going to inspire freshmen, but at minimum you have to have a process that allows them to keep up with changes and makes them feel engaged.
The LigerBots have implemented several processes that we believe they are a net positive for everyone but are specifically aimed at supporting the rookies and less involved students.
- We break up the team into small brainstorming groups for the initial design (one of our coaches spends days considering the group assignments). *]We assign students to specific build groups that are supposed to have internal communications (we’re still experimenting with how rigid to make those assignments)
- We use color coded sticky notes for tasks on a Trello style board (maintained by a very dedicated mentor!).
- We use Slack for communication, with channels for each build group,
- We have daily status meeting at which group leaders speak but everyone can listen in to catch up.