Design and Communication

FIRST General Forum
1

Leading into, and coming out of the, the 2018 season I had a nagging feeling that 973 was far, far behind other good teams in terms of how structured and organized our communication of design during season was (both within the design team, and to the rest of the team).

So I surveyed a good deal of key members on elite teams, and what I found was pretty surprising.

The closest thing most had to a structured communication process popped up in some or all of the following;

-Part number systems (that maybe also track fabrication)
-Github, etc… for code
-Version control on CAD
-Trello/spreadsheet/etc… that are sort of used by subteams until they aren’t at some point in season
-General use of slack (or similar)
-Ad hoc discussions between key students and mentors (but rarely the entire team or subteam)
-Misc updates to rest of team verbally, at meetings,via email, and/or slack

This was pretty surprising to me… So I started surveying some teams that seem to have a good institution and make quality robots, but definitely don’t have an elite culture. These teams seem more likely to rigidly be using gant charts, trello, and sometimes actual industry processes like milestone blocking concept design review, preliminary design review and critical design review (and potentially not allowing progress until a group, potentially including team outsiders, approve the design).

That being said there were definitely blue collar and middle of the pack teams that don’t have the above rigid processes… but I only seemed to find the highly structured or rigid processes in teams not in the top pack.

It really seems like most of the top teams are focused on individuals operating well with others in a flexible approach focused purely on results (and with a wide range of allowable processes to get there).

We will definitely be improving our communication in some areas based on some low overhead ideas we heard, but we will not be implementing anything large or rigid.

This is definitely not a scientific study, and I’m sure it’s choc full of confirmation bias, but I figured others might find this information interesting, helpful, distracting and/or useful.

2

Adam emailed me as part of his “survey” and asked me about how 148 operates. His very pointed questions gave me a serious sense of imposter syndrome.

“Jeez, we don’t really do any of the stuff he’s asking about. I’m almost humiliated to describe what we do. I don’t know how we’re going to be able to keep up with these California teams if this is the sort of rigorous process they’re running…”

When he reassured me we’re not the outlier, it was surprising (and comforting).

I’m now even more curious what others think about this. Where do you find that balance between flexibility and formality for an FRC team?

-JVN

3

Great post Adam!

From my perspective, 1678 would have to sacrifice competitive edge to follow a “rigid” task tracking and design development process. The main reason I believe this: our students drive the nuts and bolts work of making a world-class robot.

In a company with formal processes, you have adults, that have sometimes been at a job for decades, getting into a rhythm of stage gates, design reviews, milestones, etc.

In FRC, we have key contributors that have been a part of FRC for a maximum of three years. They don’t have the time or bandwidth to go through lots of red tape and structured processes. On 1678, we allow our Subteam Leads to set up a communication structure that works for them and gets the job done. As long as they are meeting goals, I won’t bother them to fit some mold that myself or the team has set up.

I echo JVN’s sentiment. I always wonder if we’re approaching the FRC challenge the “right way”. But things seem to work, and sometimes you just gotta go with your gut :slight_smile:

-Mike

4

I’ve always found that programs like VEX EDR, VEX IQ, FTC, etc. seem to allow for a lot more structured approaches to going about design and development of robots. I feel like I generally see students going through a more tedious and thought out design process in VEX for example, then in FRC. Not to say that inherently makes VEX a better educational program, just one of the trade-offs in the styles of the competitions.

For me, I feel like the biggest factor driving the lack of structure for teams looking to compete at a high level is just that there is not enough time to keep to rigorous structure.

That said, things like part tracking systems, organized GrabCAD and GitHub repositories, etc. definitely help, and should be easier to implement if set up well as there’s not a lot of back-end active management needed if done well.

5

I can only speak from personal experience, but early on in 3512’s history (2012-2014), we had a very rigid and long design/prototyping review process that lasted sometimes into week 3. It wasn’t uncommon for the finished robot to not be done until mere days before bag up, leaving little to no time for programming and even less for driver training/practice. Of course, in those days we also didn’t build a practice robot.

Since then, we’ve relaxed quite a lot, and focused on borrowing tried and true methods from the best teams in FIRST, and making simpler but still effective prototypes for everything else. I believe being more flexible early and not being bogged down with rigorous design reviews especially in mid/late build season on has served us well the past few years, especially when combined with making that second practice robot. Sometimes I feel like we flying by the seat of our pants, but idk… it all just comes together in the end.

One thing that has helped us was better communication between subteams, particularly between mechanical and electrical, CAD and electrical, and mechanical and Programming. We have opening and closing meetings where each subteam gives progress updates and the definitely helps with that, as well as giving everyone else a sense of what’s going on.

I don’t think there’s one right way to do things, sometimes you just have to try different things until you find something that works for your team.

6

…all of which isn’t terribly surprising to me, as FIRST rewards building an awesome functional prototype or three extremely quickly, and our design failures are extremely low cost. (There’s strong similarities to building software.)

Most of the engineering controls and thorough design communications implemented at medium- to large-size companies are designed to prevent (or fix) multi-million dollar hardware design failures, at the cost of agility. Agility is king in FIRST.

When I visited the shop of a high-performing team, I noticed that there was a key leader for final design & robot integration, who students from all the subteams would bounce progress off of for immediate guidance & steering. That leader would work with just the key students who owned each interface or problem area to describe the problem and ask them for solutions. There was a lot of “yeah but can this be better?” and a lot of students answering that call and raising the bar for their designs. The leader was a whirlwind of energy bouncing between subteams and keeping the entire team focused on achieving system objectives by creating and enforcing subsystem requirements, without ever having to stop and communicate the whole design from scratch again the way formal design reviews with outside approvers can go, and never taking design control or even design momentum away from the students who were executing on subsystems.*

That integration process was separate from part manufacturing (different leader), and separate from strategy and conceptual prototypes (different leader).

Something that holds 841 back right now is that we don’t have a single leader like that who is only executing on subsystem integration and system performance. My lead mentor and I both sometimes step into that role**, but both of us also get pulled into manufacturing/part execution and both of us also get pulled into moonshots and both of us get pulled into team logistics - neither of us spends all of our time making sure that students can see, understand, and fix system-level problems.

(*Fastest way possible to destroy morale, [strike]don’t[/strike] ask me how I know)
(**We’ve had a student do it in the past, too)

7

If you don’t mind, how do you know? I don’t want the same thing to happen to my mind so being to spot the warning signs would be helpful.

8

While it is surprising in some ways, these observations match both my professional and robotics lives. Formality is the enemy of fast. It’s a fact of life I’ve seen proven out over and over.

I think a lot of teams who build quality robots have figured out the overall process to being successful relies on failing fast, failing fast again and being flexible, in that order. ~10 or so years ago, 125 had a fairly rigid process, with sponsor design reviews, gantt charts, etc. It wasn’t very effective and really bogged us down as we tried to hit each ‘checkpoint’ on our way to a good robot.

Think of the scrappy start-up versus the established defense contractor. They both have their places in the world - especially when consumer/user safety is on the line.

The reason the ‘scrappy start up’ process likely wins out in FRC is because the time constraint is so severe. Failing frequently helps you get to something workable faster. That being said, I think a ‘scrappy start up’ team needs a wealth of wisdom/guidance from a group of mentors to ultimately be highly successful - as it helps to use the rapid iteration/failure to be as efficient as possible (ie: not wasting failure cycles on less than stellar concepts or ideas).

-Brando

9

By making mistakes as a mentor. In the past, I’ve “given design feedback” as a mentor without being careful of power dynamics. That “giving design feedback” became just taking control of the design away from the students working on the robot, and in the process I accidentally took a lot of the wind out of our sails on student engagement in that year’s robot.
(I can actually think of at least two episodes in this archetype (several years apart ::ouch:: ) that I’ve been involved in. Mostly happens when I get too caught up geeking out about robot details and forget my purpose as a mentor is to enable students.)

Also:

+1! Making sure the students pick up more lessons from fewer failures is a key mentor role.

10

Thanks for doing this Adam, the results are really surprising to me even though they shouldn’t be. I’ve always thought the 973/148/125/1678’s of the world had FRC down to a science with a rigorous and proven design process. Even when reading JVN’s Build Blog, I always thought that there must be some sort of magic structure/formula underneath all the “fake it 'till you make it” style that’s so eloquently described in the blog. I unabashedly parrot “fail faster” every chance I get when working with a team that is trying to get better…Basically I thought that there must be some magic formula that leads to consistent success at the highest levels, because how else would that level of sustained success be possible?

When I compare this line of thinking with my actual experiences within FRC, it doesn’t hold up. Arguably the best robots I’ve helped to design/build have come out of Ri3D… which I realize is kind of sad, but when I think about it, it makes a little sense. Ri3D is the ultimate when it comes to free flowing ideas and failing fast. There is no time for bureaucracy and there certainly isn’t time to implement a structured plan. I tried following a strict plan my first year of Ri3D and you can watch our videos from that year to see how that went… but actually please don’t, they’ll make your eyes bleed (it doesn’t help that the “game” was Recycle Rush).

Additionally, my personal experience with successful teams from my region seem to follow the same design/communication principles that have proven successful for the teams listed above. I can’t speak to internal team communications/structure specifically, but 4539 “finishes” (of course FRC robots are never actually done. I just mean done to the point where they could win a week 1 event) their robot by the 4th week of the build season. They fail so much faster than other teams in Minnesota and I mean that in the most complimentary way. I’m sure many other successful teams will have a similar story.

On 4607, the team has gone through many different phases, as I would call them, when it comes to design and team communication. The team has used Slack incredibly effectively for communication in recent years. The team Github is becoming more and more of an asset as documentation has gotten significantly better lately thanks in no small part to some amazing new mentors. When it comes to failure documentation, I haven’t seen a better implemented FMEA system from an FRC team, and that will only improve as historical data becomes more accurate and better tracked. The FMEA allows the team to make data-driven improvements to the robot without bogging down speed/creativity with a rigid design implementation structure. I think it strikes a fine balance between structure and speed which proves to be a net positive for the team.

The Outreach team on 4607 has significant amount of structure built into it, but unlike the design/robot side of an FRC team, I believe this is a very positive thing. The structure enables a more systematic approach to culture change, which is goal driven and well documented. Without this infrastructure, the team would not be nearly as effective in this area in my opinion.

An area where I think there will be significant growth for 2019 relates to the point this thread is making. 4607 does a lot of things right, but one thing that has eluded the team has been a well implemented design process. Some years, the design group has been too isolated with not enough new ideas. Some years there are too many ideas and not enough prototypes. Actually… there are never enough prototypes. A statement which probably rings true for most teams in most years (except 148 by the sounds of it). Last year for example (arguably 4607’s most successful year), a strict design review process prevented the team from completing the drivetrain design until the end of week 2 of the build season (this strict policy can largely be attributed to the fact that 2018 was a complete reset year for the team). Similar strict policies have held the team back in a number of other cases. The growth that I expect to see in 2019 will stem directly from a streamlined design review process, with more prototyping and a significantly more effective use of time. Gaining a few super-mentors and not graduating a single member of the driveteam, strategy/design team, and manufacturing team should also help the growth process…

Thanks again for this thread and the replies that have come in so far. It has been educational for me, and it makes me excited about the upcoming season.

11

Interesting timing, just heard a talk from Tom Wujec at my work that is highly relevant. One of the highlights is the Marshmallow Challenge, in which he has found after repeated exercises that Kindergarteners tend to outperform most educated adults. Teams who do not spend as much time on rigid organizational structures or creating management hierarchies and instead quickly iterate consistently perform the best. A great 6 minute watch if you have the time. FRC has similar time and resource constraints that would lead to better performance by this quick iteration culture, as others noted.

(Ironically my employer is a utility, one of the most conservative industries there is)

12

I think 1072 has had varying amounts of communication and coordination. We do manage to maintain part numbers and track progress of all the components of the robots, but we do not do much in regard to design reviews or any robot requirements. Do you (Adam/Mike/John) have basic design reviews / extensive requirements? I definitely see the pitfall in having too much structure and review in the design phase. How have you managed a balance?

13

I think it’s very interesting that the top 3 posters are all from teams that were recent captains or first picks from championship winning alliances and yet all share the same doubts. I can’t say I’m 100% surprised. It’s really a testament to the fact that there’s ALWAYS something you can improve on, and that FRC doesn’t come easy, not even to the best of teams out there.

That said, it really seems like the most common path to success lies at a structure just rigid enough that everyone knows roughly what they should be doing when, but not so rigid that inevitable setbacks (or occasional leaps in schedule) don’t throw a wrench in the entire operation.

14

I think Adam discovered design by committee doesn’t work. :] Actually if you want to get something stupid done, form a committee, but I digress. Some of the structures that Adam mentioned are absolutely required to large projects done, but FRC’s primary goal to to develop and inspire individuals. It seems ot be working.

15

I think I see this as traditional Waterfall project management vs Agile methodology.

Since Agile is focused around quick decisions, minimal viable products, and rapid iterations, I see it fitting into an elite team’s structure much more than a traditional Waterfall method of “Week 1 we MUST HAVE ALL CAD DONE!!!”

It makes sense to me, and I’m sure more teams would like to start working in a more Agile way, but team infrastructure and resources don’t allow for it (for example, lack of rapid prototyping through printing, laser cutting, and in house machining).

16

I can’t speak from an FRC perspective (as I haven’t been formally affiliated with a team in the last 7 years :yikes: ), but I can speak from an engineer’s perspective.

(Context: I’m a new projects engineer for a Toyota supplier. I develop equipment and run trials on such up until launch day. In a way it’s like a perpetual build season)

Here’s how projects go: Generally, Either I produce well running equipment that kicks butt but my project management suffers and I get yelled at/lousy review, or my project management is better but the resulting equipment kinda stinks (which also produces a lousy review and people yelling at me). Which is better? Don’t know. All I am told is “I want it all”. From a corporate perspective both are valuable and needed.

In a way, this is like with FRC. The former favors on-field awards/performance; the latter favors off-field performance. Which is better (or what blend is best) depends on what you want as a team and what you need as a team. People are different, and different people need differing amounts of structure to thrive.

17

I’ve recently been of the belief that process is for controlling the unpredictability of people.

If you have a group of individuals who are arbitrarily smart and arbitrarily good at communicating, there is not need for any established process. The team will simply “do its job” because everyone is exteremly capable of doing their jobs. No need to define a process that describes how to do the job.

When people left to their own devices tend to introduce chaos, that’s where a process might be able to help.

“Process” is often just a fancy term for “cat herding”.

18

The consensus seems to be that a rigid design process slows down the team, reduces flexibility, and is generally incompatible with the FRC challenge. Are there any specific practices that work well within FRC constraints (aside from flexibility to subteams, version control)? How do teams instead assess their progress and stay on schedule? Do similar considerations apply to build season planning more broadly?

Somewhat unrelated, but how do team track inventory, from purchasing to stocking the build space, to parts on the robot. Adam mentioned that Part number systems were a standard process among the elite teams, and my team is currently looking into revamping our inventory system.

Thanks everyone for sharing all this valuable information :slight_smile:

19

I think he was mostly talking about numbering parts that they design (based on what he has talked about in the RAMP videos). If you have globally unique part numbers for each robot part you design you can better track what is moving through the pipeline.

20

We’ve developed what we call our robot indexing sheets - these are a system of google sheets with information about our stock, suppliers, and ordering systems that connect to our robot parts list to automatically keep track of each part from design conception to reception and completion. Here is an example of one for our current offseason robot: https://docs.google.com/spreadsheets/d/1uF9zjr3qXdZTNbHrPBlvzGT2d3ygHhuZCLU-qf8pEBY/edit?usp=sharing. It’s by no means perfect, but we’ve found it to be helpful so far. If there’s interest I can share a blank template without all of our parts on it.

EDIT: I’ve gotten enough people asking about a blank version that (Blank 299 Indexing Template - Google Sheets). PM me with questions if you have any!