Robot Design: If there isn't a vote, who makes the decisions?

[cadandcookies]

Recently, there was a post that really raised some questions in my mind about how people run their teams (mind you, not "they're doing it wrong" questions, but just simple curiosity).

Specifically these quotes:


"Voting is not an engineering process. We will never vote on a design."


And:


Voting presumes everyone gets the same say in something and that everyone is equally qualified to have a say in something. This is simply not the case. We find voting to be a popularity contest, not a logical decision making process, and therefore we don't do it. The problem is, you get kids voting for things they are not capable of carrying out.


My question is simple: in teams that do not vote on designs, how are decisions regarding designs made?

[ksafin]

It's a discussion. I know that's a very vague answer but that's simply how it is.

Voting, as mentioned, isn't a "logical" decision, but can be to some extent used as such. When we decide which route we take, we weigh the opinions of more veteran members and mentors more heavily than newer members. Likewise, Mechanical, Electrical, Software, and Administrative student officers have greater weight to their say.

It's basically like a government-esque decision making process for us. Each sub-team captain pays attention to what their team thinks, to what different factions prefer, and together the sub-team captains form some kind of "cabinet" of sorts where we make final executive decisions.

[BJT]

Early in the process we listen to ideas from everyone. The more experienced mentors and students then discuss until a consensus is formed on the basic idea. The mechanical team (myself and a few experienced students) then work out the details.

[magnets]

The programming lead, and mechanical leads plus the main mentor make up the "design team", who makes all executive decisions on design. What they say is the law for robot design. It really helps clear up conflicts when half way through build season each half of the team wishes to totally redesign the robot a different way.

[BrendanB]

Our team's design sub-team has been expanding over the years as more students and mentors gain more experience in learning more about game history, team history, and match strategy. We as a team analyze the game and come up with priority lists but the real core design team spends most of the first two weeks taking what we learn from kickoff & prototypes to develop our robot.

We have used voting system in the past but they don't always reflect the proper thought process that goes through analyzing the game, how its played, and more importantly how we should play it. Building off of the best robot concepts on the board can work to a degree, but you need a solid strategy to start with and pick the concepts that best compliment your strategy.

[Joe Johnson]

Team Governance issues have been the source of many conflicts withing FIRST teams over the years.

In my professional life, I can't recall having been involved in voting as a means of making a significant decision. At my work we take votes from time to time but the votes are mostly about giving everyone a say and letting everyone know where everyone else stands. The purpose of the vote is not to decide but rather to help the room come to a consensus.

Consensus is how the world of engineering works for the most part. The team debates and debates and debates until everyone gets to the point where they're okay with a certain path (even if it is not the path they would have taken if it were their decision alone).

So, that is how ALMOST all decisions are made but not ALL (at least wherever I have worked). Sometimes the sides cannot come to consensus. In these cases, there is always someone empowered to make the decision. Not everyone is happy, but generally, everyone accepts the ruling because the structure of the team is understood and this is the way the team gets to move on to the next problem.

Back to FIRST. It is my experience that teams that try to come to consensus but back that up with a clearly identified "King of the Machine" (my name for this ultimate decider if consensus building failed) are more effective than any of the alternatives.

Joe J.

[Akash Rastogi]

Hey Nick,

Great question and I'm glad this is something being asked before the 2014 season. I believe the best way to learn about other decision making and consensus generating techniques is to read JVN's build journal from 2010.

http://www.chiefdelphi.com/media/papers/2360

Best of luck!

-Akash

.

[MechEng83]

When I first joined our team, voting was the process to make every decision, from which regional to go to, to what architecture we went with. There were dichotomies between factions, who took it upon themselves to ignore the vote and do whatever they wanted. There was never a final authority to make decisions, which (imho) led to a diffusion of responsibility so it ended up being "everyone's fault" as well as "no one's fault" for making a disadvantageous choice.

We have since moved to a small core group of mentors and student leaders to make design decisions and have 1 student in charge of the robot, and 1 mentor who oversees the robot design/construction. About the only vote we still do is on the robot name, which can be quite a process. We're still making adjustments and tweaks to how the team operates, but for us, it's working.

[tim-tim]

There are several tools out there to help make better decisions. Some of the these tools are quick and simple, while others appear a bit more complicated but really aren't.

Some of the tools that we use, to varying degrees, are a Weighted Decision Matrix (http://www.weighteddecision.com/), Decision Tree (http://en.wikipedia.org/wiki/Decision_tree), and QFD House of Quality Matrix (http://www.youtube.com/watch?v=tO6gmWX-cp4).

The weighted decision matrix is the simplest and easiest to do, even in large groups. Another variation is to compare ideas directly against each other following the same template. For example: a group develops 3 ideas: A, B, and C with selection criterion of criteria 1, criteria 2, criteria 3, etc. To start you would define a control idea, we will choose Idea B. Then go through each of the criteria and compare Idea A to Idea B, and then Idea C to Idea B. Tally up the columns, this will tell you what ideas are better/worse than Idea B. Repeat this process by selecting Idea A or C as a control. By doing this you will know if Idea A is better/worse than Idea C.

A decision tree is very helpful at a high level and determining how to play the game. It can be used at more specific levels, but we use this as a guide to walking through the initial strategy talks and how we will play the game. It takes a decent amount of organization, but is a great way to be exposed to a lot of scenarios.

A House of Quality is probably the most technical of all the decision tools we use. It is relatively simple to use, but looks very complicated and overwhelming at a first glance. This is something I was taught my freshman year in college and have really benefited from using the tool.

There are many free and readily available templates, how to's, and examples for each of these methods on the web. I am glad to help if you need it. Really learning the design process and these tools will start to make your brain develop a "better" problem solving process.

[Anupam Goli]

In an ideal world, every design idea that comes up is put to the test and the superior design will show its superior performance, ending any debate on others that do not show similar performance during a test. That being said, not every FRC team has the resources to rigorously test ideas to the end and see which ones are best. As previously mentioned, weighted design matrix tables, mathematical and physical calculation, and proof of concepts can be used to eliminate certain designs from the pool or promote prototyping of other designs.

It's okay to prototype more than one mechanism, and as the prototypes get more and more refined, you'll start to see the differences, and it may be enough for your team to cast their decision one way or another. Voting is the absolute last resort thing you do at 11:59pm on decision day, and even then it sometimes isn't the best answer!

[cadandcookies]

Hey Nick,

Great question and I'm glad this is something being asked before the 2014 season. I believe the best way to learn about other decision making and consensus generating techniques is to read JVN's build journal from 2010.

http://www.chiefdelphi.com/media/papers/2360

Best of luck!

-Akash

.

Thank you, this was a very interesting and informative read. I think remember reading something similar from 2009.

There have been some great and informative answers so far-- I think it's interesting how teams seem to gravitate towards the "consensus" decision method.

On a personal note, this thread has vindicated a lot of the ideas I had floating around about running a decision making process.

I figure I'll also share how my team is planning to approach design this year-- I'm from a very large team (70+ students and 30+ mentors), so getting together the entire team for discussion isn't exactly a good idea (trust me, we've tried- and failed), so we're splitting into small groups that each have a "lead" student. These groups basically go through the process of selecting a strategy individually-- with emphasis on using predictive math (expected values, etc) and logical decision making (decision matrices, QFD). After they've come to individual consensus, we gather up all the leads, who do essentially the same thing, but taking input and calculations from all the individual subteams. The idea is to cast our net as wide as possible while still keeping it manageable with so many people.

[Akash Rastogi]

Thank you, this was a very interesting and informative read. I think remember reading something similar from 2009.

There have been some great and informative answers so far-- I think it's interesting how teams seem to gravitate towards the "consensus" decision method.

On a personal note, this thread has vindicated a lot of the ideas I had floating around about running a decision making process.

I figure I'll also share how my team is planning to approach design this year-- I'm from a very large team (70+ students and 30+ mentors), so getting together the entire team for discussion isn't exactly a good idea (trust me, we've tried- and failed), so we're splitting into small groups that each have a "lead" student. These groups basically go through the process of selecting a strategy individually-- with emphasis on using predictive math (expected values, etc) and logical decision making (decision matrices, QFD). After they've come to individual consensus, we gather up all the leads, who do essentially the same thing, but taking input and calculations from all the individual subteams. The idea is to cast our net as wide as possible while still keeping it manageable with so many people.

I'm from the school of thought that, while everyone from the team should have a chance at giving their input on concepts and what to prototype, the cold hard decision making portion ultimately falls on the core leadership group of mentors and (usually) upperclassmen subteam captains, or a Design Team. This smaller core group is the one that debates strategies, shows off prototypes, and fairly evaluates each one. On a large team, this is a lot harder.

Like you said, gathering team leads is a good idea, but sometimes you ought to let one or two others that the lead picks out from his/her group join the discussion as well.

Lots of ways to go about doing this, and a lot of it depends on your team structure. Always good to evaluate if your large team is structured in a way that might even allow the use of some of the techniques JVN discusses.

[Oblarg]

It's basically like a government-esque decision making process for us. Each sub-team captain pays attention to what their team thinks, to what different factions prefer, and together the sub-team captains form some kind of "cabinet" of sorts where we make final executive decisions.

This is quite similar to the method 449 adopted last year, which worked reasonably well. We have a large group discussion in which everyone can speak their minds, and then a smaller discussion amongst team leadership where a consensus is reached.

In previous years (read: when I was a student), we had the whole team vote during downselect meetings. It was a nightmare. I hope to never do that again.

[seg9585]

Trade studies!!
Always use tools like the weighted design matrix (or figure of merit) to justify the designs. If there's not a clear winner out of this method, you can always add more parameters or, if you have the resources, build both systems and make it modular.
Last year my team made 2 completely different variations of a climber, and we kept it relatively modular so it can be swapped out. It turned out we used one design in the first regional, the other in the second, and tossed both out in favor of a 3rd for championship. That's how these things go, but it should never just come down to a popularity contest.

[TikiTech]

Since the kick off is in the early morning here in Hawaii. 5am I believe, we use this as one of our team bonding experiences. It is our "camp over kick off party". A fun night of food and many rounds of our various divergent thinking games (ask us about them sometime), and hopefully some sleep in there too...

We will spend the day using the Stanford Design Thinking process to produce as many possible design ideas. This process allows everyone to share as many ideas as they can, some reasonable and some not. It is a good time for out of the box designs.

For more information on our use of Design Thinking check our website..

http://kealakeherobotics.org/student-experience/stanford-design-thinking/

Some of our divergent thinking games are on the website as well.

Look around but please be patient as the site is still under construction..are they ever not?


By the end of the day we all have a understanding what we need to produce as a group.


As a mentor that believes that inspiration is what FRC is loudest about, this design process really allows the students to "own" the design as a team. This ownership creates a drive and passion that is easy to see in every one of our members.


Aloha!



One of our coaches and now a recently graduated member are Stanford D School instructors and travel the state doing workshops.
Let me know if you are interested in more information, we love to share.. :)

[weberr]

Decisions need to be based on measurable accurate data obtained; not voting, a dart board, human feelings, etc.

Our team works very hard to develop QFD (QUality Functional Development) in the first days so that we have a "metric" to compare all designs to and thus by measuring the designs back to the QFD, we can develop a ranking of the solutions. If possible, we combine best attributes from multiple designs and that in turn yields a better overall design.

Since there is no emotion involved, then we reduce hurt feelings and focus more on the design. Remember, you want the most impact with the littlest possible amount of effort.

http://www.theflyingtoasters.org/#!__resources/vstc79=qfd

This year we are adding a component called Probably Scoring Rate as part of the QFD.

[Taylor]

Voting for designs is a bad idea. Voting usually promotes what is cool or funny, not what is best for the team.
Like Mr. Cool said, voting leads to animosity, aggravation, and an adversarial environment.
In my experience, deep, thoughtful discussions, usually aided by a version of JVN's WOT, lead to the correct-for-the-team solution making itself obvious.

[Anthony4004]

My team simply discusses the designs we want to try, not vote. Mainly because you get students who simply want to try something because either they thought of it, or they are backing their friend's idea. We instead try to implement the "Engineering Design Process" where we find the pros and cons of designs, not simply oooh shiny......
Plus there are team members who will actually have to build it, so I try to think about the ease, the area the mechanism will take up in the robot, and how well it will work to our advantage at competition.

[BigJ]

In the past, we have used a weighted matrix to evaluate options, but in the end discuss and decide between the top options from the matrix separately based on our capabilities and the complexities of the systems.

[Jon Stratis]

In our handbook we describe out method of choosing captains very clearly: The mentors choose, with input from the students. That input comes in the form of voting. Making design decisions is much the same way.

We empower our captains to make the overall decision of the robot. This occurs with after doing a cost/benefit analysis of all the ideas, eliminating those with too much cost for too little benefit, and allowing the team to vote on what they like best to help give the captains some guidance. They then empower design leaders for each mechanism to make the decisions necessary, which are often reached after discussion (I can't really call it voting if there are only 2-3 people involved!) with the rest of the design team for that mechanism.

[Chris is me]

It's a lot easier to come to a consensus as a team if you make sure you're all starting from the same premise. If everyone on the team is dedicated to building a simple, effective robot that plays a particular strategy, everyone will converge on the optimal solution based on prototyping. We have formally had a six-member "design committee" (half students, half mentors) that picks the direction of the robot, but really it's just kind of a natural process that comes out of prototyping and vision from the lead design people on the team. We had a poor year for robot design when we voted in 2010. We stopped voting and haven't looked back, and with adequate pre-season training no decision has been controversial anyway.

[DELurker]

My question is simple: in teams that do not vote on designs, how are decisions regarding designs made?

This link (link) (http://roboraiders.com/sop2009/SOP-Kickoff-Days-I--IV.pdf) is to Team 75's kickoff week SOP. On page 7, they show how they use a weighted decision matrix to rate a given design.

Our team adopted and adapted 75's method for the 2013 season and we were surprised by two things: 1) A lot of the ratings showed a definite benefit of one choice over another, and 2) Two equally capable design concepts got an equal rating (within 1 point out of 215).

[Oblarg]

I think it's worth noting that while a decision matrix is a very good guide to help a design team come to a consensus, it's probably unwise to take it as an absolute authority on which design "wins." The numbers generated always have some degree of arbitrariness, and so you should be wary of appealing to "well, this one has a higher score" in the place of detailed discussion.

[seg9585]

I think it's worth noting that while a decision matrix is a very good guide to help a design team come to a consensus, it's probably unwise to take it as an absolute authority on which design "wins." The numbers generated always have some degree of arbitrariness, and so you should be wary of appealing to "well, this one has a higher score" in the place of detailed discussion.

Numbers in a design matrix should not be very arbitrary. If you can clearly rank attributes of designs then do so, if not then mark them with the same value. Establish a tolerance level (ie 50 pts +/- 4) and anything within that tolerance is a valid design which may be prototyped/implemented. Pick one of these or add a tiebreaker attribute.

[Jon Stratis]

It's a lot easier to come to a consensus as a team if you make sure you're all starting from the same premise. If everyone on the team is dedicated to building a simple, effective robot that plays a particular strategy, everyone will converge on the optimal solution based on prototyping. We have formally had a six-member "design committee" (half students, half mentors) that picks the direction of the robot, but really it's just kind of a natural process that comes out of prototyping and vision from the lead design people on the team. We had a poor year for robot design when we voted in 2010. We stopped voting and haven't looked back, and with adequate pre-season training no decision has been controversial anyway.

Your post reminded me of something one of my profs in grad school said... Individuals are great at finding a local max in a given solution space, but groups are much better at finding the global max. In other words, individuals might each find different solutions to a problem that can't be incrementally improved on (local max, where any change would decrease the quality of the solution), but those solutions may not be the absolute optimal solution to the problem (the global max, where there is no better solution to the problem).

[Jhultink]

On our team we have 4 student presidents: 3 executive presidents, and a "normal president." The three executive presidents have a vote, but the "normal president" (me) does not. The other three all have at least one or two more years of experience than I, so I think it's fair. So far we have always come to a common consensus and have never actually had to vote on anything. I personally like our system.

[Oblarg]

Numbers in a design matrix should not be very arbitrary. If you can clearly rank attributes of designs then do so, if not then mark them with the same value. Establish a tolerance level (ie 50 pts +/- 4) and anything within that tolerance is a valid design which may be prototyped/implemented. Pick one of these or add a tiebreaker attribute.

The problem is that many of the attributes you wish to compare are not easily quantifiable. "Reliability" is not an element of a metric space. It is useful to attempt to quantify these so you can do a rough estimation of a utility calculation, but I would never take that as anything more than a guide for discussion.

[MechEng83]

The problem is that many of the attributes you wish to compare are not easily quantifiable. "Reliability" is not an element of a metric space. It is useful to attempt to quantify these so you can do a rough estimation of a utility calculation, but I would never take that as anything more than a guide for discussion.

At my company, we have an entire function called "Reliability Engineering" There are ways to quantify reliability in terms of RPH (repairs per hundred) CPE (cost per engine - we make engines, but it could be CPR - cost per robot), and MTBF (mean time between failure). Reliability is very much a metric in industrial settings. Decisions are made and resources are applied based in part on the reliability numbers.

An FRC robot isn't going to have hundreds of like units to generate RPH, but the other items are real things that each team should consider.

[Oblarg]

At my company, we have an entire function called "Reliability Engineering" There are ways to quantify reliability in terms of RPH (repairs per hundred) CPE (cost per engine - we make engines, but it could be CPR - cost per robot), and MTBF (mean time between failure). Reliability is very much a metric in industrial settings. Decisions are made and resources are applied based in part on the reliability numbers.

An FRC robot isn't going to have hundreds of like units to generate RPH, but the other items are real things that each team should consider.

I should have qualified my post with "in the context of FRC," but I think the point still stands. In my experience, much of the rating in our design matrices essentially comes down to intuition, and anecote because we have no practical way to quantify many of the important attributes (for all of the metrics you've mentioned, I can't think of any for which I'd have sufficient data). They're still useful for facilitating the decision-making process, but they're far too uncertain to rely on in the place of in-depth discussion.