How do teams distinguish between a “Proof of Concept” vs “Optimization” prototype is? Or am I thinking of it too rigidly? Is this distinction even necessary since you generally have an idea of what kind of mechanism would work for any given game piece, so you’d really only Proof-of-Concept your most unique or unverified ideas?
Currently, I’m looking at 3847 and @AllenGregoryIV 's presentation on prototyping, and I’m not exactly clear at what point a concept is “proven” and when you’re really just changing parameters around. How does each concept get distinguished from one another — for example, a 2019-style intake with top and bottom rollers vs top and bottom wheels, is that an optimization of the same “powered top and bottom intake” concept? What happens if we stop powering one roller?
And going off of that, how do you know if a concept is “disproven” vs the prototype or tests didn’t accurately represent the concept? Is that just a workmanship/team standards type thing? I guess that’s my main fear, rather than just trying to draw arbitrary lines in an iterative process — I don’t want to end up in a scenario where we’re throwing out concepts unfairly, but I also would like to clear up some of the processes we have surrounding prototyping, especially since we have a ton of inexperienced students this year.
The first prototypes shown in these videos are Proof of Concepts
The subsequent prototypes shown in these videos are examples of prototype optimization
Allen defines a prototype as “anything we build that isn’t intended to be on the final robot.”
Triple Helix (the 2nd video in @Lil_Lavery 's post above, thanks Sean) takes the perspective that there really is no final robot. No part or mechanism is sacred; anything is subject to removal and replacement with a better version. The robot we will deploy into each competition represents simply the best system we’ve assembled to date. [1]
So what, then, is a “prototype” for Triple Helix? [2] We use prototyping as a tool to choose between two equally promising, but mutually exclusive design paths (e.g. wheeled shooter vs a catapult). Once a single path is selected, then it’s no longer a prototype, it’s just the robot, and iterative development continues as normal.
If you take this approach, then “prototypes” should always appear in sets of 2 or more, representing those possible development paths. Then you don’t have to worry about “proving” or “disproving” the concepts. The prototypes don’t even need to be good! You just need to select the one that seems like it’ll be more fruitful, down the line.
.
[1] This Ship-of-Theseus approach is different than how many other teams approach the FRC challenge… And it has a bunch of implications for design, logistics, and aesthetics. For example, note how Triple Helix robots are unpainted (some tubes even still have the material spec printed on the side)! This is because every component is provisional, not because we don’t have pride in our work.
[2] You could say that the entire robot is a prototype. That definition doesn’t really have much utility / explaining power.
First, all of those terms/distinctions in that presentation are just made up, made up for purpose but made up just the same. If they aren’t helping you and your team don’t use them. During the season we don’t think about exactly which category of prototype we are going to build next, we just think of what we need to build next to try to learn more about the problems we are trying to solve that season. The different prototype categories were there to help explain and categorize all of the various things we have used through the years to help us in the design process.
Neither amAndyMark I most of the time and that’s part of the process. How much quick iteration you can do and how functional you can make your testing equal what you will see during matches can make you more confident something will work but you’llLimelight, an integrated vision coprocessor never be 100% something is proven and works effectively until it makes it through a season. Like Nate said above even once the “prototypes” are finished you should still be learning from your and iterating through out the season. The prototypes in my opinion are just there to get you down down that path faster and help you choose which path you believe is best to follow.
No matter how much prototyping you do, your team will still have to make hard choices. This is where experience comes in to play. Deciding the best path to take in a new FRCFIRST Robotics Competition game is largely going to be based on your team’s understanding of your resources, understanding of that year’s challenge and can be informed by all the past FRCFIRST Robotics Competition challenges and what designs have been successful in those seasons. There are numerous ways to FRCFIRST Robotics Competition in a correct way. Any way that works for your team and allows your team to be successful in whatever ways you define success. However, there are countless ways to do FRCFIRST Robotics Competition in an incorrect way, ways that don’t allow you to be successful. So you have to use your experience and the experience of others to make the choices that your team believes will bring you the most success.
Building off the quick response I gave prior to heading to work.
Here’s a presentation I’ve given on Prototyping to my team and other teams in the past.
A “Proof of Concept” is a prototype that well, proves that your concept is viable. The intention to help your designers and decision makers come to an informed decision regarding whether or not more effort should be put into that concept, or if other alternatives are more promising. A Proof of Concept DOES NOT have to be full scale, made of the right materials, interact with the official game piece, or powered by FRC-legal motors. Proof of Concepts might be made out of wood, cardboard, 80/20 extrusion, PVC, or even LEGO. Whatever makes the most sense for the particular concept that needs to be proven. It’s often impractacle to prototype full-scale climbing or end-game devices, for instance, so teams will use smaller mock-ups to demonstrate the motions or packaging they want to achieve. Here’s an example of Proof of Concepts made out of LEGO.
This isn’t to say you can’t use more “real world” materials in a Proof of Concept. The videos I posted earlier in the thread have examples closer to real FRC appliations.
An “optimization prototype” is a step or two further in the process, once you’ve already down selected to one or two concepts you’re invested in. The purpose of this type of prototyping is to determine your key parameters, and feed that input to your robot designers. These prototypes discover the information that answers the questions your CADers and fabricators have. “How far should these intake wheels be apart?” “How far off the ground should the intake be?” “What wheels are we using on these rollers?” “What size flywheel are we using?” “How much compression between the flywheel and the hood?” All of these types of questions are what these prototypes are seeking to answer. This is repeated, iterative, documented testing. Try to adjust one parameter at a time, and find the sweet spots for each parameter. Then tell your designers those results.
The purpose of a prototype is to learn something. It helps tremendously to explicitly define your questions before you start building.
If the driving question of your prototype is “is this possible at all” (for example, the team a couple years ago that tried to jump to the hanging bar; or teams that lifted both their alliance partners in 2019), you’re going to build a proof-of-concept prototype. As Lil_Lavery said, these are fast and dirty checks on things like geometry, packaging, how materials interact with each other, how hard it is to balance with certain weight distributions.
If you’re using a tried-and-true type of FRC/FTC mechanism like an arm, claw, elevator, wheeled intake, flywheel shooter, etc and your driving questions for the prototype are things like what type of wheels should we use, how close to the ground should it be, how far should the sides be angled, then you’re going to build an optimization prototype.
If your driving questions are things like “where does the pivot point need to be so that the arm can fit in the starting config and also reach the goal”, or “will a gearbox strong enough to lift this mechanism be unbearably slow”, consider whether you can investigate them through math and/or CAD sketches instead of a prototype. Your goal is to get to the right answer as quickly as possible, which requires building some intuition of both how long it will take to build a prototype and how long it would take to do the math/sketch.
It’s like the definition of “high frequency”. The real definition is: Low frequency is where these sensors/antennas work, medium frequency is where those sensors/antennas work, and high frequency is where those others work.
You’re not the only one wondering about this. Every team is constantly evaluating what their various prototypes are actually telling them. There’s no hard-and-fast rule about how to know if you did your prototype well enough for it to sufficiently “count” for proving or disproving a design direction. Even if you tried to set some firmer guidelines before the test was done, in the end, you’d just be moving the problem up the chain.
At the end of the day, a big part of engineering is being able to look at a test with a group of people, weigh the objectives you were hoping to see, consider the shortcomings of the test setup vs. how the real thing could be different, and decide if you should continue down that path. I’m not quite sure how to break it down more finely than that… it’s non-trivial.
+1
Knowing what to spend your resources on for prototyping is very team dependent. Often, there are mechanisms that we do basically no prototyping on. For example, if we sketch out the robot architecture and decide we want a linear elevator, the first time one exists will be on the full robot, not as a discrete prototype. Our team knows how to build an elevator of various shapes and sizes; we don’t need to learn from a prototype of it. An intake for a new game-piece however, that’s probably going to get multiple simultaneous prototyping groups exploring the different design directions we can think of.
This calculus of what is right to spend your time on has to be made by team, considering your confidence levels in the various things you’re looking to build. If your team doesn’t have a lot of experience making elevator mechanisms, then maybe it does make sense to do a standalone prototype of that, for example.
It’s all about building confidence that the next thing you make is likely to work, or at least work better. Sometimes you find you don’t need to build any additional confidence. Sometimes you really do.
Adding on to address this part. The way I look at it, a proof of concept is needed when many/most of the team is looking at a ground-breaking concept and saying “I genuinely don’t know if that is possible for an FRCFIRST Robotics Competition team to achieve within a season”, and one or more champions of the idea believe in it and are very driven to prove it can work. If we have limited available manpower, we give them a deadline to convince us that there’s something there worth pursuing. If we can spare the manpower, we sometimes allow one or two students to work on it for the whole season (with clear priorities set when it comes to other resources like mentor time and machine time) - if they get something working it goes on the bot, if not, hopefully they had fun and learned something. But basically, I’m starting from the premise that I don’t think this can work, but I’m willing to be convinced if you can prove the concept. You won’t know 100% for certain that the idea is disproved, but it informs a judgement call. If it helps, think of it this way: the goal is not to figure out whether the idea is physically possible with infinite time, money, and other resources; the goal is to figure out whether your team can make it work in time.
This is why I say that proof of concept prototypes are for truly innovative ideas that break new ground in the world of FRCFIRST Robotics Competition, not standard mechanisms. If you want to know whether the concept of a rolling intake with polycord can intake a squishy ball, you don’t need to build anything, you can prove that concept by finding a video of a team doing that in 2016 and go straight to tuning/comparing with other concepts.
It’s like the definition of “high frequency”. The real definition is: Low frequency is where these sensors/antennas work, medium frequency is where those sensors/antennas work, and high frequency is where those others work.