Peer Judging

[Robert Thacker-]

So I realize that we, the people of FIRST, have to look and rank these final 5 Inventor submissions to decide who wins. Well, I don't know about you guys, but I have never had to judge a submission before.

So I think we should all use this thread to discuss how to judge these awards. What do we look for in design, presentation, the process, etc.

What do you think?

[Phil Ross]

sounds like a good idea to me. It something that we have to do, so we might as well do it on a thread

[JasJ002]

I'm not disagreeing with what Autodesk is doing, but I think the problem with this new sytem is that teams always built their robots and websites to adhear to the judges. Now that we are supposed to create a site that pleases our peers, we don't at which angle to aproach the problem. Younger minded students tend to look towards the more technologically advanced submissions, while judges tend to adhere to a much more complicated submission. What shall we do?

[Phil Ross]

Well, we do need to impress our peers for this. If you create something that is both pleasing to look at and everything that the judges have looked for in past years you can create a winning entry. This is the easiest way to approach this problem.

[pmax]

I can assure you that technological excellence has been the primary aspect the judges have looked at in past years. We have asked for other items (process photos, website, etc) to both provide some context for the submissions, and also to help differentiate the top 10% of the teams.

If you are using good engineering practices, documenting them, facilitiating the tools you have, and can present this well, you are doing everything you can to compete for the award.

Our goal for the peer judging aspect was to generate excitement and have everyone feel they were a part of the decision.

Of course, any feedback in the process is appreciated. This award is for your hard work!

Regards,
Pete

[Robert Thacker-]

So, overall, I feel it's crucial to regard the process and documentation of the robot's design. So an entry should pertain this, show their robot, and be neat to look at (rendering and such).

But I feel that teams may look at an amazing website and perhaps not a great inventor drawing behind it (not using advanced functions and such). Are we going to veiw the files along with the website? The award should be a paring of the two, in my best opinion. What do you guys think? Are we going to veiw those files?

[JasJ002]

One problem that I have heard from years past about peer judging in the Regional competition for animation, is that there are people judging who are in no way qualified to make such a decision. I'm just scared that the 90% of FIRST that couldn't tell you the difference between a hole and an extrusion will be deciding who the champion is.

[Robert Thacker-]

That is a growing concern for me too. I wonder if student-picked national awards is such a very good idea.

That's why I feel it's important to discuss what we want to look for in judging these award entries.

I think that a great idea would be a student's choice and a Judge's choice award winners.

[Phil Ross]

yea, when looking back at the main assembly of our robot. all the different components to it and extrusion and others parts that people wont be able to tell the difference between unless they open up the files and look at the browser. this is something that has been on my mind for awhile, we should try to make it clear to people who are voting that not everything is what it seems to be.

A peer-voted winner and a judges choice seems to be a logical and reasonable solution to this and I wish it wasnt to late to do anything about it for this year.

[Robert Thacker-]

Anybody know when the top 5 will be announced?

[Phil Ross]

about a week or two before championship. btw, i was just looking over your submission it looks great. I expected nothing less than that from what kressly has told me about your CADD work.

great job and good luck

[Robert Thacker-]

Hey thanks, I really appreciate that. Tell Kressly I said hi.

[Robert Thacker-]

I feel that teams should look at these awards as an engineer and not an art student. Something may be really cool to look at, but is it well explained? Can you clearly see all the compoents? Does everything you see that should have a function on the robot clearly explained and shown? What about where the systems are attached to each other? Are the 2D's coinciding and helping this out?

And what's really important is knowing that this is an award about honoring excellance in student design and documentaing this whole process. An award winning team should be the BEST in displaying this process and displaying the final design in such a way that it could be recreated from description. We should vote for who should be the winner and not a team we feel sorry for. Even things like stress analysis is important in understanding what kind of abuse a robot will take. An award winner should use all the features given to them in the Inventor submission itself. And any team who steps outside of Inventor, like using 3Ds Max, really shouldn't be concidered, in my opinion, b/c they did not commit to learning the entire program. And the judges have said that if a team uses any other software, even other CADD programs, it will severly hurt their overall score anyway.

The Inventor judges had once told me that they do not have the Inventor Award at regionals b/c there are not enough people to judge who know the program to be able to make a decision at a regional competiton (that and not too many teams put in an entry). But isn't this just putting that same fear in place? Do all teams really know the program (I mean the teams that do not even commit to the entry)?

So I feel we should look at these basic ideas and keep them in mind when looking these designs.

Those are all of my thoughts.

[JasJ002]

Robert hit the nail on the head with that statement.