A solution for Animation Judging for 2005

[dlavery]

Originally Posted by SteveC116

The four times that I have viewed the awards ceremony (this year and last year), I never saw the animation played at the end. Normally it's just a powerpoint slide.

Your winning animation was played during the awards ceremonies at Annapolis.

-dave

[Lev]

Originally Posted by TKatsAniMentor

3. We need a judging rubric. How are we supposed to know what it is that we are being judged on besides general categories?

The judging rubrics are (and were in all the past years) in the award rules. They are vague in some places but they actually are pretty specific.

The problem with the rubrics is what they are. I think its hard to find a more subjective category than "creativity". For example, Space and Robot scenes can be thought of as creative in a general context, but everyone in FIRST knows that this is the most common scene in FIRST animations.

Technical Execution and Content Communication are not perfect either - since these are so linked together. As an extreme example - sometimes its hard to understand whats going on in the animation simply because of poor technical execution.

Honestly, i think it would be better if there was no rubrics system at all. Even with perfectly defined rubrics, all of which make perfect sense, there will always be problem with each team having different scale - as someone already mentioned. I think a better system would be with every team just voting for best 3 (for example) animations (exluding own animation) and the one getting most votes - winning.

[SteveC116]

Originally Posted by dlavery

Your winning animation was played during the awards ceremonies at Annapolis.

-dave

Avast! The one competition I do not attend!

And about the rubrics, it doesnt matter if one team awards everyone really low scores, as long as the awards are consistant in relation to the others.

[Salik Syed]

Originally Posted by SteveC116

Avast! The one competition I do not attend!

And about the rubrics, it doesnt matter if one team awards everyone really low scores, as long as the awards are consistant in relation to the others.

Well...they get a higher score as they do not rate themselves!
Them voting lower for others will only bring the OTHERS average down and not their own.

[ThetaDot]

Originally Posted by Salik Syed

Well...they get a higher score as they do not rate themselves!
Them voting lower for others will only bring the OTHERS average down and not their own.

so then mandate that the highest score given on your sheet is the # of points you also give your own team minus a number like 5 or 10

[Terminator6]

wells heres my little bit and then ill probably shut up and let other people figure this one out....

any way you cut it i dont think that peer judging will every be....heres the dirty f word....fair...

my analogy here is this, and if im completely mistaken sombody slap me and tell me ive been ignorant.....

the chairmans award nor any of the design awards are not decided by peer judging...so why should the same be true for animation....and i know that the point of peer judging is to allow the animators to have more control....but still

so heres my comprimise idea....i know this is crazy....but 2 awards...let one be decided by peer judging....one rep from each time like this year....and one by judges....let people know what the professionals liked and what the students liked.....

---end idea-----

[White_Orpheus]

I think the student judging is fundamentally flawed. We managed to walk away with a victory this year at the Silicon Valley regionals. While i like to think that our animation is highly sexy, we really didn't convey anything about what first does for us. Frankly in my opinion, 192 really deserved the award, but the students judged based on sexiness of the animation instead of the actual message conveyed. While it may have been flawed, i'm still glad we won

[Ryan Dognaux]

Originally Posted by Terminator6

so heres my comprimise idea....i know this is crazy....but 2 awards...let one be decided by peer judging....one rep from each time like this year....and one by judges....let people know what the professionals liked and what the students liked.....

That is quite possibly the best thing I've heard yet!! Seriously... the top Professional pick, and the top animator pick! Man, that is just a great idea if I've ever heard one.

Maybe Website Award should do that too

[rowe]

Originally Posted by White_Orpheus

While i like to think that our animation is highly sexy, we really didn't convey anything about what first does for us.

As an animator judging the Silicon Valley regional, I graded team 253's animation very high in both creativity and technical execution but did not grade as high in message. I was very impressed with 253's animation...so congradulations (Look forward to competing against you in nationals). But back to the topic of the thread, although the student judging might not be "fair," it is also nice to have some imput. A solution for this could possible be to split the award into 2 awards, say one by judged by professionals, and the other by students, that way. Or, have a combination of student and professional judging, say 50/50 or some variation of the ratio to be worked out by Autodesk.

EDIT: oops, it looks like i didnt read the thread closely enough beacuse the 2 award idea has already been mentioned. Heh.

[Vivelation]

Originally Posted by White_Orpheus

I think the student judging is fundamentally flawed. We managed to walk away with a victory this year at the Silicon Valley regionals. While i like to think that our animation is highly sexy, we really didn't convey anything about what first does for us. Frankly in my opinion, 192 really deserved the award, but the students judged based on sexiness of the animation instead of the actual message conveyed. While it may have been flawed, i'm still glad we won

Hey, thanks! Our animation team works really hard, and whatever they come up with is always awesome, but your team's animation was really cool (the music and the running and the hair pretty neat stuff). Anyway, congrats again on your win (standing O from 192 ^^) See you guys at nationals!

[Ryan Dognaux]

Back on track here guys - I seriously think a Professional Award and a Student Award is the way to go. It would test both sides of the medium, and also would give animators more of a chance to win. This would work equally well with the website award, and if the Autodesk Inventor Award became regional too then it could also go along in this category.

Anyone against a Professional Award and a Student Award??

[opnickc]

Originally Posted by Terminator6

wells heres my little bit and then ill probably shut up and let other people figure this one out....

any way you cut it i dont think that peer judging will every be....heres the dirty f word....fair...

my analogy here is this, and if im completely mistaken sombody slap me and tell me ive been ignorant.....

the chairmans award nor any of the design awards are not decided by peer judging...so why should the same be true for animation....and i know that the point of peer judging is to allow the animators to have more control....but still

so heres my comprimise idea....i know this is crazy....but 2 awards...let one be decided by peer judging....one rep from each time like this year....and one by judges....let people know what the professionals liked and what the students liked.....

---end idea-----

I second that (third, whatever). I think that, even though technical is only worth 30% of the score, students grade it as 90%. And when I say techincal, I don't mean the most difficult to make (aka biped and the like), but I mean which looks (and sounds) the best. Many students grade the other areas based on how well they are potrayed by the technical aspect, even if the animation didn't use complecated things like character studio. So yeah, having 2 awards (one student, one proff.) is a great idea. I hope FIRST will seriously consider this for next year.

On a side note, I must congratulate FIRST on doing a much better job with the animation judging than last year. All entries were on DVD and had no loss in quality, and the staff worked hard to make the judging as "fair" as possible. Hopefully next year will be even better.


[EDIT]

Anyone against a Professional Award and a Student Award??

I'm all for it.