What I would suggest doing-at the competition and during the build season-is setting up some clear, understandable rules to everyone. Say, if you're not doing x, or if you do y instead of z, then you're off the team. You have 50 people on your team, I'm sure that getting rid of some people won't hurt. As for the competitions, I suggest having a responsible student or so to control the people there. Tell them and the people in the stands that if the person finds you're not doing something, you're sent out of the stands. I know some teams that don't allow students to even come to the competition if they don't do what they're supposed to do.

All in all, you need to take control over your team, via the lead mentor on your team. Set down some sturdy rules, and most importantly, enforce them.