I know the rule about destroying the balls and being penalized etc… But how much damage constitutes a penalty? 
That is a very good question. Most teams are going to use the softball shooter design to shoot the balls, which means that there is a certain a mount of inevitable damage that will happen to the ball. Just after shooting the ball a few times, the outer shell of the ball have started to come off. So my guess is that as long as there aren’t giant chunks of foam coming from the balls, then I don’t think the refs will have a problem.
Ref’s call. If the ball is too badly damaged for further play (Field Reset’s call), it will be replaced. If it was unintentional damage, like just field wear and tear or a robot driving over the ball by mistake, the refs probably won’t throw the flag.
It seems to me that if the outer shell comes off, the ball could easily be considered damaged. Hopefully FIRST will clarify this soon, as I’m sure there are several designs that may need to be modified if this is the case.