By what you're saying, it seems there are teams out there using cardboard batteries during weigh in.
I don't think a team would get to the point to use a
dummy battery, in the sense of a hollow one or something like that. What I have read here in the forums is that some teams use really old batteries for inspection, as it is a known fact that they actually get lighter with time. It is a cheap shot, nonetheless, but you can always blame it on the freshman member for putting it in place, in case you get caught.
So, yes, I think the robot should be weighted without the battery in place, or at least some "official" one, standardized to every team.