Volunteering for an event while your team is there is completely doable - in fact, most volunteers at events are there with their team as well! The specific role you volunteer for will define your interaction with your team. For example, being a ref mean's you're field-side the entire time, and can't really drop by the pit to pitch in if the team needs help. In fact, you'll barely have time to interact with them at all, as the ref's often have pow-wows by the scoring station between matches, and even when not the drive team should have other things on their mind (like prepping for the match or getting the robot off the field quickly). On the other side, being an inspector means you're in the pits at all times, and if your team needs help you can usually make it to their pit to lend a hand. Generally, any field-side role is going to keep you away from the team, while any role in the pits will free you up for plenty of interaction.
Many VC's I know hesitate to slot team mentors as refs at events their team is at - the conflict of interest potential is just too high. This can change as the VC gets to know the volunteer, though - once they know that you'll handle yourself correctly on the field, they'll be more willing to slot you at the event. One of the mentors I work with went through that, and is now a ref at pretty much all of the events we go to (actually, I get the VC to double-assign him as an inspector so I get him an extra half day on Thursdays
) In fact, there was one year, at one event, where he ended up reffing during elims while our team was on the field... and called the match-deciding penalty against our team. It was 100% the correct call, and it dropped us out of elims for that event. If you don't think you can handle that situation appropriately, then avoid being a ref at an event where your team is!