My cheeky answer is that I don't think it's possible to spend too much time with FIRST... just too little time on other things
Seriously though, if you allow much of your life to revolve around FIRST, but you draw in the other VERY important things like career, family, health, etc... then you can have a life that appears to be very FIRST centered, without neglecting the remainder.
For example, my dad is coming into town this weekend. Typically, we might hang around the house, watch TV, etc. This weekend, we're going up to the school and moving our build space to our new classroom. So I'm "doing FIRST", but also getting to spend time with my dad (who is an engineer and gets a kick out machine shops). Many of my friends are my "robot friends" so I'm not choosing FIRST over them, I'm choosing FIRST with them. Same thing with my job... my employer values the exposure of FIRST and the personal development opportunities I get through it... so they don't see the excessive robotics time as a downside in the long run. Yes, if a project is due in February and I can't work 80 hrs a week to knock it out, they might grumble, but I also tried to shift loads this year to avoid missing any key deadlines at work b/c of FIRST. I'm also really looking forward to some day raising my own robot kids, and potentially using my maker hobbies as a way to bond with my own children.
So yes, if you have poor health, low grades, loss of friends/family... and at the same time spending a lot of time in FIRST... and think additional time would fix your problems, cut back. FIRST should amplify the rest of your life, not overshadow it.