Two bolts should be plenty. I say this from both past experience with our robots as well as my limited engineering intuition.
Past Experience
--Every year, from 2009 to 2011, we direct drove wheels through a supershifter by mounting the plate with two 1/4-20 bolts. We had no problems at all with this.
Limited Engineering Intuition
--1/4-20 bolts, for the most part, are overkill when it comes to loads experienced in FRC. I know that other teams, such as 973, have standardized their bolts at a 10-32 (or -24, I forget) for that very reason. So, if you are having any worries, use a 1/4-20 and be secure in your decision for doing so, as it is a safe choice. If you want to see some further detailed engineering information, check out the link below:
Here