We had a mounting plate through which the BDM (Bosch Drill Motor) boss fit. There were two dowel pins (8 mm I think) pressed into this plate and which fit into the 8 mm holes in the BDM. This kept the motor from translating axially towards the gearbox and rotating.
We attached an aluminum tube around the motor and which screwed into the gear box plate. This tube had a slot milled into the cylindrical part and a muffin fan mounted onto it. This provided a heat sink around the motor and forced cooling.
We then had a back plate which screwed into the back end of the cylindrical shroud over the back boss of the BDM. This back plate pushed the BDM against the gearbox plate.
We've used this design for the past two years.
We had one problem when the back plate screws backed out. This allowed the BDM to come partially free and rotate when it backed off the dowel pins. The plastic brush holders broke causing a short which fried a speed controller. Now we loctite those screws and inspect them frequently.
However, we have had NO difficulties with BDMs heating up. We have also had no mechanical damage to the weak parts of the BDM.
Maybe I'll post a white paper with drawings this summer.
We did a similar design for the Fischer Price motor. Alas, we did smoke one of these in competition this year.
But that required a pretty elaborate set of circumstances that should occur very rarely.