Alright. Well, let’s say a drill motor and gearbox in high gear gives us a speed of about 1200 RPM. With a 6" wheel, this equates to a linear speed of about 21 miles per hour at the surface of the wheel (I can go into details on that if you’d like) which is good for a range of about 29 feet if shot at a 45 degree angle (assuming the ball actually leaves the “muzzle” at 21mph and there is no air resistance) Excellent projectile motion calculator/explanation
So, if about 25 feet is an acceptable range, it seems like a 6" wheel spinning at about 1200 rpm would be the hot ticket (and is approximately what a drill motor output would be). If you want to fly farther, up the size of the wheel or the speed of the motor
So, now on the the question, coupling and mounting.
If you have space to put it in line, you can do a direct couple by means of flex coupling. Search www.mcmaster.com for flex coupling. What you do is mount the wheel to an axle such that the wheel is rotationally locked to the axle. Have a couple pillow block bearings (sold at good hardware stores and included in the KOP a few years back, otherwise seach www.mcmaster.com for pillow block) on either the axle on each side of the wheel. Then mount the drill. Then you use the flex coupling to attach the axle to the drill gearbox output. This presents a problem however. Most drill gearbox outputs are threaded (usually 3/8-16 or 1/2-20). Unforunately, most flex couplings are not. So you’ll either need to tap it (cut internal threads, with a tool called a tap) or use the stock drill chuck and put a the flex coupling on a shaft that goes into the drill chuck. You either need to make sure the drill rotates in the direction that would tighten the coupling or you need to use a left hand screw anbd washer into the end of the drill gearbox output shaft (all drills have this. if you open a drill chuck as wide as it will go and shine a flashlight in there you’ll see a screw head. this is a reverse thread screw that keeps the chuck on)
Instead of a flex coupling, you could install a sprocket onto the drill geabrox output shaft and onto the wheel or axle so you wouldn’t have to have the motor inline with the axle. Use a chain to link them. You can either attach a sprocket to the wheel and have bearings in the wheel and a dead (fixed) axle, or you can attach the sprocket to the axle and have the wheel attached to the axle (like the above flex coupling method) with something like a keyway). The spoked Skyway wheels have provisions for bearings and are meant to run on a dead axle and have the sprocket drive the wheel directly. The Skyway Beadlok wheels usually have a keyway and are meant to have a keyed sprocket on a keyed axle to drive them.
To mount the motor, hose clamps (available at any auto store or in the plumbing section of your hardware store) would be a good place to start. It is also possible you could carve some mounts out of wood. Depending on the size and geometery of the motor and gearbox, U bolts might do the trick. Be careful to not take the motor out of alignment with the gearbox in your mounting setup.
If you are looking to do a two wheel setup like the above picture, you’ll need them to spin opposite directions otherwise the ball won’t be going anywhere. You can do this with gears, some really fancy chain routing (complicated, idler sprockets needed), or two independent motors (like the picture above). Two motors would probably be the best choice so you can control the spin on the ball by varying the speed of the motors.
Also, remember to factor in a bit of “squeeze” on the ball when designing your mechanism. For a tennis ball I imagine it wouldn’t be much. It takes trial and error to dial it in.
I hope this helps. Let me know if you have any questions on anything.