One option is to build a set of 0.1255" x 0.4996" bushings. The change in diameter is enough that you can put some tiny set screws in the bushing. Barring that, you can machine a pin to go through it.
Another option is to make a shaft saver similar to what RC hobbyists use to attach propellers directly onto motor shafts, such as
http://hobbyking.com/hobbyking/store...col let_.html. Some designs hold the shaft using collets, others use set screws. Both can be built with a lathe and a drill press, or you might be able to find COTS ones that fit your specs.
EDIT: Sorry, not shaft saver. Prop Adapter.