We use a push broach
like this one to make hex holes for hubs and such. The material thickness rating on this one is 1/2" to 1-3/8". I know we have broached to the upper limit in 6061 aluminum before without issue, when making live shaft hubs for Colson wheels. In that case the hub was more like 2", and we counterbored each side some to limit our overall broach length.
I don't know what tooling you have available (push, rotary, or pull broach), or what the limits of other tools are, but with the push broach and a big arbor press, 1" aluminum bar should not be a problem. Just lubricate well and make sure the broach stays straight to the hole.
In regards to thin wall tube, we have gotten in the habit of plugging the ends of the tube with snug-fit delrin (or even hardwood) blocks, for 2-3", to give a larger bearing surface for pivots, or prevent tube crush where bolting on a sprocket. No reason you couldn't plug the end of a thin wall tube and broach through both the tube and the plug.