[Tristan Lall]

There are lots of other options that might work better than steel on aluminum (but worse than a ball bearing). How thick is the plate that you're going through? You might be able to use a bronze or plastic sleeve bearing (bushing) if it's thick enough. Otherwise, you could bolt a piece of hard plastic to the aluminum, make a large hole in the aluminum for clearance, and drill a slightly oversized (e.g. 0.01 in larger) hole in the plastic for your shaft.

Depending on the specifics of the situation (how accurate everything is, how heavily loaded the shaft is, how much you intend to use it, etc.) you might even be able to get away with running it as you described—lubrication would be a very good idea, in that case (use grease).