T-shirts are defiantly the most important swag item you can buy. They provide an easy reference point for other teams, judges, sponsors and the general public to notice and remember you. The will unfortunately take up a vast majority of your budget, if not all of it. Like Billfred said it's pretty easy to lower the cost by buying in bulk, using one design, and as few colors as possible. This happens to be what 1902s team mom does for a living so PM her (her username is wendymom) and she'll be able to give you more details on pricing and stuff like that.
As for other giveaways try to be as creative as possible. In our rookie year Exploding Bacon was having trouble finding funding for pins as well, so instead of going the traditional round button route we bought about 10 dollars worth of foam core, 500 pin backs (cost about 3.00 at a craft store) and a few sticks of hot glue and we created our Wilbur face on a rocket pins. They turned out to be pretty popular and we've stuck with this method ever since.
A picture of our pins can be found here:
http://www.chiefdelphi.com/media/photos/24628
The bottle cap pins are a great idea as well, pretty much anything that incorporates a team logo can be made into a great memorable pin.