here is what I set up for (most of) chiefdelphi's robots.txt
Code:
User-agent: *
Disallow: /forums/attachment.php
Disallow: /forums/avatar.php
Disallow: /forums/editpost.php
Disallow: /forums/member.php
Disallow: /forums/member2.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/online.php
Disallow: /forums/poll.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/private2.php
Disallow: /forums/report.php
Disallow: /forums/search.php
Disallow: /forums/spotlight.php
Disallow: /forums/subscription.php
Disallow: /forums/sendmessage.php
Disallow: /forums/sendtofriend.php
Disallow: /forums/threadrate.php
Disallow: /forums/usercp.php
Disallow: /forums/images/
This hopefully keeps Google [and other search engines] out of the areas where we don't need searching capabilities.