adding your site to Google®

[RudimentaryPeni]

How can I add my site to google®, i tried doing that thing at the bottom where it says 'dissatisfied..." and it hasn't worked. I did it a couple of times infact.

[mtrawls]

You tried http://www.google.com/addurl.html?

Note, too:

We add and update new sites to our index each time we crawl the web, and we invite you to submit your URL here. We do not add all submitted URLs to our index, and we cannot make any predictions or guarantees about when or if they will appear.

(emphasis mine)

[JohnBoucher]

There are a lot of reasons why Google might not be listing your site. Once you submit your site it can take months to show up, if at all.

Here are a few simple things that Google likes that will help you get listed.

Google likes a good clean straightforward site. Good clean code and logical navigation is essential to allow any search engine spider to search your entire site. Validate your code and clean up any errors.

Make sure that the page your are submitting has substance to it. Google cannot index what is not there. A doorway page or a flash page won’t give the Googlebot much to work with.

Get other teams or sites to link to you. Google likes sites that other sites want to link to. A good place to start is with your sponsors. When Google visits their sites, then the Googlebot will follow the links to your site. Submit your site to the ODP Open Directory Project. They have a section for FIRST teams. http://dmoz.org/Computers/Robotics/C...etition/Teams/

Check your robots.txt file for the correct syntax. If you don’t have one, then put one up. It’s considered proper form to have one. Googlebot will be looking for it.

Add new content often and make small changes to your index page at regular intervals.

A good online community to join for understanding search engines and web development is Webmasterworld. You will find lots of information and useful tools there.

[Petey]

Could you expand upon this robots.txt file?

--Petey

[Brandon Martus]

here is what I set up for (most of) chiefdelphi's robots.txt

User-agent: *
 Disallow: /forums/attachment.php
 Disallow: /forums/avatar.php
 Disallow: /forums/editpost.php
 Disallow: /forums/member.php
 Disallow: /forums/member2.php
 Disallow: /forums/misc.php
 Disallow: /forums/moderator.php
 Disallow: /forums/newreply.php
 Disallow: /forums/newthread.php
 Disallow: /forums/online.php
 Disallow: /forums/poll.php
 Disallow: /forums/postings.php
 Disallow: /forums/printthread.php
 Disallow: /forums/private.php
 Disallow: /forums/private2.php
 Disallow: /forums/report.php
 Disallow: /forums/search.php
 Disallow: /forums/spotlight.php
 Disallow: /forums/subscription.php
 Disallow: /forums/sendmessage.php
 Disallow: /forums/sendtofriend.php
 Disallow: /forums/threadrate.php
 Disallow: /forums/usercp.php
 Disallow: /forums/images/

This hopefully keeps Google [and other search engines] out of the areas where we don't need searching capabilities.

[MrToast]

Originally Posted by Brandon Martus

here is what I set up for (most of) chiefdelphi's robots.txt
...
This hopefully keeps Google [and other search engines] out of the areas where we don't need searching capabilities.

Thanks Brandon! That's very helpful!

MrToast

[Phil 33]

I did it a couple of times infact

Don't do this. If you submit your site too many times in a short period of time they will not add your site to their database for "spamming" them. You just have to wait, which is unfortunate because no matter how much you wait, your site may not get added at all.

If however, you are willing to use a paid service... that's another story entirely.

[JohnBoucher]

Originally Posted by Petey

Could you expand upon this robots.txt file?

--Petey

The long answer is here http://www.robotstxt.org/wc/robots.html

The short answer is what Brandon showed us. If you want to keep specific sections of a site from being indexed, then you disallow them.

Good form dictates that you have the robots.txt file, but in reality it's not necessary.

BUT be aware, not all robots obey the directions listed in the file.

A easy way to learn about them is to go the any home page and add a /robots.txt to the url and you should see their robots.txt file
www.cnn.com/robots.txt

[Joe Ross]

I got tired of seeing 404 errors for robots.txt on my sites that I
just put blank files there.