29 Jun '14, 9am

@MissAnnalieza start with the robots.txt file. MUST BE in the root folder of Your website.

After you make your file, upload it to the root directory of your web site. If you don’t know what that is, find someone who does! This is important. Google allows for subdirectories of web sites to be registered within Google Webmaster Tools. However, robots.txt files do NOT work on a subdirectory basis. They have to go at the root level of a web site. If you don’t put them there, then you won’t be preventing access to any part of the site. Remember, after you upload to the root level, you can go back into Google Webmaster Tools and use that aforementioned analysis tool to see if it is really blocking the pages you want to keep out.

Full article: http://searchengineland.com/google-offers-robotstxt-gener...

Tweets

The robots.txt Celebrates 20 Years Of Blocking Search Engines

The robots.txt Celebrates 20 Years Of Blocking ...

searchengineland.com 30 Jun '14, 1pm

Today is the 20th anniversary of the robots.txt directive being available for webmasters to block search engines from craw...

Google Killer Robots.txt File Blocks Terminator...

seroundtable.com 04 Jul '14, 1pm

Google posted a new robots.txt file at google.com/killer-robots.txt that is designed to help secure the Google co-founders...