1 January 2008
Comments: 5

Use Robots.txt To Prevent vBulletin Duplicate Content

Using a robots.txt file can tell search engine spiders what not to crawl on your site. The file has to be named “robots.txt” and placed in the root of your site for spiders to find, and read the file. You can not have more than one robots.txt file for each site, sub-domains are of course […]