SEO - Do you use Robots.txt file effectively?

Last updated by Camilla Rosa Silva [SSW] 3 months ago.See history

If you decide to use the redirect method when linking to external pages from your site, it's a good idea to have a robots.txt file in your root directory. In the "robots.txt" file, you specify that the robot (or spider as they're sometimes known) should not look in the redirects folder. This will avoid the problem that can sometimes occur, where a Google search will incorrectly display content from another site as if it was from your site.

Also, this avoids incorrect hits on your redirects, mucking up your statistics which is one of the main reasons you would use redirects in the first place!

We open source. Powered by GitHub