Affiliates
Latest topics

What is robots.txt?

View previous topic View next topic Go down

Re: What is robots.txt?

Post by bangalorewebguru on Thu Aug 10, 2017 9:09 pm

This text file which sits in the root of your website's folder communicates a certain number of guidelines to search engine crawlers. For instance, if your robots.txt file has this line in it; User-agent: * Disallow: / it's basically telling every crawler on the web to take a hike and not index ANY of your site's content.

bangalorewebguru
platinum
platinum

Posts : 1115
Earned : 4481
Credit : 4
Join date : 2013-10-15
Age : 29
Location : Bangalore

View user profile http://www.bangalorewebguru.co.in/

Back to top Go down

View previous topic View next topic Back to top


 
Permissions in this forum:
You cannot reply to topics in this forum