What is the use of robots.txt?

View previous topic View next topic Go down

What is the use of robots.txt?

Post by artijain1900 on Fri Sep 23, 2016 8:41 pm

Hello Guys,


I would like to know that What is the use of robots.txt? tell me ..

artijain1900
platinum
platinum

Posts : 57
Earned : 285
Credit : 0
Join date : 2014-04-16

View user profile

Back to top Go down

Re: What is the use of robots.txt?

Post by bangalorewebguru on Sat Sep 24, 2016 8:27 pm

It is great when search engines frequently visit your site and index your content but often there are cases when indexing parts of your online content is not what you want. For instance, if you have two versions of a page (one for viewing in the browser and one for printing), you'd rather have the printing version excluded from crawling, otherwise you risk being imposed a duplicate content penalty.

bangalorewebguru
platinum
platinum

Posts : 1142
Earned : 4616
Credit : 4
Join date : 2013-10-15
Age : 24
Location : Bangalore

View user profile

Back to top Go down

View previous topic View next topic Back to top


 
Permissions in this forum:
You cannot reply to topics in this forum