Affiliates
Latest topics
» best slogan of love...?
Yesterday at 10:09 pm by ranisharma1531

» Best health drink for adults..?
Yesterday at 7:54 pm by kavitajain9782

» Best browser in the world...?
Yesterday at 6:11 pm by anshitajain1212

» best coldest place in India..?
Yesterday at 4:45 pm by rajshreedixit01

» What is Crawling
Tue Dec 12, 2017 9:48 pm by nidhijain1221

» What is a Domain
Tue Dec 12, 2017 9:43 pm by nidhijain1221

» Best Airline In India....?
Tue Dec 12, 2017 2:42 pm by angilpari1122

» Heading tags
Mon Dec 11, 2017 9:46 pm by sameergupta12

» Organic result
Mon Dec 11, 2017 9:39 pm by sameergupta12


Use of robot.txt ?

View previous topic View next topic Go down

Use of robot.txt ?

Post by bangalorewebguru on Thu Jun 08, 2017 5:15 pm

Use of robot.txt ?

bangalorewebguru
platinum
platinum

Posts : 1115
Earned : 4481
Credit : 4
Join date : 2013-10-15
Age : 30
Location : Bangalore

View user profile http://www.bangalorewebguru.co.in/

Back to top Go down

Re: Use of robot.txt ?

Post by Designpluz on Tue Jun 20, 2017 7:31 pm

it will instruct search engine which pages should be accessed by google bot & which pages has to be restricted for indexing
avatar
Designpluz
platinum
platinum

Posts : 1224
Earned : 5598
Credit : 0
Join date : 2017-01-04

View user profile http://www.designpluz.com.au/

Back to top Go down

Re: Use of robot.txt ?

Post by Sankara Vignesh on Wed Jun 21, 2017 8:00 pm

Website owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol. The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the site.

Sankara Vignesh
platinum
platinum

Posts : 122
Earned : 358
Credit : 0
Join date : 2015-07-28

View user profile

Back to top Go down

Re: Use of robot.txt ?

Post by Sponsored content


Sponsored content


Back to top Go down

View previous topic View next topic Back to top


 
Permissions in this forum:
You cannot reply to topics in this forum