How can i find the robots.txt file in any website?

Question

please give me clear information, with examples

Answer 1

http://www.google.com/support/webmasters/bin/answer.py?answer=35237

Answer 2

If you use CMS like Wordpress it may have an existed file there (depend on the plugin you use). If not a CMS you have to create a robots.txt file and upload to the top directory. The location will be like this http://camerawebs.com/robots.txt

Answer 3

The robots.txt file should be in the root directory of the website so

http://uk.yahoo.com/robots.txt

works.



Have fun.

Answer 4

whatever the website is ex. yourdomain.com/robots.txt



as far as a sitemap yeah, you can put a sitemap in your robots.txt

see this page as an example

http://www.howrank.com/Robots.txt-Tool.php

Answer 5

It shouldn't be in any folders in a directory, so thus...



http://heythisisawebsite/robots.txt



Here's Yahoo's:



http://www.yahoo.com.com/robots.txt



edit: ahh someone beat me to it, hehe

Answer 6

you ask for information, but we have no fricking clue what your asking for....

you provide no details as to what you are looking for