The world of Football

 

 

 

 

 

Thursday

GO Away Robots!
I got this info from Hothint today about this robot code that makes search engines to not get access to certain pages for websites... Even Google or Altavista has this .txt page but you can also use this .txt page as a map to the pages of the website...
Heres A Standard for Robot Exclusion to help you more about this.

0 Comments:

Post a Comment

<< Home

Main Menu

Home

Links


Description

Daily notes about football

Archives

05/01/2003 - 06/01/2003 06/01/2003 - 07/01/2003 07/01/2003 - 08/01/2003 08/01/2003 - 09/01/2003 09/01/2003 - 10/01/2003 10/01/2003 - 11/01/2003 11/01/2003 - 12/01/2003 12/01/2003 - 01/01/2004 01/01/2004 - 02/01/2004 03/01/2004 - 04/01/2004 06/01/2004 - 07/01/2004 07/01/2004 - 08/01/2004

Powered By