Keep It Simple, Stupid | |
PerlMonks |
Re: Re: Re: Re: Web Robotby Anonymous Monk |
on Jul 17, 2003 at 09:10 UTC ( [id://275155]=note: print w/replies, xml ) | Need Help?? |
On the topic of robots.txt, why would someone even use this? If you don't want a page accessed, limit access to it. Depending on all computers to play nice isn't a very smart move, they have many hidden motives :)
In Section
Meditations
|
|