How to block Wget/PHP/Perl bots?

[Wow]

It is so easy to change user-agent string, so don't even bother. I don't
know how yahoo blocks all. Here is what they did it

Please try to user Perl's useragent even changed the useragent to Mozzila to
get this url http://news.yahoo.com

If I use IE, it takes me 5 seconds, but with perl, it takes me 15 minutes.
Yahoo is first sending a cookie, but there is more, because using LYNX it is
ok to view the page.

So, I hope I can implement something similar. I am thinking using Javascript
to test it, since all PHP/Perl/Wget will not activate javascripts.

However, when googlebot come, I goto welcome it.

So is there a way not to display content to googlebot and human eyes, but
block all PHP/Perl/Wget even when they id themselves as Mozzila?

[Toby Inkster]

Re: How to block Wget/PHP/Perl bots?

Wow wrote:
[color=blue]
> If I use IE, it takes me 5 seconds, but with perl, it takes me 15 minutes.
> Yahoo is first sending a cookie, but there is more, because using LYNX it is
> ok to view the page.[/color]

Lynx does support cookies you know?
[color=blue]
> So is there a way not to display content to googlebot and human eyes,
> but block all PHP/Perl/Wget even when they id themselves as Mozzila?[/color]

No.

--
Toby A Inkster BSc (Hons) ARCS
Contact Me ~ http://tobyinkster.co.uk/contact
Now Playing ~ ./bruce_springste en/greatest_hits/02_thunder_road .ogg

[Wow]

Re: How to block Wget/PHP/Perl bots?


"Toby Inkster" <usenet200407@t obyinkster.co.u k> дÈëÓʼþ
news:pan.2004.0 7.24.08.57.09.1 90484@tobyinkst er.co.uk...[color=blue]
> Wow wrote:
>[color=green]
> > If I use IE, it takes me 5 seconds, but with perl, it takes me 15[/color][/color]
minutes.[color=blue][color=green]
> > Yahoo is first sending a cookie, but there is more, because using LYNX[/color][/color]
it is[color=blue][color=green]
> > ok to view the page.[/color]
>
> Lynx does support cookies you know?[/color]

YES, i know and I rejected the cookie, still able to view news.yahoo[color=blue]
>[color=green]
> > So is there a way not to display content to googlebot and human eyes,
> > but block all PHP/Perl/Wget even when they id themselves as Mozzila?[/color]
>
> No.
>
> --
> Toby A Inkster BSc (Hons) ARCS
> Contact Me ~ http://tobyinkster.co.uk/contact
> Now Playing ~ ./bruce_springste en/greatest_hits/02_thunder_road .ogg
>[/color]

[Gregory Toomey]

Re: How to block Wget/PHP/Perl bots?

Wow wrote:
[color=blue]
> It is so easy to change user-agent string, so don't even bother. I don't
> know how yahoo blocks all. Here is what they did it
>[/color]

By block them? I don't alienate customers just because they want to use
wget.

gtoomey

[Edward Alfert]

Re: How to block Wget/PHP/Perl bots?

Gregory Toomey <nospam@bigpond .com> wrote in news:2mgadlFmd3 uuU2@uni-
berlin.de:
[color=blue]
> Wow wrote:
>[color=green]
>> It is so easy to change user-agent string, so don't even bother. I don't
>> know how yahoo blocks all. Here is what they did it
>>[/color]
>
> By block them? I don't alienate customers just because they want to use
> wget.
>
> gtoomey[/color]


As long as they are respectful and throttle their requests.


--
Edward Alfert

http://www.rootmode.com/

Multiple Domain Hosting and Reseller Hosting Plans
Coupon Code (Recurring $5/month Discount): newsgroup