Webmasters must be on the lookout for those spiders which masquerade as the famous Googlebot. This visitor befell our spider trap today as it did not request our robots.txt and it was caught in the honeytrap.
2007-09-16 (Sun) 13:23:30
address is 64.27.28.154, agent is Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
64.27.28.154 resolves to oscextreme.com
Another spider that ignores robots.txt has fallen into the spider trap and has been banned.
A bad robot hit /trap/ 2007-08-18 (Sat) 17:02:14
address is 209.181.96.41, agent is Mac Finder 1.0.3
bot/1.0 Spider
This voracious spider with a malformed useragent is now banned after cobbling up much bandwidth. If the Bot owner does not have the decency nor intelligence on how to form a proper useragent, then the bot is not welcome here. If you wont identify yourself, get lost.
Log Examples:
71.13.115.117 – - [08/Aug/2007:18:47:38 -0400] “GET /B000GWLL0K.shtml [...]
Getting hit with alot of directory spam lately. The culprits are using a fake gmail account with random characters for a username.
Here are the IP’s of the offenders from the past few days.
226.126.93.58
193.69.180.120
66.165.172.163
70.86.3.178
66.132.158.88
213.42.21.77
72.32.59.213
They are trying to list sites like:Â cant-crap.flekwe dot info.
We have firewalled those IP’s and not suprisingly Rackspace and ThePlanet host a few [...]
I banned skweezer.net today by adding its range of IP’s to our firewall. It uses a blank referrer so you must block their IP’s. Â
We do not wish our copyrighted material to be used on Skweezer without our permission and have requested by email that they remove our domain from their service.
 #Skweezer.net
72.1.97.146
65.38.160.0/24For the .htaccess file:
deny [...]
