KAIST AITrc Crawler
KAIST AITrc Crawler
This robot was caught in the spider trap today. It did not request the robots.txt file and preceded to follow all links despite noindex, nofollow indications. Since it does not respect this server requests for privacy, it is now banned.
A bad robot hit /trap/ 2008-02-25 (Mon) 22:08:12
address is 143.248.134.231, agent is KAIST AITrc [...]