[marbu]

I agree, java is completelly ok for implementing a crawler. For example the well known mercator crawler was written in java and it's authors stated:

Although our use of Java as an implementation language was somewhat controversial when we be- gan the project, we have not regretted the choice. Java’s combination of features — including threads, garbage collection, objects, and exceptions — made our implementation easier and more elegant. More- over, when run under a high-quality Java runtime, Mercator’s performance compares well to other web crawlers for which performance numbers have been published.

source: [Mercator: A scalable, extensible web crawler (1999)](http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.151.5...)