[captaincrowbar]

Looking at the results for C++, it looks like the text santizing you're doing on link titles is losing the "++". All the titles seem to have C instead of C++, e.g. "Boost C Libraries" should be "Boost C++ Libraries". You're also losing the # in C# and the dot in .NET, and probably others.

[bcleary]

Thanks for the heads up, will modify the parser, again if anybody knows of a good open source crawler let me know. I rolled my own very quickly but would love it if i could find a 3rd party solution. Another option i have explored is pulling in the titles and descriptions provided by search engines but currently only DuckDuckGo offers anything useful and even then its coverage of some of these low ranking programming pages isn't great. Bing offers a pay per use access to its index but the costing structure really doesn't fit with my use case.