[astrec]

Python & Beautiful Soup (http://www.crummy.com/software/BeautifulSoup/) are your friends here.

[jreilly]

Anyone know of a similar library in rails that works well?

[rgrieselhuber]

I'm pretty sure this (HPricot) is the one most people use in Ruby:

http://code.whytheluckystiff.net/hpricot/

[abijlani]

Here's a great hpricot tutorial

http://www.igvita.com/2007/02/04/ruby-screen-scraper-in-60-s...

[qhoxie]

hpricot is great, but I hear lately that Nokogiri is faster and just as capable:

http://github.com/tenderlove/nokogiri/tree/master

[ivanstojic]

Nokogiri even has a HPricot emulation mode, if you need to port HPricot code.