[inovica]

We are working on a lot of scraping and analysis and here are a few links that you might be interested in if you are using Python:

http://pyro.sourceforge.net/

http://pyprocessing.berlios.de/

http://www.sqlalchemy.org/

http://codespeak.net/lxml/

http://nltk.org/index.php/Main_Page

The biggest hurdle is in understanding how to navigate through a complex site - such as a forum, real estate etc. We have created a visual tool for this however there are other methods. Look at dapper.net as this is useful.

I am wondering if there could be some collaborative effort from the minds on this site to create something unique and groundbreaking