I am not familiar with Yodlee, but I recently discovered that many banks support OFX [1], which is a format for exchanging financial information.
GnuCash has a list of OFX credentials for major banks. [2] In fact, there are tons of OFX open source libraries out there - I had luck with this one recently in Python. [3]
Thanks for the references. After some research, I decided to go with YNAB http://www.youneedabudget.com/ it has a desktop and mobile apps and uses your dropbox for syncing data.