[ncallaway]

I agree.

I really wanted to use mint, but couldn't bring myself to give away my bank password.

I resorted to writing my own tooling using puppeteer running on my machine to automatically login to my bank accounts and download the CSV exports of my transaction data for each bank. I then normalize that transaction data, and import the data into Lunch Money.

It was a pretty big hassle to write and get working reliably(ish), but I'm super happy now that it's done. Every 2-3 weeks I run the script, and 5 minutes later all of my transactions are available in Lunch Money. I have the peace of mind of knowing that I'm not exposing my banking credentials to random third parties.

[rendaw]

I've been doing this for a couple years, just with Selenium rather than Puppeteer. Banks are generally okay, and they don't change their websites that often which is nice. A couple banks force phone 2FA and deliberately hide the "email confirmation" option which makes full automation difficult. Also nag screens, Selenium unreliability, etc.

Some companies like Azure use a different account name every time they bill me. I ended up having to regex the credit card transactions for that. Others have something completely nonsensical like "BS 03-6743-2266" (<- this was iTunes), or use a 3rd party processor that puts their own name in the transaction.

The real issue for me is getting itemized purchases to do categorization on. Stuff like restaurants are okay since everything is food, so I can just categorize the whole transaction. Amazon banned me and forced me to change my password while trying to grab my purchase history (and only my purchase history!). I was trying to grab it from email receipts but then I realized they don't send receipts for Subscribe & Save purchases (!).

I'd kill for something better, but for now it's this or manually enter every transaction or give up on financial responsibility.

[ncallaway]

Yep, I haven't been able to get to full automation either.

I integrate with 1pw, so I don't have to enter my credentials which is nice.

I handle all the 2FA through cli prompts, so as a user you basically need to stick around as it's running because you might get a 2FA prompt as it logs in to banks.

The worst for me has been irregular marketing or other reminder popups from some banks. The kind of thing where they put some interstitial page up once per month or so after you sign in and want you to respond in some way.

[scrollaway]

You're doing this for personal or business expenses?

I'm curious to check this out, do you have a GitHub repo/account?

[rendaw]

Personal, and no repo at the moment. The code is a huge mess, from scattered multiple generations of wrappers around Selenium's "click this element" functionality to custom accounting code to bit rotted parsers for old receipt email formats, etc. My thinking was largely "as long as I have the data I can make the code better later" and that later has yet to come. I don't have any passwords in the code at least but I think I do have email address whitelists/blacklists for email management stuff and lists of merchants for credit card transaction categorization...

I'll see if I can maybe make a repo by the next time there's a thread about how customer hostile banks are.

[climb_stealth]

Did you release the sources for this by any chance?

I have been playing with the idea of doing something like this for a long time. But it seems like a huge job and I haven't been able to motivate myself towards it yet.

My bank even supports showing a lot of those spending stats as in Lunch Money, but it is entirely on their end and I'm not in control of it at all. I can look at the graphs and numbers in the app but I can't export or store it in any way.

[ncallaway]

I haven't released it. I've considered it, and it's written in a way that others could use it, but I know if it gained significant popularity it would be a really high value target to compromise. I don't have the time to dedicate to the vigilance necessary to defend that kind of a target.

If you're interested, I'd be happy to share the source with you as a starting point

[climb_stealth]

Sure, if you don't mind that would be great! My email is my hn username @noodfive.com

Thanks!

[ncallaway]

I need to jump through a few hoops to send it, since Github won't let me add a user with read access to a private repository, but I'll send that over tonight

[sagarm]

Some banks now offer proper APIs, allowing Mint to redirect you to their OAuth flow-equivalent. So it's slowly improving.

[pfranz]

I actually think the opposite is happening. Yes, a few are adding OAuth flow-equivalent, but even more are adding 2-factor and other security measures that prevent a third party.

Maybe Plaid has enough clout to prevent it, but other services have dropped functionality for more and more accounts over the past 2-3 years in my experience.

[CraigRood]

IIRC, OpenBanking in the UK requires a re-auth every 90 days with the 2FA. I think thats a very reasonable balance, and means services don't have perpetual access.

[pfranz]

Oh, I'd be ecstatic if that's how it worked. Instead, I have to go through 2FA every time I want to update with some providers. Others say "Apologies, we're working with the provider to restore support" and have stopped working for years. Others I can transfer money but can no longer see the balance.

On rare occasion I see banks that have a revocable token. I can't remember the exact linking process, but presumably they don't get my credentials, I can specify read-only access and specify specific accounts, it need periodic renewal, and from the bank end I can see who I gave permission to and revoke access from there. This doesn't seem common and I'm not sure if all the clients support those banks (or if it's a limited "in-group").

None of this is especially novel. It's just the incentives and efforts aren't there.

[vidarh]

It's frankly annoying. I wish they enforced a notification by some means instead (X still has access unless you do Y), at least for services that only need read-only access.

[ncallaway]

Yep, but unfortunately it's only a small handful of the banks. And it tends to be the larger national banks that offer proper APIs, and I tend to avoid banking with those.

I'd really like to see Open Banking API laws passed in the US to require banks to offer this kind of API.