[librasteve]

I have a pressing need to come up with a household budget and had already decided to try using LLMs to help on this task since learning LLMs/prompt engineering is more fun than just writing a dumb script to do accounts.

Thought i would try this tool - and here's a quick review of the experience:

- the quickstart instructions are very clear and I was up and running on my localhost (a mac - but I think this will work well on windows and linx too)

- the UX is good ... slight wrinkle is that the upload button has a down arrow ... also Ctrl+Shift+/ doesn't work on a mac - took me a while to find the speech bubble icon in the bottom right

- love the import / export, love the chat box - worked well with my existing OpenAI account

So - this is a fantastic concept and a well executed MLP - thanks.

That said - and I highly encourage you to keep going - there are a couple of caveats:

1. The task I set is realworld - "please categorize my bank transactions into household expense groups" - and proved too much for my ChatGPT o1 account - most lines were labelled as 'other', bank charges were labelled 'fuel', etc, etc - so the underlying AI engine is not yet ready for this sadly (I am happy to be corrected if others know the recipe)

2. I wonder if using a tool like this, a set of LLM prompts to set up the query and to comb the response would help to chip away at [1] ... so I suggest that having a way for my config to accumulate my prompts maybe a nice feature.

Please do not take this f/back as negative to your work ... it is more my getting to grips with the AI sweet spot.

[tobiadefami]

Thanks for taking the time to try it out and share your thoughts. I really appreciate the detailed feedback from a real-world use case.

Glad to hear the setup was smooth and that the chat box + import/export features worked well for you. Noted on the UI tweaks, I'll look into making them more intuitive.

On the categorization issue, yeah, LLMs can struggle with nuanced transaction labeling, especially without proper context or examples. Structured prompting could help, which ties into your second point -- having a library of refined prompts that can be reused for repetitive tasks would be really valuable.

I love your feedback -- it's exactly what helps improve the tool. And again, thanks for testing it out!

[tobiadefami]

Prompt library has also been implemented and can be opened with ctrl+shift+L; or cmd+shift+L on mac

[librasteve]

having thought about this a bit - my new conjecture is that if I had a way to feed in an example map of transaction payee => category, as one of the prompts, and a way to incrementally add prompts for outliers, then the AI _might_ be able to do a reasonable job - I am planning to mess with raku LLM::Functions to see if I can get this to work

[tim-fan]

Hi I've been thinking about the same thing, in the context of beancount / plain text accounting.

https://www.reddit.com/r/plaintextaccounting/s/BKsaLrfy3A

I already have thousands of labeled examples and a list of valid categories. I'm also hoping an llm will do a reasonable job.

At the moment I'm wondering what to do with all the example transaction data, as it's likely larger than the context window. I guess I could take a random downsample, but perhaps there's a more effective way to summarize it.