LLM hype is everywhere these days, but actual usage data is pretty hard to find. I see tons of companies pushing AI tools at me but I have almost no idea what user behaviour/preferences are.
I built a TaxGPT.ca, an AI chatbot for answering people's (Canadian) tax questions and ran a bunch of numbers over the questions submitted between March 4 to April 31, 2024.
Highlights include:
- Over 8,500 questions answered
- Users frequently asked about basic tax info and business expenses
- Feedback is not very common but surprisingly positive
- Experiments in tagging questions in relevant and for 'complexity'
My verdict here is that these bots have potential but you have to have really quick feedback loops and pretty good product chops — you can't just throw AI at some problem and assume users will find their own ways to make it useful.
LLM hype is everywhere these days, but actual usage data is pretty hard to find. I see tons of companies pushing AI tools at me but I have almost no idea what user behaviour/preferences are.
I built a TaxGPT.ca, an AI chatbot for answering people's (Canadian) tax questions and ran a bunch of numbers over the questions submitted between March 4 to April 31, 2024.
Highlights include:
- Over 8,500 questions answered - Users frequently asked about basic tax info and business expenses - Feedback is not very common but surprisingly positive - Experiments in tagging questions in relevant and for 'complexity'
My verdict here is that these bots have potential but you have to have really quick feedback loops and pretty good product chops — you can't just throw AI at some problem and assume users will find their own ways to make it useful.