[cuffe]

Hi all, eng who worked on vectors @ Retool here. We’re excited to get HN’s feedback on a new product I’ve been working on: Retool vectors. There’s been a lot of hype around AI, but as I was thinking about what we could launch, there were really very few AI apps that are today _actually_ delivering business value. For example, there are tons of chatbots, summarization bots, etc., but it wasn’t clear how to use them. There were bots that we could use for answering support tickets (we use Intercom), but when we tried using them, the results were far worse than expected (it only managed to close ~5% of our tickets, and required substantial human intervention).

I then tried building a custom AI chatbot for our support team, powered by a vector database (with previous answers, our docs, and community forum data). In my prototypes, the successful response rate was much higher (around 5 - 7x higher, ~30%). But it was surprisingly hard to build this bot: I needed to stitch together vector databases, custom integrations with Intercom, cronjobs to sync data, etc.

That’s why we built Retool Vectors. The idea is that we want to build a vector database that has full ETL from whatever inputs you have (e.g. Intercom, a postgres database, Salesforce, a community forum, a website, etc.). It’s always kept up to date. (We’re still working on some of these features, but I decided to try and launch this week because I just wanted to get feedback from HN, haha.) I think the industry has now settled on vector databases as the best way to provide context to LLMs. I hope that Retool Vectors can be a much easier way of getting data into it.

If you have any feedback please let me know!

[airstrike]

>But it was surprisingly hard to build this bot: I needed to stitch together vector databases, custom integrations with Intercom, cronjobs to sync data, etc.

>That’s why we built Retool Vectors.

Reminds me of Tobi Lütke who founded Shopify like that. When he built an ecommerce store to sell snowboards and realized how hard it was to set that up, he thought it was better to sell ecommerce stores than to sell snowboards

[hosh]

This looks exciting! I was at a shop where we adopted retool to take the burden off of the engineering team for internal tooling, and I think this is taking it to the next level. Looking forward to see more.

[yayitswei]

Congrats on the launch! Definitely felt the pain of building a custom ETL into a vector db for a recent LLM project. Is there a way to use this independently of the Retool UI builder?

[cuffe]

Thanks! Yes so under the hood it's all in a postgres DB with a pgvector column + some metadata. If you go to retool database, you can grab the connection string and use the embeddings with your own apps. I'd like to open this up to bring your own database and integrate with other common vector DBs (Chroma, Pinecone, Mongo vector, etc) next

[benpopper1]

Would you like to write about the process of creating this for Stack Overflow blog or share some quotes we can use? We're planning to publish a few things on vector DB, embedding, and RAG in the near future.

[cuffe]

Absolutely, happy to help! jamie at retool DOT com

[gk1]

We can help: [e-mail]

[cuffe]

Awesome - would love to see how we can integrate here. Will reach out

[mritchie712]

I'd imagine this is exactly what intercom is doing for their AI. Any ideas why the performance is worse? Maybe intercom didn't have access to forum data?

[cuffe]

TBH, I was pretty surprised too. It made me pretty skeptical of off-the-shelf AI apps in general. I now think that most actually effective AI apps will need to be developed in-house, and that “bolting on” AI to existing apps (e.g. Intercom, Salesforce, etc.) won’t work. I think there are a few reasons:

1. A lot of the useful data for answering questions is in our public docs and community forum answers, which Intercom doesn’t have access to. (And we wouldn’t feel comfortable giving them access to our internal Slack anyhow.) For example, we’ve debugged complicated OAuth issues in Slack, and there is a lot of “context” there that is helpful for answering future OAuth questions (but isn’t available to Intercom).

2. Intercom doesn’t allow you to customize prompts or customize context easily. In our case, for a highly technical product, “prompt engineering” allowed us to radically improve answer quality. We could also use chain-of-thought prompting, which Intercom didn’t support. Together these two improvements probably doubled the answer success rate.

3. We needed to integrate with our data warehouse for in-product context. For example, if a customer has an error with a particular product/feature, knowing what plan they’re on, which features they’re using, which feature flags are enabled, etc. is quite helpful.

[civilitty]

> (And we wouldn’t feel comfortable giving them access to our internal Slack anyhow.)

I can't wait till people start using LLM chat bots with RAG to exfiltrate private Slack conversations.

[beardedwizard]

Given how many projects like this exist, is this just to generate leads for retool sales? Does this actually make money?

[cuffe]

When I was building our support chatbot powered by a vectorized DB, I couldn’t actually find a project that helped with syncing Intercom chat data, Discourse community forum data, and web crawling data into a vectorized database. Do you have any suggestions? This was really a “scratch my own itch” product (not something to “generate leads for retool sales”), so genuinely asking out of curiosity! (If there are better products maybe we should kill this project and just recommend people use that instead, haha.)

And no, this is definitely not making money. (It’s free after all, so I guess we’re probably losing money on this.) To be honest, as an engineer @ Retool, I work on things because I think they’re cool and could be useful to our customers… not because it has to make my company money.

[beardedwizard]

Not hating, just asking - thanks for the answer! My personal observation is there are a bunch of llm + vectordb solutions, I agree the other integrations are key, but I think we both know they are tractable problems given sufficient motivation. I've also been following some of the push back on vectordb as the best way to solve this kind of problem. Overall, I was hoping to understand what retool stood to gain with this approach and I got the answer :).