[lhmiles]

You can download everything and embed it with Google api and dump it in postgres with pgvector. Works pretty well for me for similar situation

[NanaAmun]

I get that, but scale’s the issue, there are over 10k engineers at my current company. I just don’t see how it scales when people are dumping tens of thousands of lines of logs into it daily. You can download everything and embed it using the Google API, then store it in Postgres with pgvector. At this scale, I’m not sure if it would hold up.

[shadag]

I totally agree with you. The solution mentioned above is useful but in my current company there is no way that can actually work long term for an individual engineer or my team.

[shadag]

That's an awesome solution - I'm going try that! Do people have other ways of doing that?

[NanaAmun]

Yes! We just use MS Teams and keep a megathread of links honestly, it’s the best way we’ve found to manage it.

[shadag]

that's insane though. On a team scale this can somehow be managed by company has like 5,000 engineers on the main Teams channel. Surely there is a better way