[threeseed]

> By the end of this, I expect us to get something that looks a lot like a drop-in remote worker. An agent that joins your company, is onboarded like a new human hire, messages you and colleagues on Slack and uses your softwares, makes ..

I work at a company with ~50k employees each of whom has different data access rules governed by regulation.

So either (a) you train thousands of models which is cost-prohibitive or (b) it is going to be trained on what is effectively public company data i.e. making the agent pretty useless.

Never really seen how this situation gets resolved.

[tkgally]

Are separately trained models necessary for your case? As context windows get longer—Gemini 1.5 Pro now accepts up to two million tokens, and Google has talked of the goal of "infinite" context windows—couldn't a single base model be used with individualized contexts of sensitive data?

[threeseed]

> individualized contexts of sensitive data

My question is whether this capability even exists.

And if it does how robust it is to workarounds.

[tkgally]

That capability probably exists now if you are willing to accept cloud-based models and only moderately-sized contexts. With Claude 3.5 Pro, for example, one can put one’s reference data into a Project and query the model with that data in the context. In my testing, at least, it works quite well. The Projects can be shared among multiple users, too. The context size is only about one-tenth that of Gemini 1.5 Pro, though, and even the latter is probably much too small for most organizational purposes.

Of course, many organizations and regulators would not allow cloud-based models for sensitive data. A possible solution in that case might be multiple instances of an open-weight model hosted locally within the same secure environments as the sensitive data that the individual employees have access to. I don’t know how expensive that would be, whether current open-weight models are powerful enough, or whether context windows for open-weight models can be made big enough to be useful. But at least it suggests a potential path to a solution that doesn’t require training an LLM from scratch for each employee.

[neom]

They won't be used in your business and your business will be less efficient until the regulations change or you end up competing with someone who is willing to ignore the regulations. Also, there are lots of countries without as stringent regulations, it’s not about the inefficiencies that are gone that is the problem, is it about the efficiencies that are created that is the problem. This is a country to country issue, not a business to business issue.

[budududuroiu]

At this point I’m willing to short ANY traditional company that is pivoting to use “AI”. I firmly believe not using AI is the more efficient route, anything else just invites bloat

[neom]

I watched a secretary take business cards and 1 by 1 copy them into our client DB last week. All day. I showed her how to use the ChatGPT app, photo of them all, and convert it into an excel. "You just saved me weeks of work this year"

Should be some fun shorting (we have over 300 secretaries alone, our support staff is massive).

[budududuroiu]

Business card scanners have been around since the earliest versions of the iPhone, but I guess thank you ChatGPT for discovering OCR

[dado3212]

Are there any good OCR packages that are state of the art for general-purpose transcription? (i.e. give it a business card and get it to format it for you, give it a comic and have it transcribe it, give it nutritional info and have it table it)? When I looked recently I pretty much just got GPT-4o as the best API.

[budududuroiu]

Surya https://github.com/VikParuchuri/surya and Marker https://github.com/VikParuchuri/marker

[neom]

Do you have a link to the one that I can put 50 of them on the floor and it will send me back an excel file? I'd like to test it out compared to ChatGPT as I'm going to be implementing "AI" across the whole 700+ person business.

[budududuroiu]

lol good luck doing that with GPT. Right now I can tell you you’ll have missing or malformed or incorrect data, and it will be faster to just pass each one individually through a rudimentary scanner than to sit and figure out which one is correct and which is wrong from the 50 card picture

[threeseed]

I work in banking. Every single country has stringent regulations in this sector.

And fine grained access control is a foundational data governance issue for every enterprise.

[neom]

You think banking won't change? I'd surprised to hear that tbh.