[TheNewsIsHere]

I’m seeing this second hand at the Fortune 500 my spouse works for.

They are an enterprise SaaS company in SV. Where they have machine learning software that they have been selling for more than a decade, it’s all been rebranded as AI. That’s fair enough from a sales perspective, I guess. What’s odd is that their C-suite and SVPs are pressuring everyone to use LLMs everywhere, for pretty much everything, and none of them seem to understand why it’s only that level of employee that’s seeing any benefit or expressing any interest. My spouse has reported that the running joke across the company is that the executives have jobs that can be done by LLMs, but no one else does. The ICs could not be less interested, and even if they were, legal promulgated a policy against actually putting anything confidential into any LLM other than Copilot in Azure, which the whole workforce reportedly only really uses for summarizing the increasing number of meetings that are perceived as a waste of IC time. A lot of those meetings are “let’s use AI”.

It’s absolutely insane.

[dehugger]

The bit about legal is critical. Im specifically working on "how do we use LLMs in a data-safe way?". Most of the huge gains require exposing information of at least a semi-sensitive nature. For now, that's probably going to shake out as "Copilot or self-host models". I believe that the infrastructure available to self-host at an enterprise level will start growing rapidly due to this trust problem.

Copilot gets the legal nod because Microsoft already has a very, very long history of being trusted with essentially all of an organizations most sensitive information, they have a lot to work with there when it comes to convincing execs that they can safely handle sensitive data in the cloud. (Im am not convinced of that personally).

[TheNewsIsHere]

I agree. Especially with Microsoft, it seems as soon as Satya Nadella promised security was their first priority and he’d stake his compensation on it, no sooner did they get distracted once again.

If you keep leaving the barn door open, eventually your staying power won’t be enough. On the other hand, I’ve stopped thinking to myself “this must be the big one” when it comes to their more egregious recent failures.

> I believe that the infrastructure available to self-host at an enterprise level will start growing rapidly due to this trust problem.

I thought it would go in this direction more rapidly. With the way even the hyperscalers have had trouble securing enough GPUs for existing demand, I’m left wondering if regardless of technical merits, the bubble is going to burst because of basic real world logistical challenges. I’ve actually heard it said in hushed tones that (paraphrased) “the money isn’t the problem. There just aren’t enough GPUs.”

For awhile I’ve assumed that coprocessors or leveraging GPUs at the local level will be the only viable course long-term for widespread adoption. Most actual need for AI/ML technologies aren’t “consume the whole Internet” scale.