[falcolas]

For better or worse, AI is a combination of machine learning algorithms. And these algorithms are black boxes solely because we don't add observability to them - we aren't looking.

But there is a desire to understand why an AI provided the output it did (to increase trust in AI generated output), and so there's a lot of study and work going into adding that observability. Once that's in place, it becomes pretty straightforward to identify which inputs to a model provided what outputs.

[Rekksu]

I have never seen an ML researcher claim that understanding the effect of specific training inputs on outputs is straightforward given the size of these LLMs. Most view it as a very difficult if not impossible problem.

[falcolas]

And yet it's a major part of the overall concept of being responsible with our use of AIs. Throwing our hands up in the air and prematurely declaring defeat is not an option long term.

It's a non-starter for no other reason than potential copyright infringement means the government becomes involved, and they will stomp on the AI mouse with the force of an elephant - the opinions of amateurs and the anti-copyright movement notwithstanding.

As such, AI Observability is a problem that's both under active research, and the basis for B2B companies.

https://censius.ai/wiki/ai-observability

https://towardsdatascience.com/what-is-ml-observability-29e8...

https://whylabs.ai/observability

https://arize.com/

[janalsncm]

Observability is great but it doesn’t give granular enough insights into what is actually happening.

Given a black box you can do two things: watch the black box for a while to see what it does, or take it apart to see how it works.

Observability is the former. Useful in many cases, just not here.

If you want to know what LLMs are actually doing, you’ll need the latter. Looking at weight activations for example, although with billions of parameters that’s infeasible.

[Rekksu]

Those companies are not solving the problem you are describing

[falcolas]

"No they're not" and "no it's not" (simplified from the actual response) are conversation enders, so I'll follow the lead and let this conversation end.

Have a great new years!

[Rekksu]

Well it helps to be accurate in your statements