[Centigonal]

Very interesting! The one thing I don't understand is how the author made the jump from "we lost the confidence signal in the move to 4.1-mini" and "this is because of the alignment/steerability improvements."

Previous OpenAI models were instruct-tuned or otherwise aligned, and the author even mentions that model distillation might be destroying the entropy signal. How did they pinpoint alignment as the cause?

[mlin4589]

Good question! We do know from OpenAI's system card from GPT-4 that the post-trained RLHF model is significantly less calibrated compared to the pre-trained model, so it's a matter of speculation that something similar is occurring. However, it's more of a hunch more than anything. I would be curious if it's possible to reproduce this behavior, or the impact of distillation on calibration.

Disclaimer: I wrote this blog post.

[itchyjunk]

Could you please elaborate what less or more calibrated means here? Thanks!

[Scene_Cast2]

For binary labels: you take a slice of labeled data. The mean of the ML model prediction on this data is different from the mean of the label. In practice, often a synonym for "loss is worse / could be better".

Not sure if that's what the GP meant, I only worked with binary labels stuff.

[mlin4589]

Calibration (in a binary context) basically means that the confidence of a model/score matches the probability that a particular label is positive or not.

For instance, a calibrated classifier for a coin flip predictor should output 50-50. A poorly calibrated classifier would output higher confidence for heads/tails.

[Workaccount2]

Wouldn't it be something if AI parlance crept into common parlance...

[bluefirebrand]

Great Observation!

It would probably erode trust between people interacting online. Many of us are here to discuss issues with real people, not AI agents. When real people start to mimic the conversation parlance and cadence of AI agents it becomes much more difficult to trust that you are interacting with a real person

Personally I'm not interested in chatting with AI agents

I'm not even really interested in chatting with real people filtered through AI agents. If you can be bothered to type out a prompt to your AI you can take the time to write your own thoughts

I don't even want to read things edited (sanitized, really) by AI either

The same way I don't want my living space to resemble a too-clean laboratory, I don't want my conversation space to resemble an HR meeting. I want to interact with the messy side of people too. Maybe not "unfiltered", but AI speak is much too filtered and too polished

I chose every word in this post myself with no help from AI, then typed it with my thumbs, just like god intended

[dinfinity]

> Personally I'm not interested in chatting with AI agents

Why, though? If the AI agent is making sense, then what does it matter?

For certain types of conversations I've had more interesting conversations with AI than with a solid 90% of people I've ever interacted with. Really not that surprising given that most people have only an average grasp of most things and a poor grasp of very specific things.

[bluefirebrand]

> Why, though? If the AI agent is making sense, then what does it matter?

The same reason I prefer having sex with humans and not blow up dolls

If your only goal is to get off, then the blow up doll does the job. If all you care about is having an interesting conversation then I guess an LLM is fine

I care about human connection. I have no interest in spending time interacting with machines instead of people

[dinfinity]

That is a silly comparison.

1. Humans and blow up dolls feel massively different, physically. 2. Blow up dolls don't do anything autonomously.

The comparison would have to be with a sex bot that is virtually indistinguishable from a human when having sex with it, just like text chatting with an AI bot versus chatting with a human can be.

What human connection are you and I currently forming? Does it really matter that I am a human for this interaction we're having?

[Der_Einzige]

Skullface sends his regards: https://arxiv.org/abs/2409.01754v1

I literally see it with the huge amounts of people now using "delve" much more or are using ChatGPT-ish linguistic style in their personal communication. Monkey see, monkey do.