Hacker News new | ask | show | jobs
by fastball 2 days ago
Will they? This really isn't different from how humans interact with each other. The vast majority of lying is not people being explicitly asked to lie in some form, it is incentives which make lying appealing. That is what OP said and that is indeed what the constraints are incentivizing. Sure, you can say "well lying isn't incentivized to a moral agent"! And sure, that's true. But that's not how humans work either.

Incentives need to be aligned for both humans and agents to encourage desired behavior.
1 comments
soulofmischief 2 days ago
They will if they seek to master their tools, both to help them identify subtext in agent responses, and to help them modulate their own responses to achieve the desired outcome. As it currently stands, most engineers I've interacted with don't have these skills down. This subtle latent space is where prompt engineering is moving towards, as RL has created models capable of increasingly sophisticated long-horizon tasks with much less hand holding.

Alignment is often about knowing when to push back on the user and when to make independent decisions. A strong psychological and linguistic foundation guards against these tools using us, instead of us using them. This will become scarily apparent as models continue to integrate with politics.

link
fastball 2 days ago
What I meant by "will they?" was "will they any more than a human already needs to in order to understand other humans?"

I don't think this is legibly that different from human behavior, so if new graduates didn't need those things now why would they need them later (or vice versa).

link
soulofmischief 2 days ago
It's probably true that many programmers in the future will get away with a similar lack of fundamental knowledge that today's programmers get away with. To some degree, we all have blind spots, but I think if agentic processes are here to stay, as long as humans remain in the loop at all it would serve us to master a semantic capability closer to that of the models we work with, lest we lose control either in taste or in a manner more serious. The most effective engineers will understand that.
link