[somesortofthing]

I used to feel this way but... honestly, I've found that pressing on with only a vague understanding of what's happening and then diving deep with the agent's own help if it keeps making bad decisions leads to more output of comparable quality. Even without a deep understanding of the topic, you can usually tell when the LLM is BSing and you need to intervene. The model has much more knowledge "present-at-hand" than it'll actually apply to a given implementation, so you can substantially deepen your understanding with minimal reference to external resources by just taking a break from implementation to have a convo with it.

I'm sure this approach breaks down at the very frontiers of highly technical fields but... virtually all work, even work by educated professionals, happens outside that area anyway. On well-trodden ground, you can improve at supervising agents by doing things that test your ability to supervise agents.

[dummydummy1234]

I think there is a difference between things like coding where it is semi closed loop, at the end of the day the software works or not.

Vs fields where there is not a reliable feedback path, or that feedback path is much more noisy.

[somesortofthing]

There definitely is but even then, you can get a feel for a loop for more open-ended tasks too - you move forward until the model output starts to look handwavy/contradictory, then pause to talk to it/consult outside sources to improve your own knowledge. Most "fuzzy" fields also have quantitative components, and it's often worth stopping for a moment to put together some kind of quantitative evaluation suie to give the model grounding. When you've learned the right path yourself, you start moving forward again. It's for sure slower and more error-prone if you were already an expert when you started, but it's workable, and head-and-shoulders better than what you could do without the AI.