[cejast]

> All my finance and payment domain expertise, all the debugging intuition and distributed system knowledge earned through hours of sweat and tears, is now promptable.

Is it really though? Access to information is quicker, but you still need to know what ‘good’ looks like to leverage it effectively. I can prompt my way to a medical diagnosis, but I’d still want to run it by a doctor.

[zmgsabst]

I’ve found it extremely hard to get LLMs to exit the basin of your current knowledge.

One of my tests for new models is to ask about a concept I already know the mathematical model for, but as if I don’t. So far, they all answer the same way:

1. Convoluted explanations about how it kinda-sorta is common terms.

2. If you follow up with the correct mathematical term, it immediately claims that’s correct and the right way to model it.

3. If you ask it why it didn’t use that term for your question, the LLM gives some version of explaining that it tried to match your language.

I have no choice but to assume the model behaves similarly other times — and that I am largely trapped in a basin of my own ignorance, when using LLMs.

[slyzmud]

I don't think that's a good analysis.

If the LLM is wrong and gives you a wrong medical diagnosis you end up hurting your health. If an LLM gives you a wrong debugging answer you've just lost 5 minutes.

Software engineering is the only knowledge work where mistakes are usually inexpensive except for data breaches. Outside for that nobody cares for bugs.

That's not true in most other knowledge jobs. If a lawyer uses AI and hallucinates something there is a legal problem. If someone vibecodes an app and crashes, it can be fixed with more AI and try again

[cejast]

That’s my point though? Debugging a 5-minute problem is in the shallow end of the spectrum, the real complexity sits where they lean on their domain experience. Finance and payments software mistakes can absolutely be expensive.