Hacker News new | ask | show | jobs
by ypcx 1725 days ago
GPT-3 learns these rules by itself.
2 comments

No it doesn't. It generates text that looks like it has learnt common sense rules, which is both subtly and importantly different.
How would you define "common sense rules"?
This Wikipedia article is a good start if you're curious: https://en.wikipedia.org/wiki/Commonsense_knowledge_(artific...
Amazing how our own brain can delude us by creating a feeling that we understand something because (apparently) we have produced a clear definition of it.

Just a hint - each of these "facts" or "rules" mean slightly something else depending on the context they are used in. This context is not hard-encodable because it slightly mutates with every new information learned or forgotten.

can you automate your fate with gpt-3? that is where the money lives.

will sublime conclusions made by GPT-3 EVER be trusted if the reasoning to its conclusion is not understood by a human? perhaps the gestalt of gpt3 implies a meta gpt3 that could derive human grokkable explains of its "dumber" self. or maybe not.

Current set of tools in explainable AI do allow deep neural nets to tell why they gave the response that they did to a certain extent. Contact me if you want to see it applied to text based DL models, time series based DL models or Image based DL models.