Hacker News new | ask | show | jobs
by Mordisquitos 1703 days ago
To be fair, if a real human were to answer the question "How many hydrogen atoms are in a water molecule?" time and time again, it would be very easy for them to accidentally reply "two" when asked the same question about oxygen.

The real question is, after the model mistakenly replied "two" to your question, did it also internally trigger the neurons for "Wait a minute..." while inhibiting output?
2 comments
hervature 1703 days ago
Running the model multiple times doesn't reinforce the model. In general, you should not anthropomorphize algorithms as human cognition does not give any bearing on how algorithms work.
link
Scene_Cast2 1703 days ago
It can. Check out "zero shot learning" -> both sentences would be part of a single "evaluation", and the first sentence would prime for the output of the second. (You basically combine multiple "evaluations" into one, and context is held in tensors / blobs)

https://towardsdatascience.com/zero-and-few-shot-learning-c0...

link
hervature 1702 days ago
Sure, but I feel like we're talking about different things. I consider "context held in tensors" as part of the model. That is, if you zero out these registers, then the model evolves in a deterministic way every time. In this case, when you perform a query, I assume those tensors are always initialized before your query.
link
drran 1703 days ago
How are in a water molecule?

> two hydrogen atoms

water molecule?

> isotopes of hydrogen

How water molecule?

> is arranged in a tetrahedral structure

link