Hacker News new | ask | show | jobs
by nomel 563 days ago
They mean the non-normalized probabilities for each tokens is available. Many API give access to the top-n. You can color the text based on it, or include it in your pipelines, like trigger looking externally, or inject claims of uncertainty (the same things I do). It's not remotely guaranteed, but it's some low hanging fruit that can sometimes be useful.

One of these days, someone will figure out how to include that in the training/inference loop. It's probably important for communication and reasoning, considering a similar concept happens in my head (some sort of sparsity detection).