Hacker News new | ask | show | jobs
by numlocked 782 days ago
I'm seeing this as well. I don't quite understand how it's doing that in the context of LLMs to date being a "next token predictor". It is writing code, then adding more code in the middle.
2 comments
sebzim4500 782 days ago
Are you seeing a `<span>...` suffix when the model is in the middle of writing a code block? I'm pretty sure that's just a display bug in lmsys.
link
concurrentsquar 782 days ago
Is it something similar to beam search (https://huggingface.co/blog/how-to-generate#beam-search) or completely different (probably is not beam search if it's changing code in the middle of a block)?

(I can't try right now because of API rate limits)

link