[jiocrag]

It’s literally addressed in his first bullet point…

“ Response: While there is evidence that some tasks that appear emergent under exact match have smoothly improving performance under another metric, I don’t think this rebuts the significance of emergence, since metrics like exact match are what we ultimately want to optimize for many tasks.”

[ljlolel]

You haven’t read the paper so you don’t understand what I just wrote and why Wei is not responding to the paper. It’s not a different metric

[jiocrag]

I did read the paper and Wei is responding to it, and others. What do you mean "It's not a different metric?"