Hacker News new | ask | show | jobs
by euclaise 1013 days ago
RWKV also uses some sort of L2-esque regularization, which was supposedly an idea taken from PaLM (although I can't find a source on this point, other than some message in the RWKV discord)