Hacker News new | ask | show | jobs
by vlovich123 448 days ago
What’s iSWA? Can’t find any reference online
2 comments
imtringued 448 days ago
Gemma 3 has some layers with a context size of 1024 tokens and others having full length. You need to read the Gemma technical report.
link
nolist_policy 448 days ago
interleaved sliding window attention
link