|
|
|
|
|
|
by brucethemoose2
879 days ago
|
|
|
A lot of the new models coming out are long context anyway. Check out Yi, InternLM and Mixtral. Also, you really want to wait until flash attention is merged before using mega context with llama.cpp. The 8 bit KV cache would be ideal too. |
|
|