|
|
|
|
|
|
by zhemao
167 days ago
|
|
|
TPUs aren't transformer ASICs. The Ironwood TPU that Gemini was trained on was designed before LLMs became popular with ChatGPT's release. The architecture was general enough that it ended up being efficient for LLM training. A special-purpose transformer inference ASIC would be like Etched's Sohu chip. |
|
|
https://cloud.google.com/tpu
> A TPU is an application-specific integrated circuit (ASIC) designed by Google for neural networks.