|
|
|
|
|
by haltist
933 days ago
|
|
The main assumption of techno-optimism is that a large enough computer can do anything people can do and it can do it better. The goal of techno-optimism is to create a mechanical god that will rule the planet and scaling LLMs is a stepping stone to that goal. I, of course, already know how to do all this for a mere $80B. |
|
If you've made chips with latches and LUTS, any performance data you can share, no matter how old, would be helpful
It's an idea that's been bouncing around in my head since reading George Gilder's call to waste transistors. Imagine the worst possible FPGA, no routing hardware, and slow it down even more with a latch on every single LUT. Optimize it slightly, by making cells with 4 bits in, 4 bits out (64 bits of programming per cell), with the cells clocked in 2 phases, like the colors of a chess board. This means that each white cell has static inputs from the black cells.... and is thus fully deterministic, and easy to reason about. The complement happens on the other phase. Together, it becomes turning complete.
The thing is, it does computing with NO delays between compute and memory. All the memory is effectively transferred to compute every clock cycle. The latency sucks because you'll take n/2 cycles to get data across an N*N grid. However, you'll get an answer every clock cycle after that.
Imagine a million GPT-4 tokens/second.... not related to each other, of course, but parallel streams, interleaved as the data streams across the chips.
Imagine a bad cell.... you can detect it, and route around it. Yields don't have to be 100%.
The extreme downside is that tools for programming this thing don't exist. VHDL, etc... aren't appropriate. I'm going to have to build them. I've been stuck in analysis paralysis, but I've decided to try to get Advent of Code done using my bitgrid simulator. I hope to be done before it starts again next December. ;-)
[1] https://bitgrid.blogspot.com/
[2] https://github.com/mikewarot/Bitgrid