[variadix]

My understanding is that this doesn’t work for back propagation because it needs to know the forward pass computation exactly to adjust the weights. Analog computation introduces error in the form of noise (thermal, environmental, etc) which makes training on analog a non starter. It probably works for inference if the error is low enough, but training is where most of the power is spent unless it’s massively deployed.

[heisgone]

Thanks. That's interesting. If I understand correctly what you say, analog cannot be used for training but might be used to bake a trained static model. I think things are evolving too fast now for pre-trained models to be baked in silicon but it's a matter of time for it to become common.

[variadix]

There’s probably a way to make an analog AI chip programmable, but yeah the wiring (model architecture) is fixed when you fabricate/design the circuit. I think we’ll start to see simple models deployed to edge devices soon where low power solutions will be a necessity.

[tbalsam]

You will get error regardless of whether it is analog or digital.

Digital trades bandwidth for error correction, but this does not mean it is better as many ML algorithms are noise-tolerant and would benefit greatly from the increased throughput.

Like everything, I believe there are simply just tradeoffs.