[landonxjames]

Not sure about Neural DSP or reverbs in general, but real-time neural network based DSP seems very possible. The open source Neural amp modeler[1] would be a good place to start diving in.

[1] https://www.neuralampmodeler.com/the-code

[prashp]

I have tried NAM but with limited success in modeling some time-based effects (e.g. octave shifting). However, I have not tried to model reverb effects.

[intalentive]

To handle time-based effects you need a custom architecture.

https://www.research.ed.ac.uk/en/publications/neural-modelli...

Don’t use NAM. Learn PyTorch.

[prashp]

NAM uses pytorch for its NN implementation?

[intalentive]

It is 100% possible and there are a slew of tricks you can use to get big performance boosts with negligible cost to accuracy.

[prashp]

Do you know what the tricks are?

[intalentive]

1. Don’t use LSTMs (4 vector-matrix multiplies) or GRUs (3 multiplies). Use a fixed Hippo matrix to update state. Just 1 multiply and since it’s fixed you can unroll during training, much faster than backprop through time.

2. Write SIMD intrinsics by hand. None of the libraries are as fast.

3. Don’t use sigmoid or tanh functions as your nonlinear activation. Instead approximate them with the softsign function which is much cheaper.

Depends on exact architecture, but these optimizations have yielded 10-30x improvement for single threaded CPU real time audio applications.

When GPU audio matures all this may be unnecessary.