If you want to compare apples-to-apples, we have done a comparison with realistic (and not synthetic) data regarding the performance of GPUs and FPGAs.
https://medium.com/@inaccel/faster-inference-real-benchmarks...