One advantage of FastCGI is that it's inherently easy to scale horizontally and load balance because it basically forces the apps to have no state. This is one of the big benefits that PHP happens to have for that reason.
It's possible to use http/2 between the load balancer and the app server (haproxy 2.0+ supports this, caddy supports h2, but not h2c). FastCGI has been around longer, and there are probably more reverse proxies that support it on the backend than h2c (or even just h2), but I'm not sure if there are any technical advantages to using FastCGI as the protocol over http/2.
There is no difference between the two because they are the same thing, just architected differently. The networking is done by, say, nginx rather than a server that is part of the application.
fastcgi applications can "scale to 0" and cost nothing when they are not used. An application server must always have at least one open socket on a running machine.
fastcgi would probably do better over http/2 than other protocols not designed for multiplexing.
One advantage of FastCGI is that it's inherently easy to scale horizontally and load balance because it basically forces the apps to have no state. This is one of the big benefits that PHP happens to have for that reason.