With the new generation there is now a difference. This is because this generation has Phis in both addin card (PCIe) and a bootable CPU package (when you run Linux or windows or whatever on the Phi).
Generally with the PCIe one you're running something like OpenCL and with system CPU package you run threads and processes like you normally would.
Technically you could run software directly on the old addin cards since they boot to Linux but you had handle the distribution, running and communication of your software with the host. (you could run any x86_64 binary)
Both gcc & llvm can compile binaries targeting the Phi. Since it runs in Linux (as addin PCIe card or host Linux CPU) it can run any kind of ELF x86_64 binary. They also support generating AVX-512 instruction of the Phi.
Generally with the PCIe one you're running something like OpenCL and with system CPU package you run threads and processes like you normally would.
Technically you could run software directly on the old addin cards since they boot to Linux but you had handle the distribution, running and communication of your software with the host. (you could run any x86_64 binary)