Actually this is why most of the hifi heads bought iPhones, 6s had the best DAC that you can buy for a reasonable price (less than 1000 USD). I have converted all my collection to ALAC just to be on the Apple platform and be able to listen to lossless on my phone. I also use a decent entry level hifi headphone (Focal Spark - wired) this is where my hearing still be able to identify differences. The entire setup was not too expensive and it had great performance. Now, if I try to have it with the recent iPhone offerings, it will be certainly more expensive and slightly shittier, and I can just convert my entire collection to mp3s because the system does not support it anyways. On the top of that, I was able to listen to my collection on my proper audio system at home before while right now I am not able to connect the phone to the amplifier anymore. So, to summarize, Apple offers my a way more expensive solution that makes me miss out on many aspects of listening lossless music. It is simply not worth it.
The digital source material is a limiting factor for achievable quality, but so is the DAC that is to translate the source material. Some cheap DACs are not meant to translate specific bit depths and sampling rate, some supply the jack with less output power, some exhibit more crosstalk, some have more limited frequency responses. So yes, second to the source material, it's the DAC that is the limiting factor for good sound.
It's the speakers that is the limiting factor, by an extremely wide margin.
A cheap DAC can produce around 16 bits and 20 kHz. It's not difficult. A pair of $10 earplugs is much more limited in what sound it can physically play.
It actually isn’t that easy, like at all. A tiny DAC in a phone amplified to loudspeaker levels is going to sound qualitatively different from a stronger signal out of something built to that spec. Not to mention interference issues in analog signals at low power.