It's more expensive most likely because they're not outsourcing environmental and human rights costs. Also, they are not gathering and selling your data unlike many others.
The only reason it is more expensive is that people will pay the mark-up and they are a for profit company. There is no more reasons than that needed for it to be expensive. The $2 to workers is nice, but the main thing that is used for is marketing as you shared here which allows them to put another $20 mark-up on the item. Of course it is nice that workers gets better conditions, but you'd be better of donating that money instead.
They are outsourcing but they're specifically negotiating contracts that guarantee living wages, ethical worker conditions and ethically sourced raw materials, so that's gonna be factored into costs.
That's the change as of that blogpost: they're using their own initial costings as a baseline so it's still going to be much more than a $2 increase over average manufacturers.
I guess in a typical phone the display is basically a unit that is comprised of the glass, the actual display, and the digitizer sandwiched and glued together; that makes it more than a single component already. And the price difference between the iPhone and the Fairphone one might mostly be demand. There's probably a lot more iPhone display units out there than even Fairphones of all generations.