Well, I can (or I could on a personal one, as opposed to the work-supplied one I carry). I like being able to install software from outside of an app store without modifying my firmware, worrying about updating my device, losing my warranty, opening up large security holes, etc.
Reaching outside of the walled garden isn't hard, but there are more things to consider compared to installing an apk on Android.
It's an ideological complaint, not a technical one.
Not being able to install apps is a nuisance; not being able to distribute apps for others to install is a deal-breaker, but the main thing is that I'm afraid of our world becoming a place where walled gardens are the typical model for consumer electronics.
Reaching outside of the walled garden isn't hard, but there are more things to consider compared to installing an apk on Android.