They tried this. The predecessor for pledge / unveil was systrace, which included a user-controlled policy file.
One significant reason that pledge was implemented was because it is possible to disable or mis-configure user-controlled policies. Theo mentions this in his presentation that unveiled pledge, and he's basically referring to things like seccomp and systrace: