By minimizing the role of the oracle, instead of relying on one entity, you can rely on the market. Anybody could be the oracle, they'd put their reputation/offer/price on sale and the stakeholders would select one as they wish.
You're still trusting the selected oracle to behave honestly, but they will be operating in a very competitive market which hopefully would lead to better behaviour. They might cheat once, but it would be trivial to switch to a better oracle next time.
If a smart contract relies on an external data source from a normal server (oracle), then why even take the risk of deploying the smart contract?
If you're using an oracle for a data source you might as well do everything on a normal database.