Understand the new CEO's motivations.
Create a new board seat for the CEO: http://www.venturehacks.com/articles/ceo-board-seat
Bring him on board as an Advisor or Director first and see how that works out.