Probably for the sum, or weighted average of happiness of all conscious entities on the planet. This may include ants, dogs, humans, and non-biological consciousness/beings.
If an AI were created to run things, I feel like it would more likely make us stop using hydrocarbon fuels before it would go all terminator on us, as that would take into account our natural right to life right along with all other creatures. That is, if we program it that way. Asimov's The Evitable Conflict from "I, Robot" fits right along with what you're saying about there being some sort of weighted average driven decision process.
Although, I fear we'd probably end up with a "Butlerian Jihad" instead...
Although, I fear we'd probably end up with a "Butlerian Jihad" instead...