[ric2b]

> because no intelligent system would think it could produce paperclips infinitely

This is irrelevant, being turned off while it's still possible to produce more paperclips would not be the optimal strategy for the machine trying to maximize paperclip production.

> and it would already have an "off feature" which was designed for a purpose

Look up what the control problem is. We don't know how to design such a feature for a General AI that also lets it do it's job effectively. It's not as simple as it looks. If you're not talking about a General AI than sure, it's easy, but non-General AI's are not very scary.

> We already replace humans at their jobs all the time and they only very rarely kill their masters to prevent it.

But humans don't have "maximize my production for this company" as their main life objective. Things like food, not going to jail, not dying, etc usually come first.