I'd even want to put particularly nicely shaped rocks on the "don't" side of the line. Kill a plant to save a rock? It's called building maintenance. Kill a short-lived animal to protect a mighty old tree that has seen entire cultures come and go?
Sure! There might even be some rocks that I would hesitate to destroy if it saved a human life, maybe not even excluding my own. The beautiful thing about life is that in the grand scheme of things, only avoiding extinction matters.