It's simply a lack of contradiction. If murder is considered bad then murdering to prevent murder is an injustice. If kidnapping is considered negative then taking people away and putting them in cages is an injustice. In other words, it's taking away all of the "if's" and "buts" in what we decide is right. It's a pure (as in pure functional programming) approach to morality. When you get rid of side-effects then things become far more elegant. We'll never stop all of those bad things from happening but the least we can do is to not do them ourselves.
Yes, get rid of punishment. In the context of the bigger picture, people commit crimes either out of a need for resources or because of mental illness. The former is easily solved if we pull our collective heads out of our collective asses and pay for the transportation costs associated with getting food to people who need it. The latter tends to become worse when it is punished, it is best treated with that word beginning with 'L' which everyone seems to be so afraid of. To put it another way, crime is a symptom and the current legal system treats the effects and not the cause.