Have you ever seen Shawshank Redemption? Sometimes you can tell when someone is truly rehabilitated. Yes, it is a hard problem, but that doesn't mean we shouldn't attempt to employ a system more just for all involved.
I think basing our ideas about incarceration on a Stephen King movie and novella isn't the wisest strategy, any more than using The Silence of the Lambs would be.