I recommend https://electrum.org, even though the website doesn't look super awesome. You'll get a "seed" that generates your keys that is 12 words long. It looks something like this:
"help staple correct horse ..."
From there, just store that seed somewhere secure and you can use that to generate your keys whenever you want, on any machine you want. Personally I keep my seed in two different safe-deposit boxes and one on (shameless plug) my own encrypted text app I made called Onions - http://onionsapp.github.io.
Electrum is great, but not very easy to use (since it's desktop). Get Electrum, Mycelium or Breadwallet, depending on your platform. I've heard good things about GreenAddress, and I recommend a ledger hardware wallet. You can use the ledger with Electrum and Mycelium, so you can spend coins from desktop or mobile. I've also imported my Mycelium wallet onto my second ledger, just so I can move coins from the desktop.
Ledger is pretty good for the price tag it has. It is simlple yet effective. Trezor is a worthy mention since it can be used anywhere(even on an infected computer) whereas you need to setup the ledger on a secure host.
"help staple correct horse ..."
From there, just store that seed somewhere secure and you can use that to generate your keys whenever you want, on any machine you want. Personally I keep my seed in two different safe-deposit boxes and one on (shameless plug) my own encrypted text app I made called Onions - http://onionsapp.github.io.