[raverbashing]

Smartcards are vulnerable as well and there has been successful attacks towards some smartcards (google it)

And not every system has an HSM available

[ra]

Sure but smartcards and HSMs are slow and in a properly managed environment are much safer than memory.

Whether or not the OP has an HSM is moot. The OP said, "I want to design and implement a solution as secure as it can be" ... and that means (among many other things) keys on HSM.