What you need is to hold your private key. You could, for example, holds your private key somewhere else, and inject that private key to your phone when you need to use it.