Edit and Continue in Visual Studio is similar to that.
You can change the source code while the program is paused in the debugger, for instance in a breakpoint, and continue after your modifications.
Not sure, I have never tried moving the instruction pointer after Edit & Continue.
Moving the instruction pointer backwards has been fine in my experience if you choose a suitable line to reset your state.
Do you get the ability to move the execution point back to before the the error, in order to resume execution? - from what I can tell, this isn't possible - from https://msdn.microsoft.com/library/y740d9d3.aspx#BKMK_Set_th...
"Setting the next statement causes the program counter to jump directly to the new location. Use this command with caution:"
"If you move the execution point backwards, intervening instructions are not undone. "