[aetherspawn]

We used to place an INT3 vectored exception handler on function entry points and do everything interesting inside the exception handler. This made the execution stack basically invisible to every debugger since it doesn't debug exception handlers. You can enable/disable interrupts and tracing and whatever you need to do inside the exception handler to guarantee that nobody can see what you are doing and/or verify that no other program has registered another exception handler before doing anything interesting.

If you need to hook functions in third party software, this trick can be used to hook the function without modifying any of the functions code. All you need to do is modify some pointer used by the function to zero, and it will raise an exception as soon as something like p-> is executed on that pointer, then your exception handler can execute whatever code you need (i.e. write over stack, write to memory, exfiltrate handles) and on exit all you need to do is restore the correct register containing the pointer and wind back the execution counter by the size of the de-reference instruction.

Please don't use this knowledge to hurt people ...

[newgre]

A debugger sees exceptions before the application does, and it knows which exceptions it should handle by itself (e.g. breakpoints set by the user) vs passing them to the application. I don't have a windows machine rn to verify but I'd expect your assumption that it's impossible to debug an INT3 VEH to be incorrect.

[aetherspawn]

Yes, however you can easily detect when a debugger is attached and avoid placing the VEH. There are only two mechanisms for a debugger to operate: by placing its own VEH (or UHE), or by populating one of the four hardware breakpoint registers, and both are very easy to detect.

[newgre]

Sure there's a gazillion ways of detecting a debugger (specifically on windows), but then we are back to detecting debuggers. My point was that VEH (alone) doesn't prevent any sort of debugging, specifically not the VEH handler itself.

Btw, debuggers (on windows) won't usually install VEH to support BPs, they'll use the win32 debugger infrastructure where the OS manages exceptions and delivers them to the attached debugger object (which again can be detected in several ways). They also do not technically need HW bp registers, although often they will. A simple way to implement BPs is to write 0xCC (INT3) to the text section, then restore original bytes when the INT3 fires.

[aetherspawn]

I don't disagree that it's possible, just in my experience the debugger VEH was fired after my application's own VEH and not before it, so the debugger did not step into the VEH. And if you want to hide something from static analysis it's not a bad way, even if it is meta to "security through obscurity". The barrier to entry for implementation is pretty low, like ~50 LOCs so it's not going to be on the level of like state sponsored malware or something like that.

[15155]

How is the VEH target not immediately visible during static analysis?

[aetherspawn]

The VEH target is visible if you know where to look. You can use something like Themida to virtualize/obsfucate the VEH if you really need the VEH to be encrypted.

It's actually not so easy to find the VEH because if you are injecting the VEH into a third party process from another process, then not only does the VEH not exist during static analysis of the binary at rest, but its program address changes on each execution. Moreover, the VEH can be encrypted at rest before it is injected into the second process.

[harryfyx]

Do you mean using this trick in malware?

[aetherspawn]

Actually our competitor was reverse engineering our LOB app and we used this to protect trade secrets.

[bubblematrix]

No he is referring to happyware.

[layer8]

Shouldn’t that be benware?