[CyberShadow]

> There's a disassembler written in D for as3 that I used to use called rabcdasm, did your team ever take a look at that?

RABCDAsm author here. Curious about the same, I've always thought the project might be useful to VM implementers. Also wondering if you've had trouble with obfuscators that employ control flow obfuscation, and create unreachable basic blocks with junk code.

[cpeterso]

Yes, definitely! SWF obfuscators are a problem because they rely on undocumented behavior and implementation details of the Flash plugin that the Shumway developers must debug and support to maintain compatibility. Obfuscated AS1 code can do strange things like jumping to unaligned SWF addresses in the middle of bytecodes.

[CyberShadow]

> Obfuscated AS1 code can do strange things like jumping to unaligned SWF addresses in the middle of bytecodes.

Oh, I've seen much worse. The AS1 VM actually treats the entire SWF file (after decompression) as an AS program. That means it's possible to jump outside of the bytecode tag and into e.g. the metadata of a JPEG file.

[userbinator]

Reminds me of one highly-regarded (at the time) obfuscator whose operation consisted entirely of renaming the existing bytecode tags to a reserved value, and adding a bytecode tag after it with lots of useless code including opaque predicates which eventually jumped out of the tag, backwards, into the original code. A bit of a letdown as I was looking for a challenge at the time, but that jump out of the tag was sure easy to detect and unobfuscate...

(This is something that is still not fixed in Gnash, so Shumway is certainly farther along.)

[mbebenita]

We can detect when bytecode is not well formed and fall back on interpretation, which can handle these nasty cases. This is more of a problem for AVM1 than AVM2.

[mbebenita]

Thanks for the great tool, we used it quite a bit when we first started working on Shumway.

Shumway falls back on an interpreter if the control flow is not reducible.