[andreimatei1]

Print debugging is the tool most people reach for when they can, but its biggest problem is that you have to change the source code to add the printfs. This is impractical in many circumstances; it generally only works on your local machine. In particular, you can't do that in production environments, and that's where the most interesting debugging happens. Similarly, traditional debuggers are not available in production either for a lot of modern a software -- you can't really attach gdb to your distributed service, for many reasons.

What print debugging and debuggers have in common, in contrast to other tools, is that they can extract data specific to your program (e.g values of variables and data structures) that your program was not instrumented to export. It's really a shame that we generally don't have this capability for production software running at scale.

That's why I'm working on Side-Eye [1], a debugger that does work in production. With Side-Eye, you can do something analogous to print debugging, but without changing code or restarting anything. It uses a combination of debug information and dynamic instrumentation.

[1] https://side-eye.io/

[surajrmal]

How is side-eye different from dtrace?

[andreimatei1]

Side-Eye is massively inspired by DTrace in some of its raw capabilities and the basic idea of dynamic instrumentation. Beyond that, they're very different. At a low level, DTrace is primarily geared towards debugging the kernel, whereas Side-Eye is about userspace. DTrace's support for the DWARF debug information format used on linux is limited. The interaction model is different - for DTrace you write scripts to collect and process data. DTrace works at the level of one machine, whereas Side-Eye monitors processes across a fleet. In Side-Eye you interact with a web application and you collect data into a SQL database that you can analyze. Side-Eye is also a cloud service that your whole team is supposed to use together over time.

And then there are more technically superficial, but crucial, aspects related to specific programming language support. Side-Eye understands Go maps and such, and the Go runtime. It can do stuff like enumerate all the goroutines and give you a snapshot of all their stacks. We're also working on integrating with the Go execution traces collected my the Go scheduler, etc.