[jll29]

Interesting. I had a similar experience: once the debugging instrumentation with #ifdef macros was switched off, the code that worked before suddenly crashed. In my situation it had to do with the stack, because my debugging macros used some local/"automatic" variables, and that had concealed the bug before in the DEBUG build.

One thing I also noticed is that using "problem-oriented" languages like Python or Java changes where you spend your time trouble-shooting: ironically, not where the problem is (business logic) anymore, those parts of the code indeed tend to work better, but intead you waste time with libraries (Java: CLASSPATH, Python: packages, all:version conflicts). In Contrast, in C/C++ it was mostly memory management errors and bugs in the actual business logic (the former is also a great distraction, somewhat diminished by the introduction of smart pointers).

[malkia]

At Google, had to do Java on borg, and used few times the "Cloud Trace" debugger (not sure how it was called), but it allowed you to watch multiple instances of your binary in production, and then add some isolated set of java statements around code blocks, this way you can say (somehow) - if you end up on this line, then "inject" (somehow) something to log out... and then you can add whatever you want to be logged (like arguments, variables around, etc.).

But then later it got scrapped, or something like it.

Cloud "debugging" when you have multiple instances is one of the cases where there is no suitable enough debugger (yet).