Are you saying every single variable and function call should be logged every time the code runs? In a dream world sure. While we are at it, let's make it possible to freeze all the state from the production system and let me add breakpoints to rewind in time. In the real world, someone has to make a decision what is noise and what is content.
Unless you are talking about profilers, that measure execution time and memory only, but traces are a lot more than only that.
Annotating the code with logs and traces is a UX activity, not for the end users, but for the ops-team. They don't have knowledge of the internals of the code. Logs should be written in the context of levers that ops have control over.
Take the example from the OP: nr of cache hits. It's something ops can control by configuring the cache size, it is something ops can observe and correlate with request-time and network bandwidth. It would require an immensely sophisticated debugger to make all these correlations automatically.
I wouldn't call it a "debugger", but plenty of people run an instrumentation agent like New Relic or AppDynamics that records tracing information on their production web services with little or even zero modification to their application code.
> I used to use NewRelic APM with Go and it required additional code to instrument.
I'm not surprised, Go's runtime is pretty limited. But for e.g. Java if you're using a well-known/"standard" framework you can just flick the switch and it'll give you a lot of good useful information - additional manual instrumentation usually helps, but the level of instrumentation without it is useful.
Unless you are talking about profilers, that measure execution time and memory only, but traces are a lot more than only that.
Annotating the code with logs and traces is a UX activity, not for the end users, but for the ops-team. They don't have knowledge of the internals of the code. Logs should be written in the context of levers that ops have control over.
Take the example from the OP: nr of cache hits. It's something ops can control by configuring the cache size, it is something ops can observe and correlate with request-time and network bandwidth. It would require an immensely sophisticated debugger to make all these correlations automatically.