Computers aren't magic, much as they seem to be. In general, if you can reproduce a bug reliably, it is always possible to keep digging until you find the cause. It may take more and more esoteric tools to do so, and you may need to start breaking apart the components you're working with, but at least ask yourself the question "what is preventing me from examining what's going on here" and try to find out how to look inside whatever opaque box you've run up against. Those boxes will start out being like "the function I just wrote", but as your understanding increases and you become familiar with how to look inside the least opaque boxes, the hurdles become more like "the JavaScript runtime", "the C compiler", "the syscalls made to the operating system" or even "the microcode emitted by the CPU" (though I've never needed to go to that level myself!).
Further nit: Modern x86 CPUs use PLAs to generate microops ("uops") that feed the execution engine. For complex instructions (like CPUID, RDMSR, VM-related, etc.), the microcode ("ucode", which can be updated) generates microops.
The execution engine works in discrete[a] operations, hence uops. The microcode is a sequencer that tears apart ISA instructions into those uops.
[a]: I'm considering fused operations (like CMP+Jcc) as single "operations" for simplicity.
By spending a lot of time on the problem and not panicking, essentially.
It you don't rush and focus on making progress, you will find it eventually. There is no wizardry involved, skills will help you find the solution faster, but that's your ability to focus and keep your calm that will get to it in the first place.
If you are at work, ignore your boss trying to pressure you (that's not necessary easy), your boss only has the "right" to tell you if you should work or the problem or not, if you are asked to work on the problem, do it as if you have your entire life to do it. With experience, you can make estimates that can help decide if the problem is worth trying to solve or not, but with the correct mindset, one doesn't need experience to solve the problem itself, instead one builds experience by solving the problem.
I learned more about the details of how computer worked in a class on programming an Intel 8080 (on a CP/M system with dual 8" floppy drives) than any other course. Computer science majors were either nascent or non-existent at that time so I don't know what is available today. And microprocessors are a lot more complex than the 8080.
Nevertheless, I recommend learning the details of computer architecture by learning some assembly programming. Higher level languages are an abstraction meant to shield the developer from the details of the H/W but it remains useful to understand the details, particularly if the abstraction is broken.
Do students get much practice in debugging and root cause analysis on existing systems? That's something people do a lot of at work, but perhaps not students so much. I'd imagine students work more in a clean "green-field" environment.
Kind of like experience gained as a plumber working with dirty clogged real pipes in a basement, vs doing exercises in plumbing school with new pipes and theory about pipe sizes, gradients etc. In other words, school can only give part of the picture.
> Do students get much practice in debugging and root cause analysis on existing systems?
It's a waste of a student's time. Deep debugging isn't just one set of skills; each problem is different. It's usually very time-consuming, and you will need to learn new skills and tools.
That is: you have to encounter a problem that is a blocker, so that your motivation is that you have to solve the problem.
My experience was a long time ago: I had linked a library compiled with Borland C with code compiled with Microsoft C. It wouldn't work. I wss only using one function from the Borland library; that's where the error was occurring. It turned out that the Microsoft compiler required the callee to restore the flag register; the Borland compiler required the caller to do that. Therefore the carry bit wasn't being restored correctly, causing the bug. Took several days to figure out.
If you're writing a language it supports, use godbolt - https://godbolt.org/. It's great for settling arguments (do these two versions compile to the same code? If not figure out why! Make sure you're compiling with optimizations)
Spend some time learning gdb and other classic cli tools.
Short answer is start doing things. Somewhat longer answer - develop interest in understanding deeper level things because when things seem hopeless it's the interest/curiosity that will keep you going. Secondly find things to break and fix - this will give you opportunities to learn wider range of things.
I personally got there by treating bugs as problems I could try and ask questions about until I had a smaller more precise bug. So either I work out something to ask and answer about the nature of some output, or I try to ask and answer something that helps me more precisely pin down the bug.
Sometimes that can be turning an intermittent problem into something I can reproduce consistently, and sometimes it can be constructing new test cases and seeing if their results match my hypothesis.
It helps to either have a good memory, or to keep notes so you know what you’ve already explored and tested, and it’s something you get better at with practice. I spent several years on third line product support working out what was going wrong for customers, producing hot fixes, and working with devs to turn those into proper fixes in the next version.