[dragontamer]

Your #1 tool is simply a stopwatch. Write programs, time them, and then learn. See why some programs are faster or slower than others.

Your #2 tool is hardware performance counters. Its much easier to understand the cache and branch prediction when you use the hardware to count how many times the cache is hit, and when branches are taken.

http://www.brendangregg.com/perf.html#Events

Section 5 of Linux "perf" accesses these hardware performance counters for you.

Write a program, see how many cache-misses it has. Write a slightly different program, the cache-misses will be different. How did it change your #1 measurement (time to complete the program) ??

Now do that a thousand times, with a thousand different programs trying to solve the same thing. Bam, now you're an expert.

Its not really hard. Its just an issue of experience. You'll get there if you work at it, and use the right tools to "see" what is going on.

------

Why do people talk about cache hits and branch prediction? Because in THEIR experience, counting cache hits and looking at branch predictions results in performance gains.

Their experience won't necessarily match yours, but you can still learn from them in general.

[Erwin]

While "perf" is great (especially for something like optimizing a shared library loaded into Python) I had a hard time using any other counter than the instruction retirement for anything meaningful.

"perf list" list 1300 things, and it seems those counters come and go between architectures.

Intel's tools are free now, and give you a very nice graphical analysis tool. You don't need to buy/use the Intel compiler, just have a 1-2 gigs of free disk space.

Here's a page describing analysis if you want to go beyond why some section of your code spends 40% CPU: https://software.intel.com/en-us/vtune-amplifier-cookbook-to...

[dragontamer]

That's true, but "perf" is more portable between platforms.

For example, I can't use vTune at all, because I run Threadripper 1950x. Personally speaking, I use AMD uProf instead.

Without knowing the guy's setup, "perf" is the most portable option. The processor-specific tools will be more accurate, but also more complicated. Its the Linux tool that tries to work across platforms: AMD, Intel, and even ARM platforms.

---------

"Perf" will give you L1 / L2 cache hits, and accurate counts at that. The main issue is "branch prediction", which requires very complicated circuitry to really predict. You need to use Intel's trace analyzer for accurate branch prediction statistics.

For AMD, you need to use instruction-based sampling in uProf for good branch prediction statistics.

L1 / L2 cache stuff is the beginner level though, so Perf should be a good starting point in this whole mess.