[rdtsc]

Linux will often still schedule kernel threads to run on those cores so they are not totally isolated. Also cache effects. If your architecture shares caches between cores, sometimes it would be worth "wasting" a neighboring core to avoid ssh and gdb thrashing your cache.

Oh and also don't forget to set up IRQ affinity to avoid any of those cores to handle.

There is an interesting research done by Siemens, that takes this kind of isolation a step further and uses virtualization extensions to isolate resources (cores for example):

https://github.com/siemens/jailhouse

[erichocean]

> Linux will often still schedule kernel threads to run on those cores so they are not totally isolated.

Have a look at cpuset[0]. You can forcibly move kernel threads into a given set (even pinned threads), and then force that set onto whatever cores you want.

[0] http://www.kernel.org/doc/man-pages/online/pages/man7/cpuset...

[rdtsc]

Ah good stuff thanks posting that. I'll have to look into it, I haven't tried cpusets yet.

In the past I tried migrating kernel threads, the migration did work but system became unstable, so I gave up on that.

[nkurz]

Oh and also don't forget to set up IRQ affinity to avoid any of those cores to handle.

Perhaps this is what you meant, but this is straightforward. Simply disabling 'irqbalance' is a simple way to do this. Alternatively, you can also configure it to cooperate with 'isolcpus' by using 'FOLLOW_ISOLCPUS'.

[rdtsc]

Disabling it will disable auto-balancing it, it will just become static, which might not be the exact configuration you want.

[nkurz]

Yes, if you don't want all interrupts to handled by CPU0 you'll need a different approach. But doing anything else may be more difficult than it sounds. Do you know if the bug mentioned here is fixed?

http://serverfault.com/questions/380935/how-to-ban-hardware-...

http://code.google.com/p/irqbalance is 403 for me and I haven't been able to check.

[rdtsc]

Setting IRQ affinity for network controllers to either follow or not follow the "high priority" processes seems to do the trick.

I usually do stuff described here:

https://cs.uwaterloo.ca/~brecht/servers/apic/SMP-affinity.tx...

And sometimes following ends up better (same cache), sometimes isolating ends up being better.