[hedora]

I’m surprised (unless they replaced the core tcmalloc algorithm but kept the name).

tcmalloc (thread caching malloc) assumes memory allocations have good thread locality. This is often a double win (less false sharing of cache lines, and most allocations hit thread-local data structures in the allocator).

Multithreaded async systems destroy that locality, so it constantly has to run through the exception case: A allocated a buffer, went async, the request wakes up on thread B, which frees the buffer, and has to synchronize with A to give it back.

Are you using async rust, or sync rust?

[skavi]

modern tcmalloc uses per CPU caches via rseq [0]. We use async rust with multithreaded tokio executors (sometimes multiple in the same application). so relatively high thread counts.

[0]: https://github.com/google/tcmalloc/blob/master/docs/design.m...

[usrnm]

How do you control which CPU your task resumes on? If you don't then it's still the same problem described above, no?

[skavi]

on the OS scheduler side, i'd imagine there's some stickiness that keeps tasks from jumping wildly between cores. like i'd expect migration to be modelled as a non zero cost. complete speculation though.

tokio scheduler side, the executor is thread per core and work stealing of in progress tasks shouldn't be happening too much.

for all thread pool threads or threads unaffiliated with the executor, see earlier speculation on OS scheduler behavior.

[packetlost]

Correct. The Linux scheduler has been NUMA aware + sticky for awhile (which is more or less what this reduces to in common scenarios).

[jhalstead]

> I’m surprised (unless they replaced the core tcmalloc algorithm but kept the name).

Indeed, it's not the old gperftools version.

Blog: https://abseil.io/blog/20200212-tcmalloc

History / Diffs: https://google.github.io/tcmalloc/gperftools.html

[skavi]

also:

1. tcmalloc is actually the only allocator I tested which was not using thread local caches. even glibc malloc has tcache.

2. async executors typically shouldn’t have tasks jumping willy nilly between threads. i see the issue u describe more often with the use of thread pools (like rayon or tokio’s spawn_blocking). i’d argue that the use of thread pools isn’t necessarily an inherent feature of async executors. certainly tokio relies on its threadpool for fs operations, but io-uring (for example) makes that mostly unnecessary.