Hacker News new | ask | show | jobs
by bminor13 977 days ago
Does anyone happen to have expertise/pointers on how ZFS' ARC interacts with Linux disk caching currently when using ZFS-on-Linux? It seems like the ARC space shows up as "used" despite being in a similar category of "made available if needed" - is that correct?

Is data in the ARC double-cached by Linux's disk caching mentioned in the post? If so, is it possible to disable this double-caching somehow?
2 comments
MrDrMcCoy 977 days ago
ZFS ARC unfortunately does not integrate with the kernel file cache, so they step on each other a lot. ZFS does watch available system RAM and try to dynamically reduce its usage as memory pressure increases, but I've found its responsiveness for this to be far too slow. This combined with how ARC appears to just be an opaque block of RAM that cannot be reclaimed, I usually just set a hard limit on how big the ARC is allowed to get in the module load arguments and be done with it (at least for systems that are doing more than just storage).
link
drewg123 977 days ago
Is ARC really non-reclaimable on Linux?

At least on FreeBSD, there is a kmem_cache_reap() that is called from the core kernel VM system's low memory handlers.

Looking at the linux code in openzfs, it looks like there is an "spl_kmem_cache_reap_now()" function. Maybe the problem is the kernel dev's anti-ZFS stance, and it can't be hooked into the right place (eg, the kernel's VM low memory handling code)?

link
MrDrMcCoy 977 days ago
It's reclaimable, but opaque. The ARC just looks like used RAM rather than file cache, which throws off various means of accounting.
link
nubinetwork 977 days ago
echo 3 > /proc/sys/vm/drop_caches

(Bear in mind that 3 is the most aggressive but other than exporting the pool, it's the only way to dump the cache, especially if you boot off ZFS)

link
nubinetwork 977 days ago
ARC is completely separate from FS caches... if the kernel needs memory, it will tell ZFS to prune the ARC, however it's not exactly instantaneous.

Newer versions of htop also now have counters for ARC usage (compressed or uncompressed)... but it still shows up as used rather than cache.

link