| memoize is certainly not the first use of this technique. Electric Make (http://electric-cloud.com/products/electricaccelerator) uses a custom filesystem to track file accesses for purposes of augmenting dependency information, and predates memoize by five years. It even goes beyond simply tracking accesses to using the information to correct for missing dependencies on the fly. The main problem with strace is not the printing (although that doesn't help) but with the ptrace technology underneath, which basically hits the traced process with a SIGSTOP/SIGCONT pair on every system call, as well as a context switch to the tracing process and another back to the traced process. Even a no-op ptrace-based monitor that does nothing will make individual system calls ~10x slower. In my benchmarks, the best case overall performance impact was about 5%, but some processes were as much as 560% slower. LD_PRELOAD is faster but has other deficiencies, like it's trivially easy to circumvent the tracing by wiping LD_PRELOAD from the environment before starting a new process. It can also be tricky (though not impossible) to manage implicit state, such as following an application as it first chdir's to a new location, then accesses paths like "../../include". LD_PRELOAD is also tough to get right in the face of multi-threaded applications. FUSE is interesting but so far I've found the performance to be disappointing. I think that's mostly because it bounces everything through userspace and effectively doubles the filesystem activity for (nearly) everything. For example, with a normal filesystem a read from a user process basically works like this: user process
-> read system call
-> filesystem read operation
-> return result to user process With a FUSE filesystem it's something more like this: user process
-> read system call
-> FUSE filesystem read operation
-> FUSE userspace driver
-> read system call on real filesystem
-> filesystem read operation
-> return result to FUSE userspace driver
-> relay result to FUSE filesystem
-> return result to user process There are various caches in place to make this less disastrous than it seems on the surface, but fundamentally this is the architecture. For some applications that's fine; for high-performance build tools I think it's probably a deal-breaker. This is why we wrote a custom filesystem for Electric Make, and why that's still the approach we take today, nearly 15 years later: nothing else is as robust, and nothing else comes close to the performance. _Disclaimer: I'm the architect of Electric Make_ |