[jandrewrogers]

Depends on the workload and the database. Many workloads on modern hardware are more often limited by memory and network bandwidth than I/O per se. (This partly depends on database kernel architecture and presumes a modern design. Some database engines, particularly older designs, may have other bottlenecks.) Also, for scale-out databases you can compile the query once and ship the binary or IR to every relevant node, which amortizes the compilation cost.

For classic OLTP workloads, JIT compilation doesn't carry much benefit because you are (typically) directly targeting a few records. The primary benefit is for constraint matching in page scan operators. I would say the sweet spot is for low-latency operational analytics i.e. queries in mixed workload environments; the average dwell time of a query on a particular page has a big impact. These workloads (basically "real-time analytics") are increasingly popular.

Before I had experience designing systems with JIT compiled queries I worried quite a bit about overhead. The people with expertise that assured me the overhead would be pretty small if implemented well turned out to be correct, which allows you to JIT a lot of operations for material benefit.