CamperBob's flippant comment, however, got me thinking as to whether GPUs could be used to speed up this process, even if it were a bit.. "fuzzy." Anyone with GPU chops have any opinions on this?
Some modern GPUs support double precision floats, so accuracy would not be an issue. GPUs are practically built for this kind of computation, however there are probably two things holding them back:
1) GPU development requires specific developer experience. Making performant GPU code is an even nastier and less intuitive problem than making performant CPU code. A naive implementation would be shockingly inefficient.
2) Leasing GPU hardware in a datacenter is very rare. You'd have to do it at the office, or build your own servers and install them yourself at a colo. Lots of time and effort.
Even if the GPU solution was 10x faster (it could be much more, but it depends on how much the CPU, disk, network is a bottleneck), if you're talking about reducing $50k in computer time to $5k in computer time, it's almost certainly not worth it. If you're talking about $2m to $200k, that's a completely different matter.
Today relying on a GPU solution translates to vendor lock-in, it makes you much more dependent on a particular set of hardware and even a particular OS/driver stack.
I have had a small amount of success implementing GPGPU programs using webGL. If you code your solution to webGL you are able to offload OS/driver stack concerns to the web browser implementers.
Here are two examples I've done. The first is matrix multiplication on the GPU. The second is a cellular automaton like simulation (the falling sand game popular a few years ago). Each incremental game state is calculated in a shader program and then used as input for the next increment.
Below I've linked github urls and two blog posts that were done about them. Unfortunately right now webGL is kind of a moving target and I doubt these examples still work out of the box. However, if you are interested in GPGPU on webGL I would encourage you to get involved as I think there are some tweaks to the standard that would make GPGPU life easier.
That's really not true - we've been doing cloud GPU computing for a few years (all in OpenGL shaders and OpenCL) - we run in Windows, Linux, MacOS and on both NVIDIA and ATI.
We use GPUs for all of our computer vision and image processing, 10X is definitely a low-ball for what they're doing here - it would be much faster.
The important bit is at the end where they say "10% of the time is spent on polygon intersection" and the rest on image processing. We do GPU image processing on the cloud and this is (likely) where Scribd could likely obtain a significant speedup - many of our image processing operations run hundreds of times faster than on the CPU.
1) GPU development requires specific developer experience. Making performant GPU code is an even nastier and less intuitive problem than making performant CPU code. A naive implementation would be shockingly inefficient.
2) Leasing GPU hardware in a datacenter is very rare. You'd have to do it at the office, or build your own servers and install them yourself at a colo. Lots of time and effort.
Even if the GPU solution was 10x faster (it could be much more, but it depends on how much the CPU, disk, network is a bottleneck), if you're talking about reducing $50k in computer time to $5k in computer time, it's almost certainly not worth it. If you're talking about $2m to $200k, that's a completely different matter.