[Eridrus]

I was thinking the same thing, but is SSD to GPU faster than RAM to GPU? In many (not all) cases you buy a tonne of RAM and load your entire dataset into memory once and then iterate over it as necessary.

You also lose the flexibility of doing any sort of data modification or augmentation. One domain where your data usually doesn't fit in RAM is image recognition, but often you want to do things like apply random flips, crops and change hues before training to make the neural net less sensitive to those changes, which you can't really do with this.

[foobar2020]

SSD is probably not as fast as RAM, but it's much much cheaper, in the order of 10x per gigabyte. With SSD-GPU bridge you can have fast access to a multiple TiB training set, on a single machine.

Data pre-processing is indeed an issue, but hue adjustment/flipping/cropping could be implemented as Tensorflow operations, on the GPU. Similarly with input decompression - it would either have to be done on GPU, or the data would have to be stored uncompressed.

[emn13]

As long as the average bandwidth isn't a bottleneck, it's not going to matter - at worst, you're just going to need to prefetch (and due to SSD latency, that's likely optimal regardless).

[kaigai]

RAM-to-GPU is always faster than SSD-to-GPU. It is a solution to help a situation when data size does not fit RAM size (or when user has less budget to purchase enough RAM. In fact, we can purchase Intel SSD 750 (400GB) with 300USD).

[Eridrus]

For the scenario you're targeting: databases, this makes a tonne of sense, database data regularly exceeds the size of RAM and the operations you want to do on the data are pretty static in the sense that they're the SQL operators.

In deep learning you are usually doing a lot more custom processing and your datasets are usually not as big, such that just buying more RAM is often cost effective.