[stuntprogrammer]

Current publicly announced AVX512 does not support fp16. Skylake Server (SKX) and Knights Landing (KNL) are at a disadvantage here. They've not publicly said anything about extensions in Knights Hill (the long announced successor to KNL).

That said, Intel have announced the emergency "Knights Mill" processor jammed into the roundmap between KNL and Knights Hill. It's specifically targeted at deep learning workloads and one might expect FP16 support. They had a bullet point suggesting 'variable' precision too. I would guess that means Williamson style variable fixed point. (I also guess that the Nervena "flexpoint" is a trademarked variant of it).

I assume the FPGA inference card supports fp16. And Lake Crest (the first Nervena chip sampling next year) will support flex point of course. I would expect subsequent Xeon / Lake Crest successor integrations to do the same.

Fun times..

Aside on the compiler work -- I think it's not that hard to emit this instruction at least for GEMM style kernels where it's relatively obvious.

[paulsutter]

Yes a compiler can generate the instruction. But if it's alone in a for loop surrounded by random STL classes - which even if inlined - are bodging up the pipeline or (gasp) causing spurious random dram accesses, there's little performance gain. And that's what usually happens in c++ code that wasn't already designed for AVX ("it's using AVX, but it's not running any faster. i guess AVX doesn't make much difference").

Net-net, data and code need to be structured for AVX to achieve the potential performance gains, and that's 80% of the work.

Once you structure the data and code for AVX, yes you can use regular C statements, then experiment with optimization flags until the compiler generates the intended instructions (and hasn't introduced excessive register spills). But its hard to see how that's any easier than using the intrinsics.

[stuntprogrammer]

The problem is less the spurious DRAM accesses etc, as awful as they would be. The compiler problem is really a mix of 1) understanding enough about fixed-bound unit-stride loops to nonoverlapping memory (or transforming access to such) and 2) data layouts that prevent that. E.g. while there are well understood data layouts at each point of the compilation pipeline, it's hard in general for compilers to profitably shift from array of structs to struct of array layouts.

You are correct that, generally speaking, most STL heavy code would be hard to vectorize and unlikely to gain much advantage. (Plus there are the valarray misadventures). You will sometimes see clang and gcc vectorize std::vector if the code is simple enough, and they can assume strict aliasing. Intel's compiler has historically been less aggressive about assuming strict aliasing.

Various proposals are working through the standard committee to add explicit support for SIMD programming. E.g. if something like http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n418... were to be standardized we could write matrix multiply explicitly as:

  using SomeVec = Vector<T>
  for (size_t i=0; i<n; ++i) {
    for (size_t j=0; k<n; j+=SomeVec::size()) {
      SomeVec c_ij = A[i][0] * SomeVec(&B[0],j, Aligned);
      for (size_t k = 1; k < n; ++k) {
        c_ij += A[i][k] * SomeVec(&N[k][j], Aligned);
      }
      c_ij.store(&C[i][j], Aligned);
    }
  }

For my own work on vector languages and compilers I've had an easier time of it since they have been designed to enable simpler SIMD code generation.