[brendano]

The programming language analysis is pretty interesting, but you have to ask, what's the point of a brand-new Java implementation? R isn't just a programming language, but it's a software framework/ecosystem. They mentioned this in the slides, but it's problematic because R crucially relies on C and Fortran interaction (which I thought the JVM can't do efficiently, since it doesn't like giving C/Fortran raw memory access to its internals). Decades of work has gone into highly optimized Fortran linear algebra libraries, for example -- which R and all the other high-level numerical languages (NumPy/SciPy, Matlab, Julia) use. And many of the CRAN packages (the availability of which are a major reason anyone uses R in the first place) are partly or mostly C/Fortran code.

There are many other R implementation efforts going on right now -- Radford Neal lists a few (as well as his own) here: http://radfordneal.wordpress.com/2013/07/24/deferred-evaluat...

The presentation focuses on the R programming language, which they nicely show has all sorts of misfeatures that impede rapid execution. If you're going to not try to have compatibility with R and CRAN, you might as well start from scratch with design and performance in mind, as in Julia: http://julialang.org/

[pron]

> ... C and Fortran interaction (which I thought the JVM can't do efficiently, since it doesn't like giving C/Fortran raw memory access to its internals).

As of 2002 (JDK 1.4) Java has excellent integration with native memory (you can freely pass pointers from C/FORTRAN to Java and vice versa[1]).

There are numerous Java math libraries that use BLAS/LAPACK already[2]. In fact, AFAIK, most Java matrix math libraries use FORTRAN code (at least as an option).

[1]: Java side: http://docs.oracle.com/javase/7/docs/api/java/nio/ByteBuffer... C side: http://docs.oracle.com/javase/7/docs/technotes/guides/jni/sp...

[2]: For example, https://github.com/fommil/matrix-toolkits-java, http://mikiobraun.github.io/jblas/

[beagle3]

Is there any way to ensure that accessing ByteBuffers (or whatever) is fast?

I had used memory mapped buffers (which should be equivalent in performance), and there was no way to make the JIT inline access to these arrays. It was all calls (indirect, not properly branch predicted at that). Equivalent code to C++, running 10 times slower, with no way to speed it up.

(And the reason I was using memory maps, if you insist -is a 2GB read-only dataset used by multiple processes at the same time - I went to C++ eventually, because there was no way to get reasonable performance from Java, either memory use or speed. This is circa 2010)

[pron]

Yes. High performance Java code sometimes makes use of JVM intrinsics, accessible through the sun.misc.Unsafe class. Those are JITted down to a simple memory access instruction. That class also has intrinsics for CAS, and in JDK 8 it's got intrinsics for different memory fences as well. Those calls are compiled to a single native instruction.

For example, the Java Chronicle library[1] uses these techniques, as well as memory mapped files, to implement fast persistent message queues.

[1]: https://github.com/OpenHFT/Java-Chronicle

[beagle3]

Cool. Was this in Java 6? (circa 2010 - I couldn't find a way to do it back then). Also, why would you need (even on JDK7 or JDK8) unsafe access to jit a memory mapped access inline? Is there an underlying philosophical reason, or is it just that they never got to do it?

[pron]

Yes, it was in Java 6 (except for direct access to fences, which has been added in Java 8).

The sun.misc.Unsafe class is used extensively by JDK classes, and is meant for internal use. It provides intrinsics that are translated to a single machine instruction. Normally, you don't use the class directly. For example, you use, say, AtomicInt for CAS operations (which, internally uses s.m.Unsaafe) or the ByteBuffer class (which internally uses s.m.Unsafe for direct pointer access). The JDK classes add all sorts of protection (like range checks) around s.m.Unsafe, but if you know what you're doing, using s.m.Unsafe directly and eschewing some of those protections (usually adding ones more pertinent to your domain), you get some performance gains which may be significant depending on your use case.

[brendano]

Ah, wonderful! So if I understand this correctly, this doesn't give C/Fortran access to Java-native primitive arrays; but instead, it's specific to NIO byte buffers (and then the matrix libraries have to build on top of that). But that should be fine for doing R replacements, at least in theory.

(Personally, when programming Java I find it more convenient to use primitive arrays as opposed to matrix libraries, but that might be dependent on the operations I tend to do: lots of increment/decrements and only occasional linear algebra. I guess this isn't exactly relevant to the R replacement question.)

[pron]

Java also gives native libraries direct access to primitive arrays, but that requires pinning them in place for the duration of the call (i.e. not letting the GC move them) so it incurs some performance penalty.

Direct byte buffers are very common in high-performance Java code. Reading/writing from/to those buffers can be made just as fast as plain Java arrays.

[brendano]

Thanks for the information. Sorry I was misinformed before...

[bedatadriven]

I think compatibility with GNU R and CRAN is a pretty reasonable goal - even if it's not the focus of this particular Oracle research project.

We've been working hard and now systematically to get Renjin (also R on the JVM) to run CRAN packages: http://packages.renjin.org. Renjin also compiles C and Fortran code to JVM bytecode, though there is still some work to do there as well.

Regarding the hand-tuned matrix math libraries, there's nothing to stop you from using them with Renjin - you can drop in MKL or Atlas as desired, or fall back to pure-Java versions in a pinch.

[pjmlp]

Julia is quite cool, actually.

As for Java, Oracle together with AMD, are in the process of making the GPU trasparent to Java developers as part of the Sumatra project.

So this is one are where R could benefit of running on Oracle's JVM. It remains to be seen if other Java vendors would adopt such feature.