Improvements to our LuaJIT benchmarking are currently being discussed in https://github.com/JuliaLang/julia/issues/14222. LuaJIT 2.1 is significantly faster than LuaJIT 2.0, but until now I haven't had a convenient way to get an installation of LuaJIT 2.1 on our test machine. LuaSci was easy to install and we would be amenable to switching over to LuaSci instead of vanilla gsl-shell. See https://github.com/JuliaLang/julia/issues/14222#issuecomment... for a timing comparison.
I never looked at SciLua till just now. I have always liked Lua and have made several scripts for my desktop and servers. Just never thought of Lua for Data Science. My main issue is I haven't hit a data set to big for R in my own use cases.
Julia in contrast to most languages can run very fast or very slow for roughly the same program because of type information and garbage collection.
I would be willing to bet there is a lot of fat to be trimmed if you can't get close to LuaJIT unless SIMD is involved. Have you tried profiling your Julia code to see where the problem is?
I think it would be really interesting to understand exactly why Julia underperforms LuaJIT on some benchmarks. I'm pretty skeptical that Julia loses to LuaJIT uniformly; I suspect it depends on the benchmark taking advantage of LuaJIT's superior discovery of run-time type information.