[BrandonS113]

I used to think so, but I have a function that gets called about a billion times each and every day as new data comes in, and and takes about 0.01 seconds to evaluate (optimizaiton with nlopt). I tried to code it in c (30% speed improvement) python (twice as slow), Julia (about the same speed). Reason is that call has 5 parameters that operate on a vector of length 50 to return a value to minimize. Turns out R is pretty good at such vector calculations.

[n4r9]

Is this what you mean by nlopt? https://github.com/stevengj/nlopt

If so, it looks like you're interfacing from R to high-performing code written in C. Isn't that exactly what OP was describing?

[BrandonS113]

no, the function it calls is pure R and that is where the the code spends all its time.

[n4r9]

Interesting. Have you published the code and/or benchmarks anywhere? This flies against everything I've read about Julia and R.

[BrandonS113]

No, its 4 lines of code. I just benchmarked for myself. All it is for 2 vectors x and y of average length 50, and 5 parameters, with exponential, addition, and multiplication, and ultimately sum to return to the optimizer. I was also surprised as I expected c to be much faster. And with Rccp, its actually slower than the R, overhead I guess. When I looked into it, apparently R has really fast code for such vector calculations. With julia, admittingly I did not use simd which would likely make it faster.

Now, I generally use Julia for heavy computes, and usually its much faster than R. But not always.

And this little bit of code runs for hours on the largest instance on AWS every day. Why I was looking so speed it up.

[Nilshg]

If possible I would encourage you to make an MWE and post it on the Julia Discourse - "Julia slower than R for optimization problem" or something like that, there's a good chance that the community will be able to eke out some more performance. Alternatively you might have hit on a case in which Julia itself is currently leaving performance on the table, which would still be helpful for the community to know as being slow is often considered a bug in Julia world.

[BrandonS113]

Great idea, I do lurk there, just might do that next time I look at the code. My hunch is that simd is the low hanging fruit julia brings. But, and I am not an expert, both might end up doing BLAS anyways, which is why they are so similar.

[wodenokoto]

I am willing to concede, but also willing to argue that vector and data frame manipulations in R are calls to optimized code.

Like, R is “what if we made a lisp inspired version of Python built around numpy and pandas and then reversed timed”

[BrandonS113]

I think that is exactly what is happening. Most of my code is much much faster in Julia, and the code is nicer. But R has its moments. Which is good since this particular app has 3K lines, and I do not want to port it to Julia.

And data.tables in R is faster (and I think nicer to write) than DataFrames in Julia. And since data.tables feed my optimization, R still wins.