[lukego]

Author here with a little update from experience.

Generally I am quite happy with this approach, but I am not wild about the cost/benefit of writing tools in Smalltalk. I've spent a lot of energy on treading water and trying to keep things working while the environment is being rewritten underneath my feet.

I have also found that Smalltalk works best when data is imported into the Smalltalk heap, and represented by a dedicated model classes, and also represented by dedicated view objects (Bloc.) That's a damned lot of classes to write and the whole image can grind to a halt if I reasonably want to inspect millions of objects. This makes me miss simpler and more scalable data structures like R data frames.

Going forward I plan to take a more perl-inspired "there's more than one way to do it" approach and complement the Smalltalk code with e.g. R, Lua, etc. I expect to spend more time writing code generators (a la SWIG) rather than hand-crafting everything in Pharo.

[patrec]

I really hope RaptorJIT or one of the other luajit forks gains some traction, because luajit is such an impressive feat of engineering and so massively superior to more mainstream dynamic language implementations (ffi, startup time, execution speed, code size...) that it would be tragic if it just slowly fades into obsolescence.

I also think your focus on making performance more predictable and reducing the mental burden on devolopers who want to write reliably fast code is exactly the right one; even a version of luajit that's half as fast on average but more predictable could be more useful in practice to most people.

Having said that, to keep some already precarious and fringe technology alive by adding as core part of the toolchain an app written in an even more fringe and decidedly more fiddly[*]/unstable language like Pharo, and one which has basically zero community overlap with luajit users seems... bold.

As does moving forward by adding R/codegen targetting pharo to the mix.

[lukego]

That's a fair point.

I'm viewing the tooling as a separate project. RaptorJIT is one application that it supports and Pharo is one UI.

Codegen is appealing for being able to use the same data for more things e.g. also generate gdb/rr extensions for debugging the RaptorJIT VM, or generate Lua code for a simple console-based analysis tool, etc. There are already other people writing LuaJIT tooling in Python and Javascript and ... so I want to keep an open mind about target environments for the sake of potential collaboration.

(You could also think of codegen as a bridging strategy in case the tooling should be ported to some more conventional environment(s) once it is mature e.g. CLI, web, emacs, etc.)

EDIT: Should also note that the Studio tooling can be run via Docker in pure text mode, converting the binary diagnostic data into ascii that's similar to but richer than the classic LuaJIT '-jdump'. So users don't need to use the GUI, or setup any of the dependencies, or even learn different ways of working.

[patrec]

> I'm viewing the tooling as a separate project

If the quality (and acceptance) of the tooling is pretty important to the success of RaptorJIT and the tooling is unlikely to be used for anything else than RaptorJIT (e.g. pypy or at least another luajit fork), they are not really _that_ separate.

I don't think codegen is in itself bad, but R -> Pharo so you can work on some low level lua JIT stuff – you're mightily reducing the set of people who don't need to learn extra tech to be able to contribute (from an already small pool of people who have the relevant skills and inclination to improve a JIT compiler).

[lukego]

> the tooling is unlikely to be used for anything else than RaptorJIT

Here we are talking at cross-purposes. I view the tooling as an extensible framework like Emacs or Eclipse. I am talking about my intentions for extending the diagnostic tooling to support other projects beyond RaptorJIT.

Over time I would anticipate reducing the number of moving parts in the RaptorJIT tooling.

The value of a "there's more than one way to do it" approach is to make it easier to support new projects besides RaptorJIT using whatever tools that project's community are comfortable with (which, as you say, probably won't be Pharo but might be R/python/perl/emacs/etc.)

[baruchthescribe]

>that it would be tragic if it just slowly fades into obsolescence.

Not going to happen. Too many people have sworn to keep improving it.

[lukego]

For the LuaJIT family to die all that's necessary is for people to stop selecting it for new projects. Time takes care of the rest.

[baruchthescribe]

Sure. But when my fork with 10% more speed is released, some fairly large companies are going to choose it and it will continue.

[lukego]

Sure. But if your plan is to do all the work by yourself, for the benefit of companies who aren't contributing much beyond occasional bug fixes, then you will eventually burn out like Mike Pall did. There won't necessarily be another sucker to take your place.

For the project to have a healthy future you really need a community of user-developers who are supported by multiple organizations and cooperating and coordinating with each other. LuaJIT doesn't currently have such a community and bootstrapping one is a doosey.

[baruchthescribe]

You're right and I think more than anything, it's a function of the license. New code needs to be LGPL.

[ebg13]

In your opinion, should LuaJIT 2.1 users switch to RaptorJIT? I ask because the LuaJIT stewardship plan never materialized and it's been...too long.

[lukego]

Switching to RaptorJIT would get you a community that's more open to contribution, tracing and profiling with zero overhead that can be left enabled in production, and - on the downside - fewer supported platforms/architectures in the short term.