[grashalm]

Pkl was built using the GraalVM Truffle framework. So it supports runtime compilation using Futamura Projections. We have been working with Apple on this for a while, and I am quite happy that we can finally read the sources!

https://github.com/oracle/graal/tree/master/truffle

Disclaimer: graalvm dev here.

Edit: typo

[krajzeg]

> ...GraalVM Truffle framework... Futurama Projections...

I know it's partly on me for not knowing the domain, but I honestly suspected somebody is trying to make fun of me with some concentrated technobabble.

Especially since I wouldn't expect the topic (configuration languages) to require complex mathematical machinery to work with. Now I have something interesting to dig into.

[TimTheTinker]

What has most impressed me about GraalVM and Truffle is their capability of deep-optimizing high-level code like Python and Ruby.

I once saw a demo where someone took a simple operation in Ruby using inefficient-but-elegant syntax (including creating and sorting an array, where a simple loop would have been the appropriate approach in C). He compiled that using TruffleRuby and the entire array construction and sorting was completely optimized out of the generated bytecode.

[Alifatisk]

Really? Link?

[grashalm]

He probably means one of the wonderfully crafted talks by Chris Seaton.

Here is one of the many: https://youtu.be/bf5pQVgux3c?si=S8Dm5d_GXYXgJtnY

If you go looking for more you will find many more marbles.

[grashalm]

Shameless self plug: Giving an introduction in this video: https://youtu.be/pksRrON5XfU?si=CmutoA5Fcwa287Yl

[refulgentis]

Gently teasing: linking a 2 hour video with "shameless self plug" definitely did _not_ help obviate the surreality.

[hoherd]

I'm not sure if it was part of the humor, so pardon me if it was, but it's actually "Futamura" as in Yoshihiko Futamura, not "Futurama".

https://en.wikipedia.org/wiki/Partial_evaluation#Futamura_pr...

[Xiol32]

Really glad it wasn't just me. Genuinely thought someone was trying to make a joke.

[avisser]

Same - it doesn't help that I read Futamura as Futurama the first 3 times.

[avianlyric]

Probably because the original comment said “Futurama” not “Futamura” due to autocorrect [0], and was later edited to correct the misspelling.

Even now the OG comment says “Fuamura” but the quote in the GP comment has the original “Futurama” written in it.

[0] https://news.ycombinator.com/item?id=39239965

[felideon]

For me it was about 5, until I read your comment. :/

[tomcam]

Same. There was a mini subthread years ago that applies.

https://news.ycombinator.com/item?id=13752964

[EPWN3D]

Glad I'm not the only one who had this reaction. I just can't bring myself to accept that a problem that could be solved with a slightly better version of JSON or property lists requires this many buzzwords.

[duped]

Those aren't "buzzwords" though, it's a very specific way to implement programming languages. It's not really meaningful except for the PL implementation nerds.

Especially the Futamura projections. It's almost magic and very few people have even heard of them.

[grashalm]

Very few people have heard of them. That is exactly the reason why I mention them as often as I can. They are a great entry into the world of meta compilers.

[usrusr]

If Futamurma means what I think it means skimming across the Wikipedia entry, it would mean that simple value-holder-file configurations would be parsed and checked at the speed of a general purpose tokenizer. But without closing the door to what the language can express in more elaborate configuration file "landscapes". Best of both worlds and presumably all without requiring anybody but the toolmakers to understand what the buzzwords really mean.

[paufernandez]

The best video I know about this stuff is "Compilers for free" by Tom Stuart (https://youtu.be/n_k6O50Nd-4). It is hilarious at one point. Brilliant.

[dbu21]

Fantastic talk! Thanks for sharing.

[junon]

Genuinely read "Futurama Projections" and figured the same. This doesn't sound real (though I fully trust it is, just sounds funny).

[konstante]

>...suspected somebody is trying to make fun of me...

I think that too, "Futamura projections" are important but they are very very far from "complex mathematical machinery" as you may hear it. They are indeed very simple (even mathematically trivial) and require no special background to understand.

[leonheld]

> but I honestly suspected somebody is trying to make fun of me with some concentrated technobabble

Let me tell about a revolutionary device called a Turbo encabulator.

[ughitsaaron]

sounds like a perfectly cromulent topic to embiggen our knowledge.

[egberts1]

Perfectumentous!

[ChrisMarshallNY]

An author named David Duncan wrote a series of books, called A Man of His Word (and A Handful of Men)[0]. Great books.

One of the races in the books was the Anthropophagi (basically modeled on New Guinea headhunters). They talked like that.

[0] https://en.wikipedia.org/wiki/Dave_Duncan_(writer)

[yurishimo]

You joke, but this is surprisingly close to the name given to Dumbledore in the Dutch translation of Harry Potter.

[fuzztester]

supercali ...

[ativzzz]

You joke but newer rails versions come with a front end framework named Turbo, and there's also a JS bundler named Turbo, so this is actually too close to reality

[Cthulhu_]

It makes me think of this game, basically "pokemon or technobabble". Can't find it now though.

[girvo]

There's Pokémon or Big Data: http://pixelastic.github.io/pokemonorbigdata/

And the (original, I think), Pokémon or Tech Term: https://docs.google.com/forms/d/e/1FAIpQLSfsG7AEFLvlW68aIVIs...

[pushedx]

> Futamura

not Futurama :D

[stockholm]

This comment is what PKL is going to be remembered for. Tbh I wouldn’t even have the courage to write the comment myself as the framework was coming from Apple.

[jgrahamc]

> Pkl was built using the GraalVM Truffle framework. So it supports runtime compilation using Futamura Projections.

What now?

[pdpi]

As I understand it:

GraalVM is an alternate JDK that can, among other things, do ahead-of-time compilation for Java.

Truffle is a framework for building languages, that sits on top of Graal.

Futamura Projections are a particularly interesting use case for compile-time partial evaluation.

With partial evaluation you have static (known at compile time) arguments to your function, and dynamic (known at runtime) arguments.

When “your function” is an interpreter, and your static arguments are source code, the output is effectively a self-contained compiled binary.

When your function is an interpreter, and the static arguments are the source code for the interpreter itself, that “self-contained compiled binary” is now itself a compiler.

[iainmerrick]

That all sounds cool, but is any of that especially useful for a configuration language?

[usrusr]

If you want a tool to be able to generate executable validation from a schema, a compiler framework should come in handy.

It seems like they did not aim to make yet another mvp configuration language, but something that can scale across a wide range of usage scenarios spanning all the way from short-lived processes reading a number from a file to huge libraries of default/override hierarchies. Lack of universality sets an upper bound for the value of a configuration language, particularly when seen through the lens of configuring heterogeneous tech stacks.

[SOLAR_FIELDS]

I’m also curious, because Graal is pretty exciting stuff, what this might give over Jsonnet or Cuelang. It’s already a hard enough sell to try to get people to adopt these and they are much older and more robust than Pkl.

[iainmerrick]

I'm very wary of anything Java-based, having been burned by Java tooling in the past. I work on a few different Android projects and I have to switch between three different JDK versions depending on which I'm working on. What happened to "write once, run anywhere"??

I really like Pkl's comparison page, which includes its weak points as well! https://pkl-lang.org/main/current/introduction/comparison.ht...

Pkl’s native binaries are larger than those of other config languages.

It should be as fast and easy to use and reliable as something like esbuild, so I'd suggest they may want to rewrite it in Go like esbuild. I'm not a Go fan at all, but it clearly does some things really well.

[SureshG]

>"write once, run anywhere"??

You know that the code compiled using future version of java won' work in older versions..rt? I would like to know if any other programming language does that kind of thing.

> t should be as fast and easy to use and

How did you conclude that it's not fast? They are creating native binaries just like Go or any other AOT languages with GCs. Graal native images are as fast or faster than Go. Also it contains a REPL, that's why bigger size. So for CLI tooling as a developer using pkl, you won't see any difference if it's written in java + kotlin or golang.

[spullara]

There are reasons Oracle sued Google over Android and you just articulated one of them.

[bioballer]

Pkl is newly open sourced, but it not new. It's been used for years at Apple, and has been battle tested internally.

I'd actually say that our tooling in some ways is more mature. For example, I think our IDE experience (at least in JetBrains editors) is the best out there.

[Perepiska]

There is no trust in the words “we tested this internally.”

Apple employees can rate it as an old project among themselves, while it is more convenient for everyone else to rate the product from the moment of publication.

[saynay]

Looks like a more robust type system than Jsonnet (but less than Cue), with some amount of flow-control that Cue doesn't seem to support. I am not very familiar with Cue though.

[matheusmoreira]

> With partial evaluation you have static (known at compile time) arguments to your function, and dynamic (known at runtime) arguments.

That's pretty clever... How is this implemented in actual code though? I can't even begin to imagine how that magic machinery works.

[EPWN3D]

> Truffle is a framework for building languages, that sits on top of Graal.

wtf is Graal? That sounds like a supporting character from Beowulf.

[sgammon]

https://graalvm.org

Polyglot and native compilation enabled runtime for JVM, can run Js, Python, Ruby and more.

[verticalscaler]

On tonights episode of Futurama bender and the gang explore the temple of Pkl on planet VM where truffles are considered the holy graals and barely run away in time from - The Compilations - an ancient secretive order of silver voiced kong-fu monks tasked with protecting the shrine from alien invaders as has been foretold in prophecies - and strangely reminiscent of 20th century Earth doo-wop group The Drifters.

Cue chase montage shenanigans with Under The Boardwalk playing in the background

Do you smell toast.

[giancarlostoro]

I definitely did a double take to make sure they didn’t write Futurama.

[maxfurman]

I absolutely thought they wrote Futurama until I saw this comment

[macintux]

They did (autocorrect) and later fixed it.

[verticalscaler]

The mind. It is a curios thing.

[pasta1212]

holy graals

[txdv]

A LOT of projects in the Java world do add new features to java. My favorite is CraC

[catchnear4321]

new game: llm hallucination, attempt at humor, or legitimate technical explanation.

[usrusr]

Too close to the "reliably solvable by simple heuristic" end of the spectrum to be a good game: if the text is short it's probably a joke, if it is a very long wall of words it's LLM and anything of somewhat reasonable length can only be a legitimate technical explanation, no matter how unlikely.

[catchnear4321]

> and anything of somewhat reasonable length can only be a legitimate technical explanation, no matter how unlikely.

thanks, that will help improve the output.

[cyanydeez]

all I saw was oracle

[SOLAR_FIELDS]

FWIW Graal is probably one of the most exciting technologies to come out of Oracle in a long time.

[chris_wot]

It came out of Oracle. Kids of death.

[sgammon]

It actually didn’t, it came out of academia. Oracle just did the right thing one time.

[tadfisher]

It's GPL-licensed, and it works. I'm happy they haven't Oracle-ized the JVM, and have been investing into great features that are available to everyone for free.

[fmajid]

Agreed. Anything by Oracle is an automatic hard nope.

[lpage]

Your mention of Futamura Projections was a nice reminder of how very mathy/theoretical foundations underpin Nice Things in high-level languages, like Hindley–Milner inspired type systems and modern optimizing compilers targeting SSA form. Value Lattices in Cue [1], another config programmable configuration language, also fall into this bucket.

[1]: https://cuelang.org/docs/concepts/logic/

[corysama]

Currently using Cue in a major project. It can be a puzzle. But, we like it a lot. Wish it had a bigger community.

[SirGiggles]

Not completely related to the OP, but is Truffle going to be upstreamed as part of Project Galahad or will it remain as a library maintained by Oracle Labs?

I ask cause the Project Galahad page on openjdk.org is a bit sparse on details.

[grashalm]

The truffle compiler extensions in Graal will be part of Galahad. For example the partial evaluator. The truffle framework and all the languages are consumed from a maven repo as a regular java library these days.

Some background on the recent changes there: https://medium.com/graalvm/truffle-unchained-13887b77b62c

[summerlight]

It'd be interesting to understand what kind of performance problem Apple had and tried to solve with GraalVM/Truffle. I've seen some instances of heavy configs that generate literally several gigabytes of data, but those were usually not significant bottlenecks since configs are not updated very frequently.

Of course, I know those two frameworks are one of the engineering marble of the age and would understand even if they decided to go without any concrete needs for it.

[simonfxr]

I guess you mean Futamura projections?

[lionkor]

I wish people named more tech products after popular media instead of common words. Would make it equally hard to web search, but at least it would be funny for non-techies to listen to

[nindalf]

You'll really like the Quantum Resistant key exchange algorithm - Kyber. A related project is Cryptographic Suite for Algebraic Lattices or CRYSTALS.

Sadly they renamed Kyber to MLKEM.

[MarkSweep]

There is also Dilithium:

https://pq-crystals.org/

[grashalm]

Damn you autocomplete! This happens all the time :D

[aardvark179]

Oh wow, this wasn’t the sort of language I expected to see being built on Truffle, but I’ll be really interested to take a closer look when I’m on a decent net connection.

[Hixon10]

Do you know, why they use both ANTLR and Truffle?

[grashalm]

Truffle has no opinion on how you parse the sources. It cares about how you execute them from an intermediate Truffle guided representation produced by the parser.

In other words antlr and truffle are a great fit. We even use this pairing for our example language simplelanguage.

https://github.com/graalvm/simplelanguage

[Hixon10]

Thanks! I haven't seen before usage of Truffle and ANTLR together, but it makes sense.

[ithkuil]

Futamura

[beanjuiceII]

That's iron chef futamura to you