[pansa2]

> I really wish the Lua authors would add official types to Lua.

Never going to happen IMO. Adding static types would change the nature of the language completely, even more than it has in Python.

As Teal shows, it would require giving up one of Lua's core features: tables as the language's single data structure. It would significantly complicate a language known for its simplicity.

Even the implementation would need to change radically - adding a type checker would invalidate the current approach of using a single-pass source-to-bytecode compiler.

[dottrap]

>> I really wish the Lua authors would add official types to Lua.

> Never going to happen IMO. Adding static types would change the nature of the language completely, even more than it has in Python.

You both are kind of right.

The Lua authors have been working on the new companion language to Lua named Pallene. Pallene is a subset of Lua that adds types, not for the sake of types themselves, but for the purpose of performance. The Pallene compiler can generate optimized native code that potentially removes the need to manually write a module for Lua in C.

The other cool trick is that Pallene and Lua are completely interoperable with each other, so Pallene can be added to existing Lua projects, and you can opt to use regular Lua for the dynamic parts of your code where compilers won't be able to optimize much and strong types might be more trouble than help.

Here is a talk Roberto Ierusalimschy gave about Pallene. https://www.youtube.com/watch?v=pGF2UFG7n6Y

[90s_dev]

Docs https://github.com/pallene-lang/pallene/blob/master/doc/manu...

Looks similar to Teal.

This is relatively exciting.

Also, called it!

Disappointed that it maintains syntactic Lua compatibility. Would have been a good time for a clean slate on the shoulders of hindsight.

[andre-la]

> Looks similar to Teal.

That's because they share a common origin from Typed Lua and Titan languages:

https://teal-language.org/book/other_projects.html

[90s_dev]

Wait, Pallene just compiles to C using whatever local C compiler?

https://github.com/pallene-lang/pallene/blob/master/src/pall...

Well that's kinda disappointing. I expected something more in 2025, like directly generating asm like a lot of languages are starting to do.

And your article makes it ambiguous whether it's from the Lua authors or grad students. I assume it started out just the students and then the Lua authors joined in?

[dottrap]

One of Lua's goals has been extreme portability, and the main implementation works on anything that has a C compiler, going to the extreme of compiling cleanly on C89, C99, and even compiling as C++ (no extern "C"). Remember that Lua is popular in the embedded space too, so this is a big feature.

Pallene isn't designed to be a new native language on its own. Pallene is designed to be a companion language for Lua, specializing in a subset of performance.

But as importantly, Pallene isn't just compiling to C. Pallene is generating C code that directly manipulates the underlying Lua internals, which are in C.

The research thesis is that many bottlenecks are due to boxing and unboxing going through an FFI. Memory safety also incurs overhead. Python is an extreme example of how excruciatingly slow this can be, but even Lua incurs costs for this. A core tenant of the Pallene compiler is that it can generate C code that gets to cheat like crazy. Pallene gets to directly access Lua internals and things like arrays directly manipulate underlying C arrays deep inside, which sidesteps boxing/unboxing. The compiler can do the analysis to make sure it doesn't cheat in a way that is unsafe. Finally, the C optimizer now also has a chance to perform optimizations. And now operations such as crunching math on arrays of numbers may get much faster because now you get generated code that is more CPU friendly and may benefit more from prefetching and cache locality.

Pallene is built from the the extreme compatibility goals as Lua since it is designed to work with it. It it only depends on any C compiler and Lua itself. If you get Lua compiled, then you can get Pallene working. That means any existing project that uses Lua (5.4) could start adding Pallene modules to their project for new features or to try to improve performance in key areas. Since Pallene just outputs a Lua modules, it looks like any other Lua module implemented in C, so it won't create new portability constraints that you didn't have before. This is different than say LuaJIT, where not all platforms may allow JIT or you may be targeting a new CPU architecture that LuaJIT does not support.

Both Teal and Pallene were started by grad students of Roberto's. Since Roberto has started giving talks himself on Pallene, I'm assuming they are joining in.

[ufo]

I'm Pallene's lead maintainer. Currently the code is maintained by me, my students and other open sou rce collaborators. We collaborate with Roberto over some Pallene-related research, specially about the type system semantics, but he isn't an active committer.

[90s_dev]

That's actually really exciting in that case.

If it goes further and generates native C control flow statements when possible ("if", "for", native functions, native function calls, etc), I think it could be an omni-level language, generating basically Lua statements when dynamic and C when not, and mixing them all within the same program, entirely controlled by how much type information you give it (and how you use tables and metatables).

[lifthrasiir]

In reality, tables are used in specific fashions and not fully generally. (Partly because Lua itself even recognizes some of these fashions and provides relevant operations accordingly.) Lua tables are not really a single data structure; it is a single type that acts as multiple data structures at once.

[90s_dev]

Yeah Lua optimizes array tables for performance as long as they're arrays. That's the only optimization I'm aware of. I get why they didn't just add arrays, to keep the syntax and semantics clean, simple, and unambiguous. I just don't like it. If you take that to its extreme, you get Lisp. Natural human languages are messy and full of warts, but they work despite that, or perhaps because of that, because human life is messy and warty. "Pure" languages never can or do catch on. My favorite language right now is unironically TypeScript despite all its baggage.

[ufo]

More than just keeping it simple, it's also about reducing the API surface of the language. Lua's main design constraint is that it's meant to be embedded inside other applications, and the versatile table type helps a lot in that regard.

[tiffanyh]

> As Teal shows, [official typed Lua] would require giving up one of Lua's core features: tables as the language's single data structure.

Is that true ... you can't have typed tables without giving up tables as a data structure?

[krapp]

You can't have typed tables without giving up tables as the language's single data structure. You would have tables and typed tables which are essentially just arrays with extra steps.

[90s_dev]

Not with type erasure like TypeScript does. Then it would just be type checking hints as to how the table is used, not a different kind of table. Teal does this.