[crazypython]

I don't see why people won't just take the step D and Lisp do-- allowing full use of the programming language at compile time.

You can execute an ordinary functions at compile-time to read a DSL from a string or read attributes (reflective metaprogramming) on your program's classes. Take the string it outputs, use mixin(), and you have code. For example:

    // Sort a constant declaration at Compile-Time
    enum a = [ 3, 1, 2, 4, 0 ];
    static immutable b = sort(a);

"a" only appears in the compiler's memory. "sort" is a normal function that runs at compile-time. "allowing accessto the full language at compile-time" is similar to what dynamic languages such as Python and JavaScript give you, except D is a static language with GCC and LLVM backends.

[vertex-four]

Rust is getting that, gradually, with the work of replacing the ad-hoc "const evaluator" with miri (an interpreter for a Rust intermediate representation). Right now you have procedural macros, which are Rust code that operates on the syntax tree, including custom attributes.

Proper reflective metaprogramming would be a fairly big step though - right now, the macro systems happen well before the type system even gets a chance to look at the code, so the data to play with types in an interesting way isn't there at the right step.

[Animats]

Rust is getting that, gradually.

Uh oh. I'd hoped the language would settle down.

The Go crowd knows when to stop. Go is mediocre, but stable.

[shpongled]

You say that like mediocre is a compliment. But I guess "mediocre" has been the best descriptor of Go from its inception.

[Animats]

Kind of, yes. Go has a specific goal - replace C++ for server-side web related stuff. It's supposed to be dull, boring, reliable, and suitable for use by large numbers of programmers doing that kind of work. It's a success at that.

Go got some things right, like "goroutines". Other languages are trying to add threads to a language with "async" type callbacks (Javascript), or "async" type callbacks to a language with threads (Python). Those concepts do not play well together, especially when retrofitted. Go does have a one-size-fits-most solution which handles the main use case - a server process handling a very large number of intermittently sending clients.

The same can be said of "functional" add-ons. Full-on functional languages can be OK, but functional features in an imperative language tend to be on the painful side syntactically. Retrofitting "functional" is even worse.

[Rantenki]

It's possible to stop too soon though.

[dimtion]

One issue not yet mentioned with Turing complete language at compile is that it makes tooling and IDE integration much more difficult.

When you need to run an unbounded program each time you want to provide real time feedback, like type inference or in Rust case lifetime inference, you make the language tooling much less simple and accessible.

[dependenttypes]

> One issue not yet mentioned with Turing complete language at compile is that it makes tooling and IDE integration much more difficult

How so?

> When you need to run an unbounded program

How is a program that provably terminates but takes 2 years to finish any better for compile-time computation? You want timeouts in either case.

[ImprobableTruth]

>How is a program that provably terminates but takes 2 years to finish any better for compile-time computation? You want timeouts in either case.

I'm not sure how realistic that actually is. Besides certain party tricks like encoding the ackermann functino using primitive recursion, I haven't actually seen anything like that. From my experience the vast majority of programs that don't finish in a reasonable amount of time are those that have some logic bug.

A timeout seems pretty off-putting. The idea that compilation could fail on a weaker machine just because it isn't fast enough just doesn't sit right with me.

[dependenttypes]

> A timeout seems pretty off-putting. The idea that compilation could fail on a weaker machine just because it isn't fast enough just doesn't sit right with me.

The timeout should be set based on what the user of the machine considers acceptable. Why should I have to tolerate a 2 hour wait time for a compile-time computation to finish in order for the auto-complete menu to appear on emacs just because I am using a laptop from 2010?

> Besides certain party tricks like encoding the ackermann functino using primitive recursion, I haven't actually seen anything like that

Try prolog then (or something similar) and write something of the form sha512(X) = 0 this will initiate a brute-force search that will last essentially forever.

[ImprobableTruth]

>Why should I have to tolerate a 2 hour wait time for a compile-time computation to finish in order for the auto-complete menu to appear on emacs just because I am using a laptop from 2010?

I don't understand. If you aren't willing to wait that amount of time, you're simply not getting working auto-complete at all. Hell, if you can't even compile the project, I don't see how you can meaningfully work on it all.

>Try prolog then (or something similar) and write something of the form sha512(X) = 0 this will initiate a brute-force search that will last essentially forever.

This is literally just another nonsensical party trick. I'm talking about something that someone might actually want to use, not contrived examples that really don't matter at all. I don't care to consider imaginary people that intentionally brick their programs.

[dependenttypes]

I just realized that you said "are those that have some logic bug" rather than "are those that due to a logic bug loop forever" and also mentioned that this is true for the "vast majority of programs" in which case.. sure? If anything this is an argument that supports timeouts and can happen in both turing-complete and non-turing complete languages, plus I do not understand how popularity has anything to do with it.

> If you aren't willing to wait that amount of time, you're simply not getting working auto-complete at all

Or I am getting a working auto-complete for code that does not make the completion system to get killed by the timeout.

> Hell, if you can't even compile the project

I never mentioned not being able to compile the project.

[breatheoften]

A user configurable parameter for timeout would be a terrible language design mistake.

You would want as much as possible to make the timeout not based on runtime -- but on an abstract model of the computation which doesn't actually depend on any machine particular. "Hey your program didn't compile on my machine -- try a faster cpu" would be a nightmarish interaction.

You want constraints about how long these programs are allowed to execute to be tight enough for developers to learn a sense for what is reasonable to do with these features -- as opposed to waiting for runtime.

[dependenttypes]

So, the user should not have the option to say "hey, I want this done within 10 minutes because I have to leave soon" or "I want my auto-complete to be done within 1 second", got it. Instead we should care about the feelings of arm and atom cpus.

[touisteur]

The tractable replacement for a timeout is some kind of steps number. I thinks I've seen lots of long-running static-analysis tools use this for reproducible builds and results.

[qppo]

The analog of a timeout in a compiler is recursion depth limits.

[dllthomas]

Or just instruction count. Although in either case, while we are now machine agnostic (with some reasonable assumptions about the design) changing optimization flags that a build is running under might make it fail.

[ImprobableTruth]

That alone definitely won't suffice. A naïve fibonacci implementation will give you a recursion depth of n, but even fib(64) take ages.

[mikepurvis]

I don't think he's saying that a limit on recursion depth solves this problem. Rather, it's just a similar scenario where a real-world constraint (memory, time, whatever) comes to bear and may make a compilation fail on machine X where the identical code and environment would otherwise succeed on machine Y.

[btilly]

A language that is Turing complete at compile means that an IDE cannot reliably tell whether code has a syntax error without running a program that can take arbitrarily wrong.

All other tooling faces the same issue. What variables have a given type? If determining the type takes arbitrarily long, how are you supposed to discover them? Now consider the plight of an automated refactoring tool.

The IDEs and tools can still be written, of course. But they take more work and can't be as reliable.

[dependenttypes]

> A language that is Turing complete at compile means that an IDE cannot reliably tell whether code has a syntax error without running a program that can take arbitrarily wrong.

A language that is not Turing complete at compile-time does not necessarily mean that an IDE will be able to reliably tell whether a certain piece of code has a syntax error without running a program that can take an arbitrarily amount of time to finish.

[btilly]

It is possible to construct counterexamples, but in practice it tends to work out much, much better.

The various tools built around Java stand as an example.

[GolDDranks]

No, that's not true actually. It's possible that the language is defined so that the syntax checker (lexing, parsing, name resolution) and even type checker is run first and compile time execution after that. The only thing what the undecidability then affects is whether we get the constant value or whether the program runs forever. But in the latter case, we have still performed successfully syntax checking.

[codr7]

I suspect a big reason is that you need an interpreter in addition to the compiler. From what I understand, the D gods spent quite some time and effort developing theirs.

For interpreted languages, there are no excuses; hooking into the interpreter at compile time is trivial. Full macros [0] are nice, but depends a lot on the syntax. A way to evaluate expressions at compile time [1] would go a long way.

[0] https://github.com/codr7/gfoo#macros

[1] https://github.com/codr7/gfoo#bindings

[dependenttypes]

If you do not want to implement a separate interpreter you can do multiple compilation passes instead.

[mikepurvis]

True, but in the cross-compilation case, you then need to do fun stuff like compile anything invoked at compile time for the host arch as well as the target.

[zokier]

Rust have procedural macros https://doc.rust-lang.org/reference/procedural-macros.html

[octo_t]

Another issue not mentioned is cross-compiling.

If the result of my `sort ` function is dependent on `sizeof(void )` being 8, when I compile from my x86 hardware for a 32-bit only architecture, assumptions go awry.

Note that this isn't likely with something like sort, but definitely is* likely with precomputing values/structs etc.

[dilap]

Zig takes this approach. I haven't done more than kick the tires curiously, but it seems very cool.

[staticassertion]

They do. You can.

You have procedural macros, which are literally "Rust code is the input, which you write rust code to manipulate, and output more rust code".

You have const fns, which are interpreted at compile time, and are closer to what you're referring to.

[pull_my_finger]

Terra[1] does this using Lua as it's compile time language.

[1]: http://terralang.org/

[ncmncm]

The most useful values at compile time are types, so straight-up interpreting the core language at compile time is not enough. You need meta-typed named values to transport types in, and meta-functions to pass them to.

Fortraith cleverly re-purposes existing features, doing some violence to language usage conventions to achieve it.

[dependenttypes]

> so straight-up interpreting the core language at compile time is not enough. You need meta-typed named values to transport types in, and meta-functions to pass them to.

Can you please explain what you mean by this?

[neutronicus]

In Rust (for example), Types are not Values. You can't store them in variables, pass them to functions, etc.

However, at macro-expansion time, often what you want to do is examine a Type and expand the macro differently based on some information about the Type.

So you would like your Macro language to have some notion of manipulating Types that is absent from the language itself

[dependenttypes]

It should be noted that gcc and clang both execute certain functions at compile-time at their higher optimization levels.

In addition C++ has constexpr which marks an expression to be evaluated at compile-time.

[GolDDranks]

The recent signs are that in near-future Rust releases you are able to have rudimentary logic in constant expressions: if, match, loop etc. (Around Rust 1.47 ish. That's three months away.)

When the capabilities of the constant evaluation system get improved and stabilized, we are going to see the exact feature you are describing. (With the caveat that non-deterministic functions are not allowed to ensure deterministic builds.)

[zozbot234]

> I don't see why people won't just take the step D and Lisp do-- allowing full use of the programming language at compile time.

Because "full use of the programming language" implies Turing completeness, which means compilation may require unbounded time and compute resources. You can allow use of a non-Turing complete subset, and this is something that dependently-typed languages can do quite elegantly.

[pilif]

Many type systems are already Turing complete like C++ and indeed Rust as evidenced by the project linked here.

It’s a valid point. Of compilation already is Turing complete, why not just drop the pretense and allow arbitrary compile time expressions.

[dllthomas]

Even where nothing in compilation is Turing complete, many pieces may be unbounded. And where there are artificial bounds, they could as well be applied to something otherwise Turing complete.

[dependenttypes]

A program that requires 2^256 time units and 2^256 memory units is not really any better compared to unbounded computations.

[s17n]

Lisp failed as an industrial language precisely because it has powerful meta programming capabilities. In the real world, nobody wants to have to solve a puzzle every time they visit a new area of the codebase.

[zamalek]

This is just nerd flexing. Rust has a slightly more flexible compile time code generation story than D. There's even a package that connects to your SQL DB and type-checks your queries.

[BiteCode_dev]

I hope not.

I really don't want Rust to be crippled with a 1000 of DSL just because devs could write them easily.

[empath75]

Doesn’t the compiler already do optimizations like that?

[a1369209993]

Because that's fundamentally incompatible with a competently-designed compiled language. Consider:

  fn immutable foo() long:
    asm(long x:"rax") "mov rax 7"
    return x

Now try cross-compiling that from a 32-bit ARM machine.

Aside: D is kind of weird in this regard because most of it is designed to work as a interpreted language as well as a compiled one. To the extent the D is good, it's not compiled[0]; to extent that it's compiled[0], it's not good.

0: in the language design sense, not the language implementation sense.

[BeKindAndLearn]

Pretty much all languages with turing-complete compile-time expressions only expose a subset. This comfortably lands in the "not in that subset" category.

[a1369209993]

> [...] and Lisp do-- allowing full use of the programming language at compile time.

If you only expose a subset, you very specifically don't have the full use of the programming language.

[BeKindAndLearn]

Now you're just being pedantic. Also, leaving out the "D" just before your quote is plain disingenuous. That contextualizes the discussion and clearly establishes we don't mean being able to directly run ASM operations and syscalls willy-nilly at compile time.