[blagie]

For the most part, programming constructs like these split the world into two camps:

- People who point out things can be done without them, who largely see them as useless due to lack of familiarity

- People who've used them, and see critical ways to restructure code to make it cleaner using said constructs

That's been the case for a lot of progress in programming. Python, and ES2015, have done a wonderful job of bringing many previously-academic (including functional) programming constructs to a broader community, together with models like reactive programming.

That's true of about half of the bits of progress in programming. Garbage collection was seen as garbage by C/C++ programmers ("What's the big deal with adding a free() call? Stupid lazy people."). Garbage collections wasn't useful because it omitted free() calls, but because it allowed the design of flows with many exit points (for example, different kinds of exception handling, exiting in the middle of a function, or giving up an object in a half-dozen places an hour later in code where tracking for free() requires building an ad-hoc reference counter or garbage collector).

The place where tail calls are a huge deal is when dealing with deep (or even infinitely deep) tree-like structures:

    if condition: 
        return red(left_child)
    else:
        return blue(right_child)

I don't mind opt-in versus opt-out versus neither. All the reasons listed for not having them are dumb, though, and have good and easy work-arounds. The major one -- debugging -- it's basically always good enough to just have a list of functions called, without having the whole tree. A 10,000 element stack trace is no help at all. You can, for example, keep the first 20 elements of the stack trace (don't start PTCs unless the stack is of a certain depth), and then still keep a list of functions called:

    ipython
    webapp.main
    webapp.handler
    render.make_tree
    [PTC: render.red*91001, render.blue*10201]
    webapp.callback

I have literally never seen a case where having a list of 100k calls in a stack traces is at all useful for anything.

[PathOfEclipse]

On a tangent, I've always felt garbage collection, and, more importantly, safe memory management, was important because it allows you to mostly pretend that memory allocation isn't a globally side-effecting operation, as such operations are difficult to reason about. It's the same logic that leads to design choices that don't explicitly use global variables or global state.

In reality, all memory allocation is still globally side-effecting, and you'll find that out when your program starts GC spiraling, or consuming more memory than you want it to, but being able to pretend otherwise and mostly get away with it means automatic memory management brings a tangible, measurable productivity multiplier to programming that few, if any, other programming language features can boast of.

[blagie]

I disagree. My experience with codebases with programmers who program like that is that you eventually get memory leaks, some of which are near-impossible to debug or fix. You pay for that kind of thinking down-the-line.

Enabling new design patterns == good

Not thinking about what happens under the hood == bad

Catastrophes are rare, but expensive enough to cost more than thinking things through.

[tsimionescu]

In my experience, memory leaks in GC languages are very easy to track down, since the timing for analyzing the heap is excellent. While discovering that you do have a memory leak is not very easy, once you know about it you can easily compare heap snapshots, find the increasing objects, and track down their GC roots, all from the same tool. With Java especially, you can even do this on a running service in production, with minimal downtime.

In contrast, it's much harder to track down memory leaks in C, since the runtime has little information about the contents of the heap. You typically end up using valgrind to instrument your code, assuming it can still run fast enough to reproduce the problem.

The only exception is Go, which has awful tooling for analyzing memory. For some bizarre reason, it can't even show you the gc roots of an object in memory, even though the GC obviously needs this information to work properly.

[xigoi]

Where do memory leaks appear more often, in C, or in Python?

[blagie]

My experience is that Python has *many* more memory leaks than C code.

They're less visible since I ran C code on a computer with 32MB of RAM, and I run Python code on a computer with 32GB of RAM, but they're very common in modern Python programs.

As another comment pointed out, memory issues in Python are less dangerous too. I'm glad not to worry about buffer overflows in Python.

[hajile]

Memory leaks appear in BOTH (specifically, any data structure that accidentally keeps references to stuff will leak memory in python). The biggest difference is that Python won't create the particularly bad/dangerous memory leaks (eg, use after free).

[Jtsummers]

Use after free is not a consequence of memory leaks. A memory leak is, specifically, memory which is still allocated but not referenced. Use after free errors can't happen if you don't free the memory.

[jcranmer]

The problem with tail calls destroying stack traces isn't with the recurse-for-the-tree functions. It's with code like this:

  function frobnicate_something(foo, bar) {
    let baz = antifrobnicate(foo, bar);
    // Hey, I'm a tail call!
    return exfoliate_meteor(baz);
  }

With tail calls, you now lose the frobnicate_something stack frame and you just see a call to exfoliate_meteor, which can produce confusing results. This lost stack frame is potentially quite injurious, especially when the function that's lost actually does a serious amount of work.

This is the rationale behind the proposal to support tail calls only on explicit scenarios, where the developer opts in to throwing away stack frames.

[gumby]

A tail call is just a goto; using a special syntax for it (`for…` or `while…`) is simply syntactic sugar. Nobody complains about the lack of a stack frame in a for loop.

[ajuc]

Goto can be anything - a function call, a loop, an if clause, exception handler. That's the whole reason we created structured programming - so we don't have to guess what it is THIS time.

Tail call optimization is nice, but it has its problems. What's the downside to requiring special syntax for it? It frees the programmer reading the code from determining if it's a tail call every time.

[hajile]

The downside is that your code doesn't optimize for free.

iOS is over 50% of the mobile browser share in the US and desktop Safari is 11-12% (roughly 1 in 9) and both implement proper tail calls and have for around 6 years now. Despite this, the world has not collapsed and people aren't constantly complaining that websites are constantly breaking.

[kreetx]

Right, but optimizing tail calls won't lose that structure. But sure, just using goto is a slippery slope to confusion.

[ajuc]

Optimizing calls using the same syntax as regular ones risk introducing stack overflow without noticing.

    function rec(x, y, z) {
       if (something(z)) {
          return rec(x, y, z-1);
       }
       return 1;
    }

Now you need to add 1 to the result.

    function rec(x, y, z) {
       if (something(z)) {
          return rec(x, y, z-1) + 1;
       }
       return 1;
    }

Ups, now it fails with stack overflow for big z values. Hope you have good unit tests to catch this.

To avoid this problem whenever you modify a possibly recursive function in a language with transparent TCO - you have to look through the code to see if it's using TCO. This is wasted time. And it can be non-trivial if you have recursive functions calling each other and other funny stuff.

If instead the language required special syntax for TCO:

    function rec(x, y, z) {
       if (something(z)) {
          return TAIL_CALL rec(x, y, z-1) + 1;
       }
       return 1;
    }

This would be a compilation error cause it's not a tail call contrary to the declaration.

What is the downside to requiring special syntax?

[zelphirkalt]

Idk, rarely (never?) happened to me, that I make a tail call into a non-tail call, because I ad-hoc add something like in your example to it. Seems like a beginner mistake. When I design a function to be tail recursive, which is all the time in langs supporting that, I wont suddenly disregard all care and add code like that. And hopefully other people will at least spent a minute reading the function they are modifying, before slapping that change on and calling it a day. If not, then some syntax wont protect you from that either.

[kreetx]

I don't quite get the argument. Isn't this similar to e.g requiring the `z` parameter to be annotated with something that ensures that it is a small number? ("What is the downside of having special syntax for that?")

My undrstanding is that a tail call is a tail call, and the variant with +1 is not that, thus being a candidate to be looked at.

Perhaps also IDEs can help here if it's difficult to spot?

[xigoi]

The downside is that most web developers don't care about performance, so they wouldn't bother to use it.

[blagie]

The proposal to be explicit is fine, but there are alternative proposals which avoid this problem too:

- Only tail recurse once the stack is n objects deep. Having a 10,000 element stack trace isn't helpful.

- Keep a log of which functions were called, but not a full stack trace of each call. That's O(1).

I mentioned both of those in my post.

[kreetx]

The parent mentioned keeping a list of functions called, so a note on calling frobnicate_something isn't lost.

[saagarjha]

> Python, and ES2015, have done a wonderful job of bringing many previously-academic (including functional) programming constructs to a broader community

This is kind of amusing to hear considering that Python seems to actively design itself around making typical functional programming constructs difficult and JavaScript has a bunch of weird quirks with how it implements things (you’ve seen the “map” joke perhaps?)

[blagie]

The major thing Python has done is bring major functional design patterns, like maps, reductions, everything in itertools, and made human-friendly versions and syntax for them. Right now, that covers 70% of the stuff I did in Scheme over C/C++ (weighed by volume of use, rather than by feature list). A lot of beginner programmers now use features like closures and decorators, and they're accessible.

The remaining 30% is awkward, but usually possible.

JavaScript is convoluted.... but after the learning curve, it does functional as well as anything, and better than most other things it does. It certainly does functional better than OO.

[int_19h]

I would argue that, as far as popularizing sequence comprehensions goes, C# (LINQ) was probably more influential than Python.

[blagie]

Hejlsberg is a genius.

Before C#, he designed Delphi, which was a pleasure to use. It made Pascal beautiful, which is not something many considered possible. Functionally, Delphi combined the ease-of-use of VisualBASIC with the power of C++, with an elegance unparalleled in any environment before.

He did Turbo Pascal. It's hard to overstate how much of a revolution that was. Compile-wait-wait-wait-run turned into run.

Now, he's doing TypeScript.

I'm not sure it's really fair to compare anyone to Hejlsberg.

"Sure, your kid won a Nobel Prize for his research, but Einstein did relativity...." "Sure, your business hit a billion dollars, but it's no Apple...."

Acknowledging brilliance elsewhere doesn't reduce my appreciation of Python. It's a good language.

[xigoi]

LINQ weirdly renamed basic FP concepts. Why is the map operation called “Select”? If anything, I'd expect the filter operation to be called that.

[tsimionescu]

It's SQL inspired language, because SQL is far and away more well known, and understandable, than functional programming.

And that is one reason why LINQ is still one of the only functional collections APIs that exposes GroupBy, which is an extremely useful operation.

Edit: correction, LINQ is no longer the only one that introduced GroupBy, but it was probably the first.

[greymalik]

I program professionally in both languages and in my experience the two are very different when it comes to functional programming. FP is viable in JavaScript, if imperfect. And as the language continues to evolve to better support that paradigm its community continues to embrace it. But Python’s choices seem to be actively antagonistic to FP. I’d like to be able to use a functional approach in Python when it makes sense but it’s too hard, too awkward, too flawed, and too against the grain of the language.

[saagarjha]

I’d agree in the sense that JavaScript gives you poor out-of-the-box support but it definitely can be shaped into something pleasant. Python will make you hate yourself if you try, yes.

[moonchild]

> Garbage collections wasn't useful because it omitted free() calls, but because it allowed the design of flows with many exit points

This misses the mark a bit. C++ destructors allow for multiple exits. Design at the level of an individual subroutine is not very important, ultimately.

Garbage collection is useful because it enables greater modularity, at the level of full applications and protocols. It does not force details of memory management and ownership into api contracts, leaving them free to be changed. (This, incidentally, is one of the reasons smarter people than I take issue with rust: it forces details of ownership into api contracts, reducing modularity. It is also the reason why reference counting is less modular than tracing gc: it does not deal with cycles, so cyclicality is part of api contracts, and a very subtle part at that.)

[quietbritishjim]

> People who point out things can be done without them, who largely see them as useless due to lack of familiarity

There's no need to dismiss the those who disagree as just having a lack of familiarity. There is a technical trade off to be made, with pros and cons each way. The only people that are objectively wrong are those that claim the decision is clear cut.

The con, in particular, is language complexity. You only have to look at C++ to see that it is possible to include to many features in a language. There's no disputing that tail calls are useful. The question is whether they are so useful that everyone who learns the language has to understand them.

[edoloughlin]

Garbage collection was seen as garbage by C/C++ programmers ("What's the big deal with adding a free() call? Stupid lazy people.")

As a former C++ programmer working on a large code base when/before Java was introduced, garbage collection was seen as expensive and slow, but definitely not a waste of time. I would have given my right arm to be free (pun intended) from the need to manually manage my heap across threads. Adding a free() call was never not a big deal.

[froh]

exactly, it was never about attitude, it's about deterministic real-time. C/C++ are low level systems languages for real-time control applications. In that case garbage collection, a non-deterministic unpredictable stop-the world pause, simply is out. so you have to have a base language without GC. you can use garbage collection via libraries in c/c++ if you don't need deterministic real time. It never was about "not liking" GC or "thinking it's garbage".