[curryhoward]

Regarding the need for evidence in programming languages research: maybe it's just my bias for theory, but I think of PL research as a branch of mathematics more than an experimental discipline. In mathematics, you don't cite evidence, you write proofs. Most of the papers in conferences like POPL deal with topics like domain theory, type theory, category theory, formal logic, semantics, models of computation, etc. Results in these fields are achieved on paper or in a proof assistant, not by taking a survey.

For whatever reason, papers about programming languages that attempt to use data to make arguments about productivity, safety, etc. always seem unconvincing to me. Maybe it's because they usually aren't reproducible, or maybe it's just because the way most programmers write code doesn't matter to me, because most programmers aren't aware of what is possible on the frontier.

[StefanKarpinski]

Given your user name, that's a fitting perspective. I have a very different view: programming language design is applied psychology, not mathematics. Programming languages are the ultimate human-computer interface. It may be tempting to ignore the human side of that interface: after all that side is messy, incomprehensible, impossible to write neat proofs about, and almost as hard to publish tidy convincing papers about. Without the human side of programming languages, however, PL design is a fairly pointless endeavor — without the humans, we wouldn't need programming languages in the first place.

I fear that, for these rather tempting reasons, academic programming language research has fallen into the trap of treating PL design as a form of mathematics, ignoring the human side of the field, and as a result has contributed far less to the practical practice of programming in the past several decades than one might naïvely expect and hope. Academic PL has, for example, almost completely disregarded the wild success and popularity of dynamic programming languages. Instead, prominent PL researchers glibly dismiss dynamic languages as "unityped", ignoring that there is, in fact, a rich life of types in every language if only you shift your notion of types to match what the users of those languages actually mean by the term. Shouldn't more academics be interested in how and why this approach is so successful to see what can be learned from that success, rather than just dismissing it out of hand because it doesn't fit neatly into the theory?

We have seen a recent cross-pollination of ideas between dynamic and static programming languages resulting in the likes of TypeScript, Go and Julia—and these hybrids have been wildly successful. But none of this is coming from academic PL circles. It's coming form the people who view languages as tools and want tools that fit the hands of their human users, rather than designing beautiful but alien tools that don't fit human hands. Even heavily static languages like Rust that have become very successful, have done so largely by going against the grain of static language research.

So yes, academic PL does seem to have largely become a subfield of mathematics, but to that extent it has, it has also become as detached from actual programming as modern literary theory is from the practice of writing books that people want to read.

[coolgeek]

> I have a very different view: programming language design is applied psychology, not mathematics.

It's both, and probably more.

As you note, programming languages are the ultimate human-computer interface. They have to serve two masters - the computer's unforgiving hardware, and the inevitably fallible humans attempting to harness the computer's power. Each master requires different types of proof/evidence of the language's suitability to the task.

[joe_the_user]

Maybe it's because they usually aren't reproducible, or maybe it's just because the way most programmers write code doesn't matter to me, because most programmers aren't aware of what is possible on the frontier.

I have to say that sounds remarkably arrogant.

I think a significant portion of programmers are very smart people who work hard at producing good, generalizable, understandable effective code. PL designer would do well to pay attention to them (not that I know whether or not other PL designers have the parent's attitude. I hope they don't).

[curryhoward]

> I think a significant portion of programmers are very smart people who work hard at producing good, generalizable, understandable effective code.

To be clear, I don't disagree with this, but it's not relevant. You wouldn't expect a typical mechanical engineer to have research-level mastery of theoretical physics, even if they can build impressive, reliable machines. It's not an insult to the engineer.

[joe_the_user]

You wouldn't expect a typical mechanical engineer to have research-level mastery of theoretical physics, even if they can build impressive, reliable machines.

I'm trying to figure out the world you're imagining here. I can see several possibilities.

A) The programming language research you are doing will eventually yield some programming practice that will be so advanced that the programming that happening now, before this change, will turn out too be irrelevant. Thus you are paying no attention to what's happening.

B) The programming language research you are doing will never intersect with the world of the ordinary programmer. You will prove interesting theories to say things about mathematical objects that happen to be programming languages, working on a track forever parallel to what ordinary programmers are doing.

C) Like a theoretical physicist, you're producing insights about physical reality on a much lower level than the average in engineer. If your insights yield an advance in understanding, you won't be the one to turn into a practical tool. That would be the work of the many layers applied-sciences practitioners that sit between the physicist and the engineer.

Choice C seems at least logical. But I'd claim that programming language designers considering things this way is not plausible. The world of programming abstractions just doesn't have enough layers that you're going to get anything like a pure theoretical science without relation ordinary human step-by-step problem solving. Moreover, we know layers of applied scientists don't exist between the ordinary programmer and the language designer. If you want your stuff to be relevant, you'll need to sell it yourself, unlike the theoretical physicist.

[emteycz]

I spent few years along the most average developers you could find. They didn't care about their work outside their work hours, didn't care about anything other than Java and EJB and their hourly rate. I am okay with it, it's not like a less able programmer/gamer/cook/... would anger me, but these people indeed are distinct, and there is a lot of them

[tasogare]

> significant portion of programmers are very smart people who work hard at producing good, generalizable, understandable effective code

Yet this is very different from what GP is talking about: knowledge about the research frontier. Which is OK, programmer and researcher are two different jobs, and much like it isn’t expected from the scientist to write good code (or write code at all), the programmer job is not to be knowledgeable about the state of art in PL theory.

[nv-vn]

>For whatever reason, papers ... that ... use data to make arguments about productivity, safety, etc. always seem unconvincing to me. ... maybe it's just because the way most programmers write code doesn't matter to me, because most programmers aren't aware of what is possible on the frontier.

As someone involved in PL research I partially agree, but I think this effect happens in both directions. There are plenty of great ideas in PL that haven't been applied to real problems whereas, simultaneously, there are plenty of huge problems for programmers where PL hasn't been applied. That is to say, us researchers often ignore the problems of the masses in favor of ones with more elegant (or more rigorous) solutions because we enjoy writing fancy proofs and theory more than we enjoy pragmatism. I think both sides of the equation are very important here, but it would be great to see some more direct interaction between the two sides. As it stands, a conference like POPL contributes about as much to recreational math as it does to the state of the art in programming.

[ssivark]

Taking a step back, mathematics just happens to be our most developed tool/perspective for understanding patterns. So, to the extent that programming languages are concerned with modeling certain patterns in programs or domains or computing devices, at one extreme they might end up looking like (applied) mathematics. I consider category theory to be an exercise in this spirit (and this approach is broadly quite valuable precisely because math is often our best modeling framework for phenomena!)

At the other extreme for detecting patterns is simplistic empirical study of the kind that you allude to. It might be difficult to reproduce or generalize from these datasets, but it's hard to do much better without a better framework to operate in.

In between these two extremes is one of the most interesting threads of PL research, where it is treated as a design problem. For this to be possible, the researcher uses a hunch based on their human intuition of a domain or a problem-solving technique, and then tries to reify it into a programming language to provide it as an affordance to users of that system. It is strongly influenced by the taste of the researcher and the reviewers, and often needs to be iterated on to explore an interesting space.

What is the distribution between these three in academic PL research -- I have no idea. But I have seen many interesting demonstrations of the last kind, especially in conference talks at Strange Loop.

[higerordermap]

Unpopular opinion: The abstract mathematics side of PL research is progressing good enough. More important thing is to improve tooling, and it matters we should be able to design efficient and ergonomic programming languages with tooling focus in mind.

Maybe hardcore Set theory people don't accept progresses in Zig or Rust as PL research, but they are very important thing as well.

[MaxBarraclough]

From what I gather, good progress is being made on the SPARK Ada tooling, for formally verified software development.

Do you have something specific in mind? IDEs? I don't know much about this but I presume work is being done on Rust/Zig IDEs.

[TheNewAndy]

That is an interesting take, and it seems common in PL research (even if unspoken).

I love the formalism and "hard" maths of PL design - but if the purpose of a PL is to bridge the chasm between how computers work (i.e. machine code) and how people work, then disregarding one side of this isn't right.

The belief I previously held was that this is ok because computers don't get smarter, but you can teach people about the difference between contravariant and covariant types, and then pretty soon they will be happy scala programmers. Now I am not convinced - a simple language which fits in people's heads, and a way of leaving comments. Yes there will be bugs that don't get picked up at compile time - this happens in all languages - but the bugs that would be caught by a richer type system in another language tend to be the ones that are the easiest to fix.

[moonchild]

> papers about programming languages that attempt to use data to make arguments about productivity, safety, etc. always seem unconvincing to me

Productivity and safety are aspects of human interaction. No programming language is safe, or productive in a vacuum. It is impossible to make claims about safety or productivity in a programming language without relating them to human use.

[samth]

While most POPL papers do contain some formalism, your description is strikes me as out of date, even for POPL.

[Guthur]

And here is the disconnect. While your claim is pedantically correct the reality is that most practical programming activites are engineering not mathematically science.

Like the architect, steel worker, joiner, bricklayer and so on we use many materials and techniques that can be explained and proved with science but we do so while be broadly ignorant of it all. And that's totally fine.

The problem if you disappear down the rabbit hole of academic self indulgence you may forget you were suppose to actually build something.

[azhenley]

> Maybe it's because they usually aren't reproducible

Not reproducible is different than no one attempting to reproduce it.