[chromatic]

I recall things differently! The leaders of the Parrot VM and the leaders of the P6 implementation and the P6 language designers were on the phone directly every week talking through things.

Up until we weren't, but that was basically the point where the P6 language designers said they wanted to write their own VM anyway.

[microtherion]

Well, you were there and I was not; all my impressions are secondhand. But, looking back at my grandparent comment, I worded this really poorly.

It was not whether the people were talking, but whether the components grew in convergent directions. My impression was that for extended periods they did not. Am I wrong?

[lizmat]

My take:

1. Parrot was initially intended to be the backend for Perl 6. 2. Perl 6 didn't materialize fast enough, so Parrot started focussing on hosting other scripting languages. 3. Parrot decided to become the backend for all scripting languages. 4. Parrot became less and less targeted towards Perl 6. 5. Parrot never really became a backend for any other scripting language. 6. Parrot became decicedly a poor fit for Perl 6. 7. Jonathan decided that maybe the JVM could also be a backend, and started porting 8. Building on that experience, Jonathan decided that they could write a dedicated backend for Perl 6, which became MoarVM 9. Parrot development dwindled to effectively 0. 10. During the Great List Refactor of 2015 (about 6 months before the first release) it became clear that porting the Parrot backend would be a lot of additional work, with no apparent upside 11. Parrot support was dropped before the first official release

[chromatic]

Parrot was always intended to support multiple languages, from Simon's original April Fools joke to Larry's dual desire to run both P6 and Perl in the same process on the same VM as well as to make CPAN available to all dynamic languages in the same process on the same VM.

That's why Dan and Guido had a bet that Parrot would run Python faster than CPython, and that's why Parrot attracted contributors like Sam Ruby.

It's definitely true that, as P6 dragged on and burned out a lot of contributors (including multiple project managers), many Parrot contributors thought that coupling Parrot's success to P6 was a mistake, but I can't see that as a change in focus because the multi-language focus was a goal from the start.

Granted, Parrot did wander in the wilderness for a while waiting for the P6 design process to proceed, and Parrot did evolve in some unfortunate directions because of that. The rest of this comment covers 2007 onwards though.

Your question about components is particularly interesting, because of what really happened.

A couple of P6 developers wrote PCT, a set of tools and libraries to make developing compilers and languages on Parrot easier. This was a good thing, and Parrot users and developers started migrating their language implementations to that toolkit.

The PCT developers refactored and revised this a couple of times, and to make that easier, eventually they took the code out of the Parrot tree and started syncing PCT into the Parrot tree on a semi-regular basis.

I was working with them weekly to fix Parrot bugs, improve Parrot performance, and add Parrot features to reduce the amount of code necessary in PCT. Thus it's difficult for me to believe that the Parrot developers were designing and implementing Parrot away from P6.

Toward the end of 2010, the P6/PCT developers said they were going to rewrite PCT to support multiple VM backends. I argued this would create a lot of churn in the language implementations actively using PCT (ironic, because the PCT developers spent a lot of time before and after this complaining bitterly that they couldn't keep up with Parrot churn) and would provide very little value for Parrot (PCT was out of tree already, and syncing in code designed explicitly to undercut Parrot seemed like a stupid idea). These discussions all took place in the public Parrot IRC planning sessions, which were logged.

I also had concerns about yet another rewrite undercutting P6, especially coming six months after the underwhelming Rakudo Star release, but it was clear which direction the winds were blowing, and that's when I stopped working on both P6 and Parrot.

Five years later, P6 had its first official release and Parrot was well buried.

I know this contradicts the official Raku hagiography, but I've gone back and read the primary sources a few times over the years as people have asked about it, and I'm confident the record backs up what I've said.

[lizmat]

Well, you are definitely more informed about the pre-2012 period. I only really got involved from 2012 onward, so I have no first hand knowledge about that period, except maybe a few weeks in 2000 (when I was dragged in by Johan Vromans) and 2005 (when I was dragged in by Audrey).

I just know second-hand that something broke the trust between the Perl 6 and Parrot developers around the first Rakudo Star release. And that strengthened the notion that another backend for Perl 6 was needed.

> I know this contradicts the official Raku hagiography

OOC, which official hagiography are you referring to?

Also, I find the use of "hagiography" in relation to Raku a bit strange: if anything, with the rename, the last saint has basically left the building?

[raiph]

> I only really got involved from 2012 onward, so I have no first hand knowledge about that period, except maybe a few weeks

I'm in roughly the same boat. I too dipped in and out starting in 2000, and had been reading contemporary discussions on and off from around 2009.

I want to draw your attention to some things I know due to publicly available records, things I think shed light on what really happened.

The first thing is that Rakudo began technically shifting to a multiple backend architecture in mid 2009:

* Following Jesse Vincent's summer 2009 discussion with Patrick Michaud about Perl 6 running on JVM and .NET, NQP was rewritten in the fall of 2009 to support multiple backends.[0]

* jnthn edited his website to say his first interest in working on Perl 6 going forward was "Transforming Rakudo from a single backend compiler to one capable of targeting multiple backends".[1]

> I just know second-hand that something broke the trust between the Perl 6 and Parrot developers around the first Rakudo Star release. And that strengthened the notion that another backend for Perl 6 was needed.

As explained above, Patrick had already focused his and jnthn's efforts on a multiple backend Rakudo in mid 2009, a year before Rakudo Star.

And, as chromatic has said, and the record shows, commits to Parrot "fell off a cliff" in early 2011, about 6 months after Rakudo Star, and a year before the dates of the first commit timestamps in MoarVM's git history, in Q1 2012.

----

[0] https://www.youtube.com/watch?v=XgPh5Li3k4g#t=2m

[1] http://web.archive.org/web/20091218092004/http://jnthn.net/p...

[raiph]

> Weird how almost nothing Patrick talked about in that talk panned out -- no JVM backend ... no JavaScript backend

At the time of writing this comment:

js Use new nqp::time instead of nqp::time_(i|n) 20 days ago

jvm [JVM] Add op 'nativecallinvoke' 15 days ago

moar Disallow explicity specifying op write registers 8 days ago

From https://github.com/Raku/nqp/tree/master/src/vm

[chromatic]

The first thing is that Rakudo began technically shifting to a multiple backend architecture in mid 2009

Weird how almost nothing Patrick talked about in that talk panned out -- no JVM backend, no CLR backend (the one that Jonathan talked about in that blog post too), no JavaScript backend, no Tcl, no Perl, no Ruby, no Python frontends.

In fact, it's amazing how that fourth rewrite of NQP ended up not working at all for any of the backends Patrick showed on his slide. Or at least not working yet.

the record shows, commits to Parrot "fell off a cliff" in early 2011, about 6 months after Rakudo Star

How strange that that happened just after the developer summit where I said that the NQP strategy was going to leave Parrot developers in a dead end. Imagine that -- people don't want to stick around on a project that's continually undercut by bad technical decisions.

Thanks for finally confirming what I've been saying for years.

[lizmat]

> that the NQP strategy was going to leave Parrot developers in a dead end

If Parrot had provided all that Perl 6 needed at that time, I don't think the multiple backend strategy would have been selected. Why would one?

> where I said that the NQP strategy was going to leave Parrot developers in a dead end

To me that feels like a self-fulfilling prophecy. Also, had Parrot been able to attract other languages to a significant extent, the abandoning of Parrot by Perl 6 would not have been an issue.

Personally, I think Parrot was not able to attract other languages because:

1. it didn't provide enough benefits to account for the extra effort needed by language developers. It definitely did not for Perl 6 at the time.

2. it didn't have a core team welcoming enough to keep anybody vaguely interested in Parrot

In the end, it is always about being able to create a community in open source. Without it, a project is indeed on a dead end.

[chromatic]

I just know second-hand that something broke the trust between the Perl 6 and Parrot developers around the first Rakudo Star release. And that strengthened the notion that another backend for Perl 6 was needed.

If I recall correctly, that's about the time Jonathan started complaining that the Parrot object model didn't work very well for P6 and wrote his own for PCT, which is still weird to me because he was the primary implementer of the Parrot object model in Parrot. (I don't remember the dates, but he also wrote an object model in C# and another in Java, which you mentioned in another comment.)

I tried for a long time to migrate the PCT object model back into Parrot, but both Jonathan and Patrick told me repeatedly it wasn't a priority, so I stopped offering.

I'm not the only person who had this experience; I remember someone else having a similar experience with Unicode support.

OOC, which official hagiography are you referring to?

The one that says "Parrot dumped P6, so the P6 developers had no choice but to write their own VM".

I was there. That's not true at all.

[microtherion]

As a further clarification of my earlier remarks, when I was talking about Parrot developers not being interested in Perl 6 at all, I was not referring to your era, but to considerably earlier times: https://www.sidhe.org/oldblog/archives/000435.html

Different eras with different dysfunctions, and different people involved, but for some reason, always personality clashes.

[chromatic]

That was my era too, but it's definitely true that Dan lost interest in P6 well before he stepped down, and Leo never seemed to have a specific interest in P6.

[lizmat]

> The one that says "Parrot dumped P6, so the P6 developers had no choice but to write their own VM".

I don't think that I ever implied that. To me, in one sentence, it's more like "Parrot and Perl 6 diverged, so that Parrot was no longer a good fit for Perl 6, so the Perl 6 developers had no choice but to write their own VM"

[chromatic]

Parrot and Perl 6 diverged

To me this reads like the "mistakes were made" phrasing used when someone wants to be technically accurate but not say anything meaningful.

Regardless of whatever Raku developers and advocates say, I, as a lead developer of Parrot, went out of my way to make it work better for P6. Patrick asked me not to do certain things and so I didn't do those things.

Maybe that was my mistake.

But it's categorically untrue that Parrot wanted to get rid of P6 or considered P6 an afterthought or wasn't willing and able to be receptive to P6's needs, and I'm disappointed that that untruth continues to persist.

[b2gills]

The MoarVM and JVM backend supporting code was, and still are, very similar.

The Parrot backend supporting code was significantly different. So much so that to properly support Rakudo on Parrot would require a rewrite of it.

There are various reasons for that, but the main takeaway was that continuing to support Parrot made everything significantly more difficult.

(Even now the JavaScript backend code is more similar to the other two than Parrot backend code ever was.)

---

Parrot made a lot of assumptions about how objects worked.

It knew simultaneously too much about how objects worked, and not enough about how objects worked. It basically assumed that the Meta object protocol would be written in C for every object type.

MoarVM doesn't know how objects work. It gets told how they work everytime it runs. (Necessary since Raku can mutate its own object system.)

---

Rakudo on Parrot was slow. If you happened to use a Unicode character in your source file it got dog-slow.

Worse yet, Unicode was an optional feature of Parrot! WTF!!

That's not good backend for a language that has excellent Unicode support.

---

Not only did the NQP intermediate code for supporting Parrot need to be rewritten from scratch. A large portion of Parrot also needed significant work.

The main idea of Parrot was that there would be integers, numbers, strings, and other. It turns out that other was really the main category. And again Parrot was opinionated about what other even was.

It was thought that if something like integer bytecode was similar to machine-code that it would be easier to optimize it.

The problem is that something as simple as calling a function involves taking a Signature object and asking it if it accepts a given Capture object. Since even just adding two values together is a function call; doing objects fast is more important than special casing integers.

MoarVM is basically the other category first and foremost.

By that I mean if you took out the special-casing of integers, numbers, and strings from Parrot. Then redesigned the PMC code to support what Rakudo really needed, you would end up with something that is very similar to MoarVM.

The reason you might take that special-casing out is that making bytecode similar to machine-code was completely and utterly pointless. At least as far as Raku is concerned.

It doesn't matter that much if integer operations are fast if you are only ever going to run one in a row.

The reason that MoarVM isn't extremely slow is that it knows enough about objects to be able to pull them apart to only run the parts that need to be run.

---

I think that Parrot as initially designed may have been a really good fit for a Perl5+, Ruby, Python, or Lua etc; but it turns out that it really wasn't a good fit for what Perl6 eventually became.

---

Let's do a hypothetical thought experiment.

Imagine if someone did all of the work to make Parrot work again the day after it was dropped. That includes rewriting the middleware and doing enough to make the PMC part usable. So then it would have continued to work over the years.

I'm fairly certain that it would today be tied for 2nd or 3rd with JVM or JavaScript. (If not 4th.)

If the Unicode support of Parrot got better and faster I think it might be more likely to tied for 2nd.

For it to beat, or even meet MoarVM for first; it would end up needing so many changes that I'm not sure that it would even resemble what it once was.

---

I would have liked for Parrot to survive and be one of the backends that Rakudo runs on.

But dropping it was the correct move.

[chromatic]

Rakudo on Parrot was slow. If you happened to use a Unicode character in your source file it got dog-slow.

Worse yet, Unicode was an optional feature of Parrot! WTF!!

This is nonsense, and I say that as the person who spent months profiling Rakudo to figure out why it was slow.

Rakudo's parser was (maybe still is) slow because it can't optimize anything, even the <ws> token.

Adding NFG could have helped by allowing fixed-width access to normalized codepoints, but IIRC Patrick told the person who was going to implement NFG not to do it.

Seems like a pattern.

[b2gills]

You might be right about why Unicode was slow.

In which case I was right.

> So much so that to properly support Rakudo on Parrot would require a rewrite of [the middleware].

Which also means that most of the work you would have done to Parrot would needed to be reworked again afterwards.

There was `ifdefs` all over the NQP and Rakudo codebases to work around Parrot's differences. Which was annoying and error-prone.

The `ifdefs` are now mostly in NQP. And even those tend to be constrained a bit.

---

> Rakudo's parser was (maybe still is) slow because it can't optimize anything, even the <ws> token.

That is factually incorrect. There are several known optimizations that have not been implemented yet. One of which was even in STD.

Also since Raku treats regexes as code, optimizations to regular code paths can also apply to regexes. That includes optimizations to calling methods like using the <ws> token.

The main reason that they haven't been many is that the people that are competent enough and confident enough to do that work have been busy doing other things. Both their dayjobs and other optimizations or design work.

Really as far as I know, there have been next to no attempts to optimize specifically regexes and grammars since they first got to the current feature set. Certainly not in the several years where I was looking over every commit to Rakudo.

[b2gills]

The thing that P6 really needed was a new object system.

There were two options.

1. Create a new object system for Parrot. 2. Create a new object system and a VM around it.

Considering the object system needed is almost nothing like the one that was in Parrot, creating a new VM was probably considerably less work.

Then there is also the problem that some of the major contributors to Parrot didn't care about P6. They have said so publically. I don't think they would have taken too kindly to a completely new object system dumped on their doorsteps. (That's how they might feel about it anyway.)

---

Honestly thinking about it now, I think that if option 1 had been taken there would have been a fork of Parrot. I can almost guarantee it. And I'm not just talking about a fork of the codebase. There would have been a fork of the people behind it as well. It wouldn't have been a peaceful fork either.

I see all of the anger that you have towards what happened. I can only see it as being significantly worse and bigger if they had created a new object system for Parrot instead.

Creating a new object system, and a new VM to support it, was the correct move.

Especially considering that a number of the fundamental design decisions of Parrot which were completely unnecessary or plainly wrong.

I can only imagine the hell it would be to remove or replace any of the features.

---

You seem to be the only one still angry, still here, and still talking about it. Most of the other people who would have been angry regardless of which of those two decisions were made have either gone, or are quiet.

If the Parrot object system was replaced instead, a number of those people would still be angry, and still be here, and be loud. They would just be angry for different reasons.

Instead what happened is they got angry, but mostly drifted away with Parrot.

Come to think about it; things seemed to have gotten quieter after that event. Even though progress started going faster.

---

I would have preferred if they (you) didn't get angry about it. I would have very much preferred if there wasn't a reason to get angry.

The thing is that there would be some number of people would have gotten angry regardless. It's only a matter of who, and for what.

In the reality where Parrot got the new object system instead, I strongly suspect that one of the people who got angry would include jnthn. Since he is responsible for a lot of the progress in recent years, I think I prefer this reality.

I mean I miss the people from back then, but I think problems were piling up, and it was coming to a head at some point.