[fro0116]

I had a pretty positive view of Matrix until I came across this post and the follow up responses from the developer behind an unofficial Matrix server implementation: https://news.ycombinator.com/item?id=19365968

I'd say tread carefully given their apparent hostility towards competing server implementations, which is literally the only thing that makes the protocol meaningfully "federated" to begin with.

This part in particular was deeply disturbing:

> I can quote the CEO of new vector in an argument we had about the insecurities of the protocol and what needs to be done to fix them where he said "good luck talking to your own federation." That reveals a lot.

Not exactly the kind of attitude I'd like to see from the stewards of an "open" protocol.

[Pneumaticat]

Matrix isn't really hostile to competing server implementations. If it were, then they wouldn't have spent a long time and a lot of effort to finalize the Server-to-Server spec r0.1, which anyone can implement [1].

And just to add my personal anecdote, Matthew (CEO of New Vector, co-founder of Matrix.org) is present quite often in the Matrix HQ chat room, and willing to field questions from anyone. Perhaps with slightly less willingness when people start such accusations.

Also, to address some of the technical issues, I think what they're talking about as "The blocks have hashes but their implementation does not check the hash" in the DAG is being fixed with the next (v3) room version [2]. Again, this development is all being done in public: see e.g. https://github.com/matrix-org/matrix-doc/pull/1659 for the v3 room proposal.

[1]: https://matrix.org/blog/2019/02/04/matrix-at-fosdem-2019/ [2]: https://matrix.org/docs/spec/rooms/v3.html

[fro0116]

To quote one of the follow up comments from the author:

> If you go to matrix.org and look at the list of about a dozen or so servers: you will find that none of them actually work except the reference implementation, and maybe sometimes Construct. Even thus, the phrase "able to build" is questionable. I have spent months reverse-engineering their software and its interactions before, and after, it was at all documented in this so-called standard (by the way, it's just documentation of their software -- errata and all (and rather poor)).

> Construct server is the single survivor out of the ones listed and even more who have attempted and given up early which we don't know about. That being said, it is still incomplete.

A spec that "Anyone can implement" doesn't have much value if it's so bad/incomplete that in practice nobody else is actually able to create a complete working implementation of it as the post claims.

And regardless of what they're planning on doing about any security issue, there's never a good reason for a discussion of a potential security issue in an "open" protocol to end in "good luck talking to your own federation." That's the behavior I'm pointing to that's hostile to competing server implementations.

Of course, the quote could be taken out of context, or straight up made up. But until proven otherwise, I'm willing to give the benefit of the doubt to the small independent developer who seems to genuinely care enough about the openness of the protocol to build their own server implementation.

[Arathorn]

The "good luck talking to your own federation" comment is being taken out of context. I can't find it in my history (so I assume it's been paraphrased), plus from memory I think it was the final line in an argument over how some bit of the protocol should be implemented. After failing to convince the guy that our approach was better than his proposal, I gave up with "okay, you can always do it your way, but you'll only be speaking to yourself" or something similar. I'm not sure this is evidence of being hostile to alternative servers, but if anything an encouragement for people to experiment with variations on the spec if they think they can do better :|

See https://news.ycombinator.com/item?id=19418111 for a detailed response to these accusations.

[StudentStuff]

It'd be great if there were usable non-Riot clients that fully supported end to end cryptography. At this point Signal has better alternative clients in Pyre.chat and Signal-Weechat than Matrix has, as the existing alternative clients are almost all MVPs missing major features like e2e.

The lack of a simple, common library is a real sticking point. Each client & bot shouldn't have to implement the baseline network protocol and then build e2e atop it, rather the clients should be focused on what to do with the messages after receiving them, how to display them, etc.

I say this as a daily non-Matrix.org user of Riot, the functionality of the Riot ecosystem on its own is good, but I'd be hard pressed to port my own Signal bots to Matrix, let alone build a featureful client in a performant toolkit (eg: Not Electron) for Matrix today.

There isn't a viable ecosystem outside Riot/Synapse currently, which makes it feel like Matrix is just a self-hosted Slack/Hipchat solution with just Riot as usable clients. IMO there needs to be diversity in the Matrix ecosystem just like the Fediverse has with Mastodon, Pleroma, PeerTube, PixelFed, NextCloud, Rustodon, etc all inter-operating in a sane, functional manner.

[Arathorn]

So, other clients with fully working E2E encryption include: Seaglass, Weechat, Pattle and nheko (although it needs to implement attachments, but they is easy). The weechat and nheko implementations are independent implementations (ie not the same as the Riot ones).

We are starting work this week on closing the E2E gap for all the rest.

[jcgruenhage]

I don't think Pattle has e2ee support, from what I can see on my phone. While it is built on the Matrix Android SDK which has e2ee support, the app itself doesn't seem to use that support yet.

[mindB]

Oh neat. Is the plan to write a proxy sitting between client and server and decrypting/encrypting all events so any client can be used in e2ee rooms?

[feanaro]

There's weechat-matrix[1] which supports E2EE and the accompanying python library, nio[2].

[1]: https://github.com/poljar/weechat-matrix/

[2]: https://github.com/poljar/matrix-nio/

[iknowstuff]

Good idea. A library written in Rust and compiled to WASM for Riot would be a good fit.

[maxidorius]

You're right, we need to do better, and we have started doing so under a fork of the protocol that will respect its user's privacy: The Grid - Come talk to us at #thegrid:libremonde.org

[ADogWithPurpose]

(Full disclosure - I'm contracted by New Vector to work on their integration services, but the below experiences are about my open source work which is unfunded and unrelated to my day job)

I wanted to chime in as another developer in the server area. I've recently been working on a service that sits atop of an existing homeserver (like Synapse) to handle outgoing federation in .NET rather than Python (which has proven to give quite an nice performance boost). The project implements it's own federation and signing code.

So far I've had to do almost no special casing around Synapse and followed the new S2S spec without issues, which has all been done in the open. The special casing I did have to do was brought before the team, judged a Synapse bug and will be fixed in an upcoming update.

I don't have any evidence to point to whether the spec can be used to write a complete homeserver implementation, but everything so far has been done painlessly following the docs online. I believe that I could, with enough time.

The project is developed by myself and another, and we've experienced no hostility from the team when asking questions about the protocol or the above bug. Quite the contrary, they were quick to help and the project has been promoted by matrix.org every week in the news blog posts.

On the project itself, we hope one day to branch out from working on services that sit on top, to being a standalone homeserver (which should progress naturally. I don't foresee any spec related problems with doing this from reading the spec and chatting to folks in the community.

So I felt it would be worth sharing my experiences as another server developer, albeit one that hasn't gotten as far in their journey yet.

[mindB]

The server-to-server specification was only formally released on February 5 of this year. Before that, it was certainly necessary to do reverse-engineering of synapse in order to build a HS, but it no longer is. The lack of server implementations just over a month after the release of the spec isn't good evidence of a bad spec. I strongly disagree with Jason's (who has been openly hostile to most of the matrix community including me) characterization of the spec in its current form.

Disclosure: I like matrix and have hacked on a few matrix-related projects as a hobby activity (Jason would refer to this as buying into Matthew's (the CEO of New Vector, the company primarily driving matrix) personal cult). That's it.

[maxidorius]

As the author of mxhsd [1] and Gridepo [2], I can sadly confirm Jason's claims that it is STILL necessary to reverse engineer to talk to the existing federation.

Synapse doesn't follow the spec, which is a fact I'm experiencing every day. But more to that, there is no desire from its developers to ever comply with the spec, rather than changing the spec on a whim.

And that are the reasons why making a HS is not possible: endless moving target.

[1]: https://github.com/kamax-matrix/mxhsd [2]: https://gitlab.com/kamax-io/grid/gridepo

[Arathorn]

if anyone's curious about the unspecific complaints here, it transpires that the actual complaint is synapse had a bug where it some endpoints were called with a trailing slash when the spec said it shouldn't have one... it got fixed in https://github.com/matrix-org/synapse/pull/4840 and https://github.com/matrix-org/synapse/pull/4935.

[Pneumaticat]

At least from my perspective (I've never implemented the spec), I wouldn't call it just documentation [1]. Sure, there are likely some holes, but overall it seems like they've put a pretty good-faith effort into making it detailed enough for someone to implement.

I feel like the difficulty comes mainly from the breadth/size of the spec. Which is a major issue (the Matrix.org team's own alternative server, Dendrite, has been slow to progress as well), but ultimately I'm not sure what they can really do about that considering one of their key tenets is to contain all the functionality out-of-the-box, so to speak (no extensions like XMPP).

[1]: https://matrix.org/docs/spec/server_server/latest.html

[maxidorius]

I am the author of the now-frozen mxhsd project [1] and its resurected version, Gridepo [2]. I have contributed several times to the specification, reported issues for the past two years and still, to this day, I can confirm that it is not possible to implement a Matrix server with the current specification that can talk to the reference implementation.

Most fundamental problem: synapse doesn't sign the events like the spec tell us to do it, making every event invalid. Whatever you sent that is spec compliant will be refused by synapse.

So talking "synapse" is possible. Talking Matrix and be able to federate with the ecosystem is not. I wish people would not blindly believe what a for-profit is saying, especially one that is in full control of the said protocol, and start checking for themselves.

[1]: https://github.com/kamax-matrix/mxhsd [2]: https://gitlab.com/kamax-io/grid/gridepo

[Arathorn]

I'm not aware of a bug in how synapse signs events, and whilst it's possible there's a bug there, we can't fix stuff which we're unaware of which hasn't been disclosed. (Max would probably claim he can't disclose because he's been banned from our Github for consistently being obstructive; unfortunately this is a case in point).

Edit: he might be talking about https://github.com/matrix-org/synapse/issues/4787? Except that was fixed a month ago...

In terms of "don't blindly believe what a for-profit is saying, especially one that is in full control of the said protocol", I'd agree, were it true. However:

* These days the protocol is controlled entirely by the non-profit Matrix Foundation.

* The majority (3 out of 5) of the Foundation's directors have zero affiliation to New Vector, the company that the core Matrix set up to try to keep the project funded.

* The governance of the foundation & protocol is entirely open, as per https://github.com/matrix-org/matrix-doc/blob/matthew/msc177...

* We're announcing the final board once formally appointed; due in the next few weeks.

Meanwhile, it's also true to say that you shouldn't believe the skewed perspective of a disgruntled ex-community member with a chip on their shoulder, who is trying to plug their hostile fork: https://news.ycombinator.com/item?id=19473694.

[Perceptes]

I'm the original creator of Ruma, another homeserver implementation, and our project has received nothing but support and appreciation from the Matrix team. It's true that Synapse is the only homeserver that is in a "finished" state, but that has more to do with Matrix being young, the spec still needing work (although it's come a long way recently), and the lack of time and resources to complete such a big project if it's not your full time job.

[jeltz]

Is the Ruma project still alive? I liked your website and your goals but last I checked there was very little activity.

[Perceptes]

Yes, it's alive. We've been on a long hiatus while we wait for the dust to settle with async code in the Rust ecosystem. This is the GitHub issue that best tracks our ongoing blockers: https://github.com/ruma/ruma/issues/189

[maxidorius]

As the author of one frozen homeserver [1] and an hybrid implementation [2], with various work made all across the spec, I'm not shy to say synapse is the only homeserver because it talks its own dialect of the spec, not because there wasn't attempts to build something else.

[1]: https://github.com/kamax-matrix/mxhsd [2]: https://gitlab.com/kamax-io/grid/gridepo

[feanaro]

I'm one of the people in that thread and I'm unconvinced by Jason's insinuations regarding the hostility of the New Vector team. From everything I've seen, interoperability and federation is extremely important for New Vector. The fact that Matrix is a rather innovative and semi-research project has to be taken into account.

For instance, here[1] is the github thread on verifying event hashes which he refers to in that HN thread. From what I can see, he notes some of his (insightful!) thoughts on the matter, nothing but a civil discussion ensues, with parts of his suggestions getting implemented and others being worked on.

[1]: https://github.com/matrix-org/synapse/issues/1252

[maxidorius]

And here is three documents/issues that all related to synapse breaking systems in the federation, despite repeated extensive technical feedback.

And to this day, despite people that keep on reporting about the issue, nothing is done to actually solve the problem.

https://github.com/matrix-org/matrix-doc/issues/1194 https://github.com/matrix-org/synapse/issues/4540 https://github.com/matrix-org/matrix-doc/pull/1915

If interoperability and federation is important, why push changes that break those, and do nothing for >6 months?

[Arathorn]

fwiw, the linked MSC https://github.com/matrix-org/matrix-doc/pull/1915 solves the problem, is merged, and is implemented in https://github.com/matrix-org/synapse/pull/4982.

Yes, it would have been great to have done this sooner, but unfortunately fixing a bug which only impacts deactivating accounts when using custom identity servers took a while to get to the top of the todo list.

[ATsch]

I think it's easy for bystanders to mistake hostility towards the developer of construct as hostility towards the project or the specific implementation without knowing the full context, however this is not the case.

The developer is without doubt a brilliant individual, however also extremely toxic. Not in the Linus sense, but in the Terry A Davis sense.

If anything, I think the matrix team has been too reasonable with him. He actually agreed with that himself, making fun of the team for not having the integrity to ban him out of sympathy for his situation and wanting to avoid the optics of banning a developer of an alternative server.

I have a lot of sympathy him as it sounds like he has a very serious substance abuse issue. I really really hope that he can beat the problems he faces, and I'm sure he will be accepted by the community again if he does. I understand that he is not fully under control of his situation but until he is, I don't think he should be tolerated when his presence is so hurtful to his victims and the community as a whole.

Here's some fun highlights I've experience first hand for context:

- spamming rooms with BSDM porn

- spamming rooms with links to his implementation

- hurling sexist abuse at a female matrix team member

- hurling sexually abusive language and rape threats at people

- numerous other drug-fueled breakdowns and caps locks tirades

- sockpuppeting and circumventing bans

- attempting to blackmail the team with his exploits

- deliberately bricking rooms

[baroffoos]

I think the situation is not that matrix.org is hostile to outside developers but that they are working on something new and rapidly changing and other implementations aren't able to keep up until the protocol changes slow down and the docs get better.

[subway]

It seems like a few folks have been burned by the failure to put the spec before the implementation.

Another example: Kamax provides mxisd[1], the only federated identity server (the official identity server is/was controlled by vector, and doesnt openly federate). They had to give up on mxhsd[2], their home server implementation due to spec inadequacies, and ended up forking the spec into Grid[3]

[1] https://github.com/kamax-matrix/mxisd [2] https://github.com/kamax-matrix/mxhsd [3] https://gitlab.com/kamax-io/grid/gridepo

[Arathorn]

Matrix has always deliberately put implementation before formalising the spec, because there's no point in speccing things which haven't first been tested in the wild. This is also partially influenced by observing that many XMPP XEPs seem to have lacked a proven implementation at the point of being specced, contributing to the fragmentation of the standard.

Now, the reason that we didn't formalise the federation spec until very recently (Jan 2019 - https://matrix.org/blog/2019/02/04/matrix-at-fosdem-2019/) is because the implementation in the wild was showing problems which we wanted to resolve first. Now, this took ages to do, mainly because the team got mired in technical debt in Synapse and keeping the matrix.org server running, whilst trying to support the overall featureset (E2E etc) needed for Matrix to be successful. But we made it in the end. Meanwhile, it's true that server implementors got bitten by the lack of formal spec. The Ruma project for instance went on hiatus until a stable spec was released. Meanwhile unfortunately the mxhsd project lost patience and forked.

In an ideal world we'd have been able to move faster on releasing the stable federation spec, but the reality is that Matrix is a lot more than the federation API (although server implementors obviously focus particularly on it): we've put a lot of work into the CS API, E2E encryption, AS API, bridges, bots, reference clients etc too. History will tell if in the long term we got the prioritisation right.

Meanwhile, the federation API should now be sufficiently specced for server implementors to be able to successfully implement it - see https://matrix.org/docs/spec/server_server/r0.1.1.html if you're interested.

[maxidorius]

Still can't implement to this day.

[meditate]

I am not surprised at this and I am not sure why the Construct developer was either. Protocols like this during the initial growth phase have a ton of churn on the backend and because of that, reluctance to finalize any spec -- it's understood to be a moving standard. So presumably this person knew the risks.

[jancsika]

> Not exactly the kind of attitude I'd like to see from the stewards of an "open" protocol.

It depends on what exactly the complainant was proposing to change in the protocol.

There are no details in the link you provided about what was insecure about the protocol at that time, not to mention the proposed fixes. Is there another link for that?

> Not exactly the kind of attitude I'd like to see from the stewards of an "open" protocol.

Are you talking about the development process that led of the 1.0 protocol, or the spec itself?

[meruru]

I think you're letting some rando's accusations color your perception of the project too much.

[cetra3]

I would be interested in knowing whether anyone else has confirmed these accusations or created a writeup as to why the protocol is insecure.

[maxidorius]

It also has very questionable choices towards respect of Privacy: https://github.com/matrix-org/synapse/issues/4540

[maxidorius]

I can confirm. One of the write-ups I did: https://gist.github.com/maxidorius/b25769f1a89c8860b928babe7...

[Arathorn]

this write-up boils down to saying that servers that don't uphold server ACLs (server-bans) will leak messages from banned servers.

it's a feature, not a bug; if you want to deploy the nuclear option of a server-ban in a room, you also have to ban any other servers which don't know what a server-ban is.

[Arathorn]

I'm the project lead for Matrix (and CEO of New Vector, the company which hires most of the core Matrix team) and can try to clarify this.

1. We don't have any hostility to alternative server implementations; it would be utterly idiotic to sabotage the project by doing so. Instead, we promote them, even when they're written by people who for whatever reason have issues with the project. For instance, if you look at https://matrix.org/blog/category/general/this-week-in-matrix... you can see us publishing almost weekly updates on Construct (the server written by the guy who is levelling the accusations here). Meanwhile, Construct appears to work well enough to talk to the rest of Matrix in practice.

2. Yup, there have been some security issues pre-1.0 in Matrix around federation, particularly around state resets (thinkos in the state merge resolution algorithm), event ID collision, incorrectly trusting potentially malicious DAG depth parameters, and issues around the perspectives logic. As far as we're aware, these have all been fixed now, or will be once everyone has migrated from perspectives to real TLS, as per the original article - hence us making a big noise about it with AreWeReadyYet.com. Most of the gory details are at: https://github.com/matrix-org/matrix-doc/issues/1442, https://github.com/matrix-org/matrix-doc/pull/1659, https://github.com/matrix-org/matrix-doc/issues/1229 and https://github.com/matrix-org/matrix-doc/pull/1711 respectively. You can also see me talking through these issues one by one on the main stage at FOSDEM, starting around https://youtu.be/C2eE7rCUKlE?t=2035.

3. I can't remember the precise context where I said "good luck talking to your own federation", but I suspect it was the result of a disagreement over how federation should be designed - probably over whether DAG depth parameters should be calculated locally or proposed remotely and then validated. We chose one solution, there was a lengthy disagreement, my eventual response on giving up on the argument was "okay, if you want to do it the other way, good luck with that" or words to that effect.

For context, the guy levelling the accusations here is also responsible for maliciously exploiting the security issues on discovering them (e.g. https://matrix.org/blog/2018/06/14/security-update-synapse-0...). He is also banned from our github and the core-team chatrooms on Matrix after exhibiting pretty much every flavour of obnoxious and destructive behaviour, culminating with ad hominems against me and most of the individuals on the core team, illustrating his points with hardcore porn, and asking how we're going to compensate him for not launching further exploits. He's also filled up the network with sockpuppet accounts to spam his project (despite us, for better or worse, already promoting it on the weekly blog), and I'd assume he's also seeding sockpuppets on HN too.

So, TL;DR: whilst it's true that pre-1.0 we had some security issues around federation, we believe they are now fixed (or will be, once we've upgraded all the rooms to 1.0). Meanwhile, be aware that the complaints are coming from a deeply disingenuous and malicious source.

[fro0116]

Thank you for providing your side of the story. While I can no longer edit my post to point to this, I have upvoted this post to hopefully bring some visibility to it, and encourage others to do the same.

Until someone can provide a chat log of what was actually said in that conversation that led up to that comment, it's still going to be difficult for any outside observer to make up their own mind on whether or not those words were indeed taken out of context to the degree that you suggest. But I can appreciate that it could be frustrating to deal with someone who behaves as you claim, even if his intentions might be to keep the protocol open and secure.

[speedplane]

Matrix is a fantastic project. As the technology matures and becomes more useful, I hope there will be a large push to formalize the organization and bring in other stakeholders. As I'm sure you know, Matrix will not be able to get off the ground by itself, it'll need institutional buy-in from corporations and existing tech companies. Hope there is a plan for that type of out-reach.

[Arathorn]

Totally agreed. https://github.com/matrix-org/matrix-doc/blob/matthew/msc177... is the core of the plan.

[zinid]

> it'll need institutional buy-in from corporations and existing tech companies. Hope there is a plan for that type of out-reach.

"Failing to plan is planning to fail".

I don't see why would corporations and existing tech companies be interested in yet another IM protocol they cannot control, especially in the era of total silos and FAANGs. Doesn't sound like a solid plan to me at all.

[fro0116]

I can absolutely imagine why existing tech companies would be interested if the protocol became so popular that they'd be stupid to ignore it and start from scratch with their own protocol, like XMPP was at one point.

I'd be more interested in seeing how they plan on preventing the whole Embrace, Extend, Extinguish thing pretty much every company pulled with their initially XMPP based chat apps that gained market share, turning them into back into closed silos.

[Arathorn]

Preventing the embrace, extend, extinguish manoeuvre that WhatsApp, Facebook Messenger, Google Talk, even Apple Push Notifications did with XMPP is indeed a tough one.

The best solutions we have right now are:

* Ensure there's enough value in the wider network (e.g. available services, integrations, bridges, public chatrooms) that you'd be taking a massive step backwards not to federate.

* Try to build the protocol to be capable enough that vendors don't feel that they have to fork and close it in order to make it do what they want.

I think it's mainly the first one that will make the difference. If there hadn't been such great content out there on the public internet, we might still be on AOL & Compuserve today.

[ge0rg]

The second point will only address the vendors that are actually interested in federation, not the ones that want to close down their silo. XMPP fifteen years ago is a prime example.

Regarding the first point, I'm not sure it will be ever possible to pull this off. The proprietary IM silos are many orders of magnitude larger than IRC, XMPP and Matrix taken together. You are doing a good job with promoting Matrix to nerds, and there will be _some_ value in being able to directly contact the French government (provided that they will allow federation from the public network, which I have a hard time imagining).

I think the only viable route today is to push for legislation (e.g. in the EU, in the context of ePrivacy / GDPR) that will force silo providers to open up their silos and to offer interop by means of standardized protocols. But even in this improbable case they will probably rather create their own rubber-stamped standards (remember Office Open XML) than follow what is already out there.

[techntoke]

I find it strange to suggest that he is using sockpuppet accounts on HN without any proof. You may as well be using sockpuppet accounts to promote Matrix. It is also really suspicious to attack someone that has spent a significant amount of time on your project.

[Arathorn]

Here is a typical sockpuppet: https://news.ycombinator.com/item?id=19081596 (which matches the @senora_acero:tedomum.net puppet which was being used on Matrix around the same time)

We do not use sockpuppets to promote Matrix, and I don't see the link between pointing out someone who does as somehow meaning "we may as well be using sockpuppet accounts too"; it's a false equivalence if ever there was one. :|

And I'm not sure that calling out the unpleasant behaviour here is 'attacking someone who has spent a significant amount of time on the project' - but thankfully enough other people have independently pointed out the reality of the situation on this thread (e.g. https://news.ycombinator.com/item?id=19422064).

[hestefisk]

Thanks for building Matrix. It’s a great product / standard / protocol

[gnomewascool]

Thank you for continuing your FOSS work despite the attacks and for taking the high ground / mostly maintaining civility from your side (even keeping up with updates to Construct)!

[mquander]

Thanks for working on Matrix!

[EvangelicalPig]

Damaging if true :/