[mikelehen]

Hey Saurik,

Thanks for the thorough and correct analysis as usual. :-)

The key things I would point out are that:

1) The checkpointing is an optimization. You could either remove it (which will hurt initial load time) or delegate it to trusted server code (which will be very lightweight; you could run hundreds of rooms off of a tiny EC2 instance or whatever).

2) In general, the whole point of collaborative editing is that you trust your collaborators. If they're malicious, they can already cause mayhem on your editing experience with constant edits, obscene content, etc.

[saurik]

1) I do not need to modify the snapshots: I can simply inject corrupt history state. The problem is that the server in these kinds of systems is normally supposed to be running the OT algorithm in order to verify that the data being uploaded and stored as part of the permanent document record is valid.

(edit: That said, you would be hard-pressed to do this kind of OT-based text editor without the snapshots, especially with the very large number of separate objects being used to store the history state. While looking into how you were storing the data for this in Firebase, I had tried resetting the snapshot for a document to A0=[''], and attempting to open the document then bogged down so far that I wasn't certain if it would even recover; this problem will just get worse as the document ages... that only had a few hours of history behind it: a real document would just be screwed.)

2) There is a difference between trusting your collaborators with your data, and trusting your collaborators with your program state. Yes: if I am collaborating with people using Google Docs, the other people can ctrl-a+del all of the "data". However, they shouldn't be able to break the editor itself :(.

(edit:) As an example of this, if you remove the snapshots from the mechanism, then you can make the argument that "well, if I validate and ignore all history state that is invalid, this isn't a problem: I just need to keep the clients in sync and skipping things that are broken is valid" (so I'm happily willing to cede that my having added "whether or not it uses snapshots" was going too far). I personally think that this is still a problem, as the document record is still corrupt.

However, with the snapshots certainly, it isn't that I'm able to delete the data: it is that I'm able to break the synchronization system itself. I can setup situations where one party thinks they are editing the document, but their edits are being discarded. I can make it so that one person sees a document different than other people. In addition to doing all of this, I can make it nearly impossible to figure out who's doing it and to fix the situation. This is simply not the same problem as "well, you can always just ctrl-a+del the data from the document".

[mikelehen]

1) The client ignores invalid history items (unless there's a bug). So while you can pollute the Firebase data if you desire, it shouldn't affect the behavior of the app in any way. (i.e. Other than the checkpointing thing you brought up, you can't corrupt the history.)

That said, Firebase is certainly pushing the envelope in terms of what you would normally do with client-only code. =] And with that comes some challenges. In some ways Firebase is more like a peer-to-peer system than a traditional client-server system (since the Firebase server isn't doing complete data validation / processing). This sometimes affects the way you write code (doing extra validation / sanitization on the client-side for instance), but I think the advantages that come with Firebase outweigh that by far.

[saurik]

Well, the alternative is something like what many of your competitors (such as Parse) ended up deploying for handling these kinds of security situations, which allows you to write "real code" that runs in the cloud as part of the verification process: if you could run the OT verification algorithm on Firebase's servers, it allows you to avoid the problem of being unable to store trusted snapshots, but continues to offer the advantages of having someone else manage the complexity of operating the server and handling synchronizing the data. In such a case, the server could automatically generate the snapshots as part of a hook that would occur when the data is stored to the history buffer.

In this particular case, yes: you can drop the snapshots entirely, and have the clients download and replay the entire history state in order to synchronize as they open the document. That really isn't practical, though, and with your current implementation it is actually painfully slow to the point of being intolerable (although of course, you would then spend more time optimizing that path). I continue to not be convinced that you can implement a collaborative text editor that can be deployed in the myriad circumstances that Firepad both seems targeted at and that other HN users are commenting on with interest, and have it not have this problem of "users can break the synchronization".

[mikelehen]

[I think HN is throttling us; I had to wait a while before a reply button appeared. Feel free to email me (michael at firebase) if you want to continue the conversation.]

If the standard mitigation strategies (adding authentication, banning malicious users, etc.) aren't enough, and you're worried about people breaking the synchronization, I agree you'd need to move the checkpointing logic to node.js server code. Sounds like a good example app for me to write when I've caught up on sleep and have some free time. :-)

We're also looking to do a security v2 in the future to expand on our existing security rule capabilities and we've discussed going the "real code" route or else allowing tighter integration with your own server-side node.js/firebase code.

[saurik]

(You can just click "link" and get immediate access to a "reply" button.) As soon as I'm setting up my own servers and having to make certain they are secure, available, and scaling with the number of documents I have, I'm losing a lot of the advantages of using Firebase ;P. In comparison, with a model like Parse's, I can just push the code to them and have everything be handled without me having to get get my hands dirty. (Also, I'm currently 12 days behind on e-mail, but you guys can get ahold of me using other routes if you want or need to; at least Andrew should know how to get me quickly. I'm more just responding to the things you say here at this point, though: I have nothing new to add.) Great to hear that you may add "run real code on the server"!