[dgreensp]

Having been a dev on EtherPad, Google Wave, Coda, and other real-time collaborative apps with OT, undo, and so on...

I think it's correct that, ideally, there would be a framework that handles real-time collaboration, undo/redo, and offline support for you, and then you build your app with these problems already solved. I will probably create such a framework eventually. I don't see it as a database engineering problem, it's more like a framework or application architecture, which every app like Google Docs or Figma has its own version of. Writing such a framework is not too much harder than writing such an app, it just requires a little more abstraction and some documentation.

If you've never written an undo manager, sync engine, etc., and you aren't writing a complex app, it's hard to arrive at the right design by thinking about pure data sync. Also, storing and querying data are solved problems; it's more a question of coming up with a generic data model for an application and defining its semantics.

[YousefED]

I’m exploring the ideas (an easy to use framework to build local-first [1] apps) in my library Reactive-CRDT (https://github.com/yousefed/reactive-crdt). Feedback welcome!

All credit for the underlying tech to YJS, which has been amazing as mentioned by others in this thread.

[1]: https://www.inkandswitch.com/local-first.html

[mathgladiator]

The common abstraction is a shared log. I'm boiling down my current side project ( http://www.adama-lang.org/ ) into reusable components.

I'm also looking at WebAssembly as the way of doing work ( http://www.adama-lang.org/blog/micro-monoliths ) versus generic operations.

Something interesting to consider is how important is offline use these days? If we get to an online-99.9% world, then the solution feels ... simple.

[ithkuil]

Not long ago for work I was forced to use office365. It has collaborative editing feature that at the surface looks equivalent to Google docs, but in practice it's not: it wasn't reliable, I lost some edits a few times and even when it worked it was slow and sluggish (I knew that because I could see another person's shared screen in zoom), and it actually co-editing a file definitely felt a different experience.

The relevant bit your comment is: while most people indeed testified that the experience was not as good as Google docs, it wasn't that terrible to most of the people.

Turns out that it really depends how well you're connected.

Being connected 99.9% sucks more if that 0.1% is a crappy connection that drops packets occasionally as opposed to just saying "I'm online most of the time, I don't care if I need to edit the doc that rare 0.1% of the time when my connection is broken for good)

[cyanydeez]

real time collab is a figment of coders. real business processes are async in nature. what people really want is a easy way to open someone elses work, review it or add to it, and move on without all the emailing, chatting, while important, tend to be wlso a aync need thats note mvp to the process

[ithkuil]

Yeah whatever, I'm not a product guy and I do not pretend to have any clue about what people really want.

My point of my anecdote was that "connected" is not a boolean property.

[BiteCode_dev]

Offline first is a majors selection criteria for me.

First, it's just faster. Offline apps are an order of magnitude faster and the speed is stable.

Secondly, data is way easier to backup. I don't trust any provider with important data. Your account can be banned, or you could be locked out. There are outage, data corruption, human errors from people you share data with...

Then there is the no internet part. Internet at home can go off for many reasons, you may be abroad, in a plane, in a tunnel, in the country side. Or internet could be slow, unreliable, behind an agressive proxy, etc.

That's why i use thunderbird and not a web mail, dynalist and not the competition, mp3 and not spotify... That's why i keep osmand next to waze and torrents next to netflix.

Last week internet went down in accounting. They took the day off, because they used office 365. Great for them, not so much for the work to be done.

[aabbcc1241]

But you'd want to allow the client query a snapshot before doing a full sync on the shared log.

[mathgladiator]

Absolutely, this is a key reason why I strongly believe what goes in the log matters. If the log is a list of commands, then you have two representations to contend with: the differential/patch form and then the aggregated state.

An area that I'm playing around with is a log reducer which transforms a region within a log into a single item. For instance, if the log entries are just JSON (without arrays) then json merge (RFC 7396) is an example reducer. See http://www.adama-lang.org/blog/json-on-the-brain for more detail.

[aabbcc1241]

Interesting approach to reduce or even eliminate the workload of left folding the logs

[austincheney]

I have been thinking about these concepts in a person app. The primary point of collaboration in this personal app is the file system in real time. The framework I settled on for this problem is a security model.

In this app there are two types of agents: personal devices and users. Collaboration between devices is unrestricted because they are only physically separated segments of the same single user. Collaboration between users requires a share which is a segment of availability. By default a share is read only. Device identity is not exposed to other users so a separate user will have no idea they are collaborating across different computers if they are owned by the same user.

I have not written undo for the file system yet though. A requested feature is file/directory synchronization and I am considering adding desktop/camera video sharing.

[pumanoir]

There are a few of those frameworks. The first one that comes to mind is YJS here is a getting started[1]. 1. https://www.tag1consulting.com/blog/deep-dive-real-time-coll...

[sambroner]

YJS (and Kevin Jahns!) is fantastic, although I'm not sure they're interested in addressing all of these problems. In so far YJS has been very focused on replicating state from one place to another.

And the parent comment may be making light of some of the big issues in the space. You need to be generic, but also performant and simple compared to the competition.

Real time text editing is not trivial although there's solid prior art now. Even undo management is a whole problem space (what should a cross user undo look like if there have been dependent changes).

[kevinjahns]

Yjs is exactly that. It is a simple abstraction for building any kind of collaborative application. It has ready to use solutions for most problems related to this problem space. The selective UndoManager, for example, is generic&configurable and can be reused for all kinds of stuff. It supports many different editors. It supports many different (scalable) backends.

It is much more than state synchronization.

If there is anything missing, then let's work on it. Yjs is extensible and allows for custom features that others can reuse.

[kybernetikos]

Do you know if any of these frameworks support different users being permissioned to see different documents or different views of collections.

[sambroner]

Fluid Framework (a project I contribute to/work on through Microsoft) supports this via token in our default service implementation, but we don't have integration with an ACL DB.

[kwerk]

Where would one go to follow along if you start building this framework? You’ve got one pending GH star from me :)

[sambroner]

You could start by watching the Fluid Framework github.com/microsoft/fluidframework (I'm a dev on the team).

We solve a lot of these problems. This whole article speaks to the longer term vision. Even the article's Title is effectively our internal pitch: a web first database... that enables very low latency collab

[stopachka]

Brought a smile to read this comment. Will look deeper on fluid framework — rooting for ya’ll!

[BiteCode_dev]

+1. I would even pay to watch you code this.

[gurjeet]

Didn’t know there was a market for coding session watchers. Let me know if someone’s interested in watch me slow-code my current long running side project: Focusly.

[TeMPOraL]

> Didn’t know there was a market for coding session watchers.

I think this is an actual niche on Twitch.

[hanspagel]

You might like what I’m working on. A Node.js backend powered by Y.js.

https://www.hocuspocus.dev

Send me an email to [e-mail] if you want to take a peek.

[chirau]

I definitely want to check it out. Also, I love the simplicity of your site. Did you use a template or design it from scratch? I have a project that I need a page for and would want something similar to yours. Happy to pay for the work too.

[Santosh83]

Appears to be compiled using Gridsome, from looking briefly at source (CTRL+U).

[alfonsodev]

something like Microsoft's Fluid Framework ?

[unknown_error]

Firebase?

[stopachka]

Very excited to hear this!