[robmueller]

It's not just WebDAV that's an abomination (returning HTTP status codes and messages in XML, WTF were they thinking), but vCard and iCal as well.

The vCard "group construct" (see rfc6350) is one of the dumbest things ever added to a spec. It seems a trivial thing to add, but completely screws up your internal storage and manipulation formats. It's horrible, on top of all the other horribleness that is vCard.

Of course, I'm biased, given we're trying to push alternatives to IMAP, CardDAV and CalDAV - http://jmap.io/ - but please do check it out. Having actually worked on IMAP servers and clients, CalDAV servers and clients and CardDAV servers and clients, we've learned a lot about creating a saner alternative.

[untitaker_]

>returning HTTP status codes and messages in XML, WTF were they thinking

You're giving me so many ideas right now for a follow-up post.

>but vCard and iCal as well

The group construct is bad, the syntax is bad, but other than that, I never had to implement or parse it properly. Vdirsyncer is mostly passing strings through.

I suspect that this part is much harder to replace than CardDAV and CalDAV, since it's not just a file structure with a ton of bullcrap on top of it. For that reason I currently have those file formats in the remoteStorage folder structure i'm syncing to, but it's still a step forward. I might switch to jCard and jCal, but I don't think it'd be worth the breakage.

[untitaker_]

So one of your guys just pinged me in private about this, and I responded lengthily. Here's the basic summary:

* I didn't know JMAP was also about calendar and contacts.

* Those parts seem nice.

* However, I'm the kinda guy who self-hosts his data servers. JMAP seems like a powerful protocol. I'm not sure those two things mix well, since JMAP server implementations probably will require a good database, not just a dumb FS like in remoteStorage.

* However, I would like to collaborate wherever possible.

[robn_fastmail]

> JMAP server implementations probably will require a good database, not just a dumb FS like in remoteStorage.

To do JMAP well you need to be able to calculate change sets, which does mean a database, though a fairly light one. I think you could do a server without delta updates by always returning a cannotCalculateChanges error in response to getUpdates calls, but it would be very inefficient on both the client and the wire.

> However, I would like to collaborate wherever possible.

Sure! Best place to start is probably the jmap-discuss list:

  https://groups.google.com/forum/#!forum/jmap-discuss

[hobarrera]

> For anybody who wants to be "careful" when syncing and thinks they want this option, consider putting your collections into git repos and commit after sync.

There's a demo/beta DAV<->JMAP proxy out there by fastmail. You might want to look into that (and keep the current DAV backend in the meantime).

[scrollaway]

Can you talk a bit more about jmap? It looks very sensible. How are your efforts going? Any uptake?

[binbasti]

I didn't notice that JMAP includes calendars and contacts. Looks good!

Personally, I'd prefer a dumb storage with a generic API over yet another custom protocol/API just for these data formats alone. Worked great on people's local drives before the Web, and I think it should work similar on the Web.

[robn_fastmail]

Dumb storage is fine if you only have a single actor that can maintain a copy of the state at all times. As soon as you've got multiple actors on the data (another client on your desktop or phone, or even something doing mail delivery), then you need a way to determine what changed on the server so you can update your state.

If the server has no ability to tell you what changed, then you're left with having to download and check everything. On a large data set, that's pretty much impossible to do quickly and without a lot of network traffic.

Of course, no server is actually that dumb - even a file listing with file sizes can get you part of the way there. But if you've got 10000 files in a file store, that list can still get pretty heavy. If you're willing to make the server smarter, eventually you can get to the point where the server can give you only what changed since the last time you checked. JMAP isn't unique in this; IMAP has MODSEQs, *DAV has collection synchronisation, etc.

JMAP specifically doesn't really care much about the actual format of the data it works with. The only thing it really needs is an immutable ID, so you could use the same model to store all sorts of things (and at FastMail we do, with things like client settings).

[binbasti]

> Of course, no server is actually that dumb - even a file listing with file sizes can get you part of the way there. But if you've got 10000 files in a file store, that list can still get pretty heavy.

Exactly. remoteStorage has ETAGs in folder listings for that. The point is that you can implement a folder structure that enables you to update say just the last week of events plus upcoming ones, which is usually nowhere near 10000. Except with CalDAV you can't (according to the article, I haven't actually looked into it myself).

[untitaker_]

I guess you can dump a [Maildir](http://cr.yp.to/proto/maildir.html) (or at least a modified version) into a remoteStorage, but this is not really an acceptable protocol for mobile clients with limited space. As mentioned in the blogpost I have the same problems with storing calendars in remoteStorage.

[treve]

Except that a dumb storage cannot do scheduling, and many other things.

[binbasti]

The storage can't do the scheduling itself, but apps can do it on top of dumb storages.

[untitaker_]

I guess that's an expendable feature.

[philsnow]

Hey robmueller, I think it bears moving this paragraph on proxy.jmap.io

    The service is not running with all of the security measures employed on a production site such as FastMail, so DO NOT USE THIS PROXY FOR ACCOUNTS WITH SENSITIVE DATA.

above the "Oauth to a gmail account or log in to an IMAP server below".

Of course it stands to reason that one shouldn't grant access to third parties to one's gmail, but you may save some heartache from people who are enthusiastic-past-the-point-of-reason if you move that bit up.

I would have just sent you a PR but proxy.jmap.io isn't hosted on github pages.

[robn_fastmail]

No, because it's an actual application. Edit this and submit a PR: https://github.com/jmapio/jmap-perl/blob/master/htdocs/index...

[qznc]

My pet peeve with vCard is the lack of an encoding and the resulting mess with various implementations. JMAP fixes this by enforcing utf8. You have my vote. :)

[untitaker_]

To be fair the newer versions of vcard also specify this, and I haven't had a single encoding issue with even the older versions. Every server just seems to send UTF-8, no questions asked.

[qznc]

So far all my phones had a different way to encode Unicode strings. When I wanted to merge the data, I tried a bunch of Python libraries and all failed differently. In the end I wrote my own one-off script.

[treve]

Both vCard 4 and jCard are strictly UTF-8 too.