[hackuser]

> Sending is easy. Receiving and sorting and filtering is harder.

I don't understand: Receiving messages on any of the transports is done every day and billions of platforms. After receiving, loading the messages into a common database, including mapping metadata such as sender, date, etc., seems relatively simple. Once in the database, sorting and filtering are simple.

What am I missing?

[zombition]

> Why, as a user, do I give %#@%! what the underlying transport is? > I don't understand: Receiving messages on any of the transports is done every day and billions of platforms. After receiving, loading the messages into a common database, including mapping metadata such as sender, date, etc., seems relatively simple. Once in the database, sorting and filtering are simple.

I agree with this completely--so much so that I decided to make my own communication protocol (and client-server implementation) that would do whatever I wanted and still be backwards compatible with email. You can see the results here: https://github.com/Zombition/nsmtp-monolith. So clearly it's not impossible.

> What am I missing?

The biggest difficulty isn't from pulling out metadata and mapping it to whatever you store locally. The specifications of different protocols determine the user behaviors that are possible with each protocol, and mapping behaviors between different protocols in a way that makes sense to end users is much more difficult if you want to retain quality communication with users who are not using your software to pull everything into one place. What I found from implementing a new protocol that aimed to be backwards compatible with SMTP was that maintaining backwards compatibility with SMTP guided 99% of my decisions for making the new protocol. If I wanted to add a feature, but it would be confusing for end users who might be trying to communicate with regular SMTP users (or the SMTP users that they were trying to communicate with), the only practical solution is to axe the feature or rehash it in a way that would be less confusing for communication between the two.

In practice this is quite difficult, because you have to balance awareness of both protocols with awareness of how data sent through one can be meaningfully translated to the other, and while you're considering those things, you also have to consider what the user experience is going to be like for people on both ends of the software. If you're trying to maintain compatibility with a widespread protocol like SMTP, there may be an enormous number of different client interfaces that can add a significant amount of variation to the user experience. It's a bit like unfolding a fractal.

[baudehlo]

Spam, generally.

Because email uniquely among communication protocols allows sending a message without any previous agreement from the recipient, will always be susceptible to spam (and getting rid of that feature would destroy emails benefits).

Plus people want to filter differently. People deal with their inboxes uniquely.

[hackuser]

Why can't the universal mail client I propose have a spam filter? More likely, my incoming mail server will have the spam filter; the client will have the spam folder. What does it matter if only SMTP messages end up in that folder?

> Plus people want to filter differently. People deal with their inboxes uniquely.

That issue affects every messaging client's filtering. How does it affect a client any differently if it combines, for example, SMTP and SMS?

[baudehlo]

Oh you absolutely can, but there are a mix of problems some of which makes spam filtering better on the server (knowing bad sources of email, like infected desktop machines), some of which makes filtering better on the client (like knowing your users' preferences for what they consider spam).

There's a huge long history of very smart people (not even including myself in that list - I've worked with some people MUCH smarter than myself on this) working on this problem, including the very creator of this site, pg himself, and even he doesn't consider this a solved problem. People are still paid well to combat spam every day on behalf of demanding customers (and I'm grateful I'm not one of them any more) and it will always require new techniques and new input. Hopefully you will contribute too.

[baudehlo]

Oh and in terms of the other question: how many people get enough SMS messages that they want to filter them? Filtering is uniquely suited to a non realtime messaging system. Like email. I don't want to filter Slack/IRC messages either. I just ignore what I didn't read.