[vidarh]

I built a webmail system on top of Qmail back in the day, and I loved it. We rewrote component by component as our needs changed, but the beauty of Qmail was that we could rewrite component by component by sticking to very simple contracts between them and/or start with the Qmail components themselves that for the most part are extremely simple.

Our web frontend for example worked on top of a slightly modified Qmail POP3 server that relied on encoding some message state in the filename, so we need only scan the Maildir without opening the files to be able to e.g. get message read state, and various flags. We also added caching of metadata etc. The changes were tiny and self-contained, and added one extra non-standard command to the pop3 server to retrieve a message list with much more data. The design of Qmail let us start with just changing the pop3 server, and later some tweaks to local delivery, and know we could test those programs in isolation.

The flexibility of Qmail even got us to use it as a messaging middleware later coupled with tinydns - all the routing and retry logic made it very convenient and extremely simple to troubleshoot.

[geocar]

That's great. Do you still have details of it?

I did a webmail "based on qmail" as well. It's still running[1] if you're curious.

[1]: http://demo.internetconnection.net/netmail/

[vidarh]

I have an ancient "backup" sitting somewhere. It was a provider called Nameplanet which provided vanity addresses (lastname.[assorted TLDs] for example), and morphed into Global Name Registry when we launched the .name TLD... The webmail system itself was sold to NetIdentity in 2001 or 2002...

It had a few interesting details - we ran it on ReiserFS for the fast/efficient small file support (at the time it really stood out) which made it great for Maildir's.

We also eventually used a small daemon to poll backends for which server a user belonged to, which had a mechanism to let us mark a user as "busy" so that we could balance accounts between backends by marking it as "busy" on both servers, sync the files over, and then mark it as available again without triggering errors anywhere. qmail on our MX's was modified to look up the right server that way.

The biggest changes were the POP modifications I mentioned. The ones I remember off the top of my head were:

* We modified qmail-local and the pop server to append size changes (from writing a new message or deleting one) to a file used to manage quotas. We appended rather than rewrite because it reduced the need for file locking (we took care to do single writes). We'd lock and coalesce the changes when the file got over a certain size.

* qmail-local was also changed to append the message size, and read-status to the filename. That let us avoid stat() calls for the files for the filesize, and opening and reading the files for unread counts etc. It was one of the first optimizations we did.

* Then we added a cache file that contained subject, sender, size, attachment status etc., for the web frontend, which would be dynamically re-generated automatically as needed.

* We made "+[something]" sort directly into folder "something".

* When we sold it I was most of the way through adding Sieve support to our qmail-local replacement.

These changes were quite small, and each successive changes lowered IO load dramatically (we handled about 2 million accounts before it was sold). Today, we could probably handle the IO load and storage we had with a single NVMe card...

The web frontend would try to use our extended POP3 command, and then fall back to scan the messages (and store a cache locally on the frontend) if it wasn't available, so we could use it as a POP3 client for other backends too. (The frontend is a story in itself - C++ CGI statically linked to shorted load time (it made a big difference at the time) and with "delete" only for really large allocations, to avoid wasting time on deallocation since we knew each process would at most live for a few seconds.

[geocar]

Neat!

It's a shame that more people don't develop software with this kind of... change-for-purpose.