The future of email storage

In the early days of the internet, there were only few servers. When users wanted to exchange messages, they would log on to their server via a terminal physically connected to the server. Then they would send their mail. If a message is directed to someone on another server, the sender’s server would then take care of transferring the contents of the mail to the other server (in plain-text, over SMTP), where it would get sorted into the recipients’ mailbox.

As personal computers started to become more widespread, people wanted to also send mail from their PCs. This split the process into three parts;

1. get the message from the sender’s PC to the sender’s mail server
2. get the message from the senders’ mail server to the recipient’s mail server
3. get the message from the recipient’s mail server to the recipient’s PC

Steps one and three usually happened in internal organization or company networks, while the mail servers of two organizations talked over the internet.

The internal communication usually consisted of authenticating the user, so you could not send mail as somebody else or read others’ received mail.

The communication between the servers wasn’t secure at all - let’s hope the server at mail.recipientcompany.com is really who he claims to be.

Skip a few years.

A market research study reports 3.3 billion active email accounts for 2012. Google, Microsoft and Yahoo alone account for over one billion of them. With the decentralized nature of the email system and the simplicity of the involved network protocols, how did it come to that?

Since the authenticity of mails couldn’t always be guaranteed, and you really never knew who you were actually talking to, mail servers began to ‘distrust’ each other. A strange and convoluted combination of spam-flagging, transparent or silent mail rejection, grey-listing and reputation scoring ensued. Message-signing systems such as DomainKeys Identified Messages (DKIM) have been developed to provide sending mail servers methods to prove the integrity of their mails, but sadly, they gain no widesprea use.

If you were one of the big sender companies already, you’re in luck: you’re so trustable, every mail sent gets through. But if you were not, you’ll have to beg, tinker, and pray to get your mail into the recipient’s inbox, even though you are just peacefully sending a few messages from your own domain. The whole sending process becomes so painful, application developers often decide to just setup a Google account for the app and let all sent mail go through Google’s servers in order to get the trusty “Don’t be evil” stamp.

Actually, I recently tried to send email ‘the old-fashioned way’ from my home cable connection - by using netcat to connect to the receiving SMTP server, acting on behalf of a DynDNS domain I set up. And it really is like that: nobody will accept your messages. Either with statements guiding to you policies (which then basically say “you’re screwed”) or by just silently dropping your messages after an unsuspicious 250 OK SMTP reply.

Okay, great, let’s stick to the trusted mail senders, then. But, you say, what about my privacy? In 1991, Phil Zimmermann wrote Pretty Good Privacy, known as PGP, providing paranoia-homed cypherpunks (yes, I’m playing devil’s advocate here) with the ability to encrypt their communication. While mathematically perfect, the system lacked another core ingredient: a good user experience. Before widespread client-side adoption, you had to manually copy the Base64-encoded blob of text out of your mail client and decrypt it before being able to see the message, and in any case, you still have to have your ultimately-secret private key on hand all the time every time you want to read your mail.

Let’s head back to 2012. We are here today with our iPhones and iPads and S3s and Laptops and whatever 3G-enabled devices (or glasses, or fridges) are going to come up during the next few years, and we still want to read our mail. It’s so easy, they say, just route every single of your mails through our system, then you can easily read it on every web-enabled device. It just works! Spotted the catch yet?

Combining the ease of use offered by cloud-based systems with strong cryptography doesn’t work right now. Reading PGP-encrypted mail on mobile devices, if possible at all, is a painful experience. Yes, security always comes at the price of convenience, but email has gained such utmost importance for almost everyone that it’s ridiculous to argue that the present state is quite good enough in terms of either security or convenience.

So, what’s the point?

From a high-level view, this is what I would expect from a modern email infrastructure:

• worldwide 1-click access to email from authorized mobile devices
• no storage of private keys within those devices - their loss would be catastrophic
• perfect encryption from sender’s keyboard to receiver’s eyes
• which also leads to: no single company should have access to all of my messages
• “batteries included” set-up

Technical aspects would include full backwards compatibility with existing email services, registration of authorized devices (and revocation in case of loss), automatic encrypted backups, and near-real-time performance. By the way, the only exception regarding the No-Dynamic-IPs policy is Google. They have really gotten it right. If you try to send your mail 70s-style like I did above, without any authentication or signing, it will arrive but be flagged as spam. If you sign your message using SPF and DKIM (which is really doable - it took me about four seconds to manually sign a message using the Ruby DKIM gem), it will actually arrive as a legit message in the recipient’s Gmail account.

Current email providers all fail at least some of these requirements, as do old-fashioned PGP-based write-your-key-fingerprint-on-a-napkin encryption models. There needs to be a secure location to store the email data, serving it to authorized clients over the net, all using strong cryptography on trusted hardware in a trusted environment.

In my vision of the email future, the cloud’s coming home. Literally.