[KishanBagaria]

Apple's chat.db has an esoteric schema owing to the fact they never designed it from the ground up and instead kept adding new columns and tables with each macOS release. This makes their queries super complicated with multiple joins.

Once you have the schema figured out, it's dead easy to build a third-party client that works better than the official one. Even search works great with a simple LIKE query but Apple re-indexes all messages leading to your CPU going over 1000%: https://twitter.com/KrauseFx/status/1396433852126670852

Source: I built a third-party desktop client for iMessage at https://texts.com and reverse engineered the complete sqlite structure.

[m463]

> "Apple's chat.db has an esoteric schema"

This is basically one taken out of microsoft's .doc format playbook.

At first I thought Apple was slow and just didn't have time to get to some sort of export for iMessage chats. Mail had imap, calendar had caldav, address book had carddav. The original mac stuff did other standards like AIM and jabber.

But over time I began to suspect, then later those internal emails exposed in the lawsuits made it clear - apple was holding everyone's messages hostage.

What's annoying is that I couldn't save my messages, I could only upgrade to a bigger iphone.

What's downright evil is all the people - I know several - who have lost messages + embedded photos/videos from loved ones who have died because there was no export.

[coldtea]

>But over time I began to suspect, then later those internal emails exposed in the lawsuits made it clear - apple was holding everyone's messages hostage.

In an SQLiteDB any programmer can learn to navigate, and any third party can build a tool to export them?

Sorry but the "internal emails" prove nothing about Messages. It's just a quick and dirty implementation that "works good enough" for reading the latest, but not as an archive.

[dan-robertson]

Yeah I agree. The lack of a good schema/export function sounds to me like negligence not maliciousness. It’s a different issue from only allowing Apple-tech-users to use iMessage

[dvfjsdhgfv]

> But over time I began to suspect, then later those internal emails exposed in the lawsuits made it clear - apple was holding everyone's messages hostage.

Any source for that? I couldn't find any direct evidence for your claim.

[m463]

There are probably better links:

https://texasnewsexpress.com/index.php/04162021-1249/apple-r...

[dvfjsdhgfv]

Thanks, that's interesting, I didn't know that. On the one hand, I shouldn't be surprised since vendor lock-in is the standard way tech companies try to gain corporate advantage. On the other hand, it's kind of depressing to learn once again that customers count very little and no matter what these companies say, they will always try to abuse us in every way the law allows.

[600frogs]

Just took a look at your product, I'm very impressed and have added myself to the waitlist. I'm almost even more impressed by the domain name itself though - how on earth did you manage to bag that domain?!

[KishanBagaria]

Bought it at the right time from the right person :)

[600frogs]

I presume the right time was 20-30 years ago?

[cozzyd]

domain first, then figured out the product?

[CRConrad]

Hmm.... Has Julian M. sold tele.com yet?

EDIT: No, apparently not. Weird; we all thought that was going to make him a multi-millionaire.

[breakfastduck]

That alone must be worth some serious money.

[controlledchaos]

GoDaddy estimates it at $25,000

[missedthecue]

I would not even raise an eyebrow if there were offers out there for 10x that.

[hibern8]

Easily worth 6 or 7 figures to the right person.

[foobarbazetc]

Those estimates mean nothing. GoDaddy would list it at 250k+.

[hibern8]

Christmas.com sold for over $3M. I would think texts.com would be worth a lot more than $250k.

[SavantIdiot]

I modified this script to pull my entire db into a text file.

https://github.com/michael-danello/iMessageWrapped

EDIT: It is absurd Apple doesn't provide better message organization and UX/UI.

[spicybright]

The fact so many people on this site can throw something functional together without much effort is even more disappointing apple doesn't do anything about it.

[alexanderdmitri]

iPhone is the only apple device I own but it crashes it can't go too far back into my messaging history for any one individual too far.

Usually once I scroll to ~5 months back with people I text regularly the iMessage app just straight crashes ... it's a little absurd. To top it off, every couple pages it gives a loading icon and introduces severe latency for whatever reason. There don't seem to be any apps in their App Store that let you search and lookup old messages either.

This is actively biting me right now. Are there any solutions for accessing the iMessage/chat.db database on my phone or via iCloud from a Linux system (Ubuntu in my case)?

[kevingrahl]

I’d also be interested in something that works with Ubuntu. I’d love it if you could give me a shout should you find something.

[leipert]

Looks cool, will give it a try.

A little feedback on the landing page: Almost turned away because I thought „Sign up with Google“ was the only choice and I missed the little „Prefer entering email instead?“ below. I just prefer not having a Google account ;)

Maybe the styling of that link could be adjusted so that it isn’t glanced over that easily.

[KishanBagaria]

Good call. We'll de-emphasize Google, it's just for convenience atm.

[megraf]

I don't mean to be pessimistic , but I wonder how you're going to be anything but reactionary to apples changes to the database. I've written a similar (much simpler) version of what you're proposing to sell- the database queries are hysterical.

I've been thinking about using the accessibility I framework to ETL the messages into Kinesis to facilitate event driven architecture. 'archived' Messages and attachments are stored as bsob.

Hell, I've even experimented with some goofy stuff, like FaceTime to VOIP- asterisk is just so flexible

[voltagex_]

How did you bridge FaceTime and VoIP?

[atatatat]

Matrix/Beeper?

[voltagex_]

Those are text-only bridges, I thought.

[eric-hu]

Looking over your site, I'm curious about a few things:

- The FAQ says you'll eventually open source your SDK. I admire that as a software engineer, but from a business perspective, isn't this risky? It seems like--if you don't pass messages through an intermediary service--your value-add is mostly in this SDK + your design. All of these elements are copy-able. - The jobs page includes a backend engineer, so what will go through your backend, if not any message data?

- Are there legal concerns about reverse-engineering these APIs and using them? Some of your supported chat services have open APIs, but others seem intentionally closed. (And a couple parent companies seem downright hostile to 3rd party use.)

[PhantomGremlin]

Source: I built a third-party desktop client for iMessage at https://texts.com

Does the texts.com client solve the original problem? Does it make it easy to go back and view all those old text messages? I know the website says "from forever ago easily". But you didn't say that explicitly in your post.

I assume you're being modest and not "pushing" your solution. But if it simply solves the problem, maybe you should be shouting this out loud!

[KishanBagaria]

It absolutely does, scrolling up shows older messages instantly.

Also, a "jump to date" feature (like Telegram) is planned (and super easy to build.)

[mdasen]

A piece of feedback: it would be nice to know what the price is on your site. You say "We make money by charging you a monthly subscription," but there's no pricing.

[KishanBagaria]

Indeed, we’re in private beta atm experimenting with pricing and product. Don’t want to mislead or turn away people until we finalize it.

[TwoBit]

OK, so document that?

[CoolGuySteve]

I've built an iMessage chat.db reader as well. To me the most astounding thing is that chat.db is unencrypted sqlite. Anyone who can see your home folder can read all your private chats or upload the chat.db to be read somewhere else.

It's impressively insecure.

[KishanBagaria]

macOS now has System Integrity Protection which requires manual Full Disk Access perms to read chat.db so any random app cannot touch it.

[tl]

Which you end up applying to random things like ruby because Emacs.app uses a ruby-based launcher and anything less than Full Disk Access breaks dired.

[interactivecode]

So as a developer you need full disk access to run your programs? Oh my god, the surprise ;-)

[ascar]

Reading this after GP called it reverse engineering makes me feel he should've use different terminology.

[b3morales]

Why would a lack of encryption make it not reverse engineering? There's a database schema, it was engineered* and used to build a database. They went from the live data back to the schema.

*For some definitions of "engineered", apparently.

[threatofrain]

Very eager to see how your app works when it opens up to the public. Are there ways to provide assurances for your end-to-end encryption which may be digested by experts?

You also mention on your website that end-to-end only works when the platform "supports" it — does that include Windows with your currently always-on Mac solution?

[KishanBagaria]

> Very eager to see how your app works when it opens up to the public. Are there ways to provide assurances for your end-to-end encryption which may be digested by experts?

We recommend using a simple MITM proxy or a firewall app like Little Snitch / LuLu to inspect the network traffic.

> You also mention on your website that end-to-end only works when the platform "supports" it — does that include Windows with your currently always-on Mac solution?

Yes – there's a peer-to-peer connection between the Windows and Mac devices.

[prpl]

I had done something too, but I also needed a Manifest.mbdb parser to reconstruct backup to disk, then process the messages and copy all the images around to make it work. I did it mostly to make a simple-to-read archive of chats before deleting them.

[judge2020]

Did your work turn into some potentially sharable shell scripts, perhaps? Would be a useful tool to avoid having to store messages in iCloud.

[prpl]

I have not used it for a very long time, so I can't comment to how well it works.

That said, this is it: https://gist.github.com/brianv0/35f36a32366a2c34be8d

(You need an unencrypted backup directory to run it over)

Note: It looks like the flat directory structure of the backup is now 256 dirs prefixed by first two characters of file name. I'm not sure, but a small change to the name "sha_file" may be needed to just path.join that with the first two chars of the sha.

[prpl]

couple of extra hints:

Python 2 but can be ported forward without much effort. Manifest.mbdb is now Manifest.plist (there's a sqlite thing now too). timestamp sql might not work so great, would need to check query.

[camhenlin]

I also have a similar but older personal project that allows you to send iMessages over the web via an always on Mac.

https://github.com/CamHenlin/iMessageWebClient

I also have some other iMessage related git repos on my GitHub. Interesting stuff!

[cmckn]

Woah, that's a heroic effort. Can't wait to try it! Huge thanks :)

[propogandist]

why the need to sign up with Google? I don't want Google to be associated with text messages.

[cfqycwz]

Maybe they just added it, but there's also a "Prefer entering email instead?" link below, which I appreciated

[getcrunk]

Does what's app have and open api? Otherwise how did you integrate it?

[slowraise]

Is texts.com a work around for sending live imessages on a PC?

[KishanBagaria]

Yes, with a catch: it requires an always on Mac (which can be in the cloud) or a jailbroken iOS device. We plan to make it work without the catch.

I've worked with 10+ messaging platforms so far and Apple's protocol is the most obfuscated and complicated to reverse engineer. Apple has invested millions of dollars to make the iMessage protocol super hard to reverse engineer. It's how they sell tons of iPhones after all:

> However, Craig Federighi, Apple’s Senior Vice President of Software Engineering and the executive in charge of iOS, feared that “iMessage on Android would simply serve to remove [an] obstacle to iPhone families giving their kids Android phones”. (https://www.androidpolice.com/2021/04/28/apple-admits-that-i...)

[xoa]

Looks like a super cool project. I'm always happy to see new client-side software for communication protocols written that aims to improve upon built-in! And iMessages in particular could certainly use it. With the demise of iTunes, it now feels like iMessages is probably one of the most crufty-but-heavily-utilized user facing pieces of software Apple puts out. Maybe they have some long term plans to refactor it ala iTunes but yeesh.

This bit did make me wonder though:

>We plan to make it work without the catch.

Out of curiosity, do really think that's realistic, or even desirable long term? iMessage is ultimately an Apple service that runs heavily on Apple's infrastructure, and is directly subsidized by sales of their highly vertically integrated hardware platforms. If it turns into a cat-and-mouse fight it seems like they're always going to have the eternal upper hand, which in turn seems like it'd make for a subpar user experience (ie., breaks randomly which for an instant messaging service would be pretty bad). Also seems like they might actually be motivated to respond rather than ignore it since it'd actually be directly leeching their infra if it will work on PCs/Android without a Mac/iDevice purchase in the equation (unlike hackintoshes for example, where whatever debate there is to be had about probably very low "lost sales" it doesn't actually cost them anything).

Obviously you've probably thought this all through, but seems like just requiring an old cheap Mac or old jb'd idevice and thus avoiding Apple might be an easier path. Or alternately just stick to offering a flat out better client with the iMessage bit being Mac-only. Will be interested to see how it goes though!

[b3morales]

> one of the most crufty-but-heavily-utilized user facing pieces of software Apple puts out. Maybe they have some long term plans to refactor it ala iTunes but yeesh.

The Messages app on Mac was rewritten for Big Sur (last year). It gained some features but also some bugs in the process.

[saagarjha]

iMessage is backwards compatible with older devices. This means Apple can’t stop supporting old versions of the protocol, even as they are “cracked”.

[account42]

They could release updates for those older devices to unbreak them after protocol changes.

[saagarjha]

They could, but it seems like a lot of work for them. I am not sure they even have the pipeline to build old OSes anymore…

[ComputerGuru]

Agreed on intentional obfuscation. It even has booby traps. https://neosmart.net/blog/2018/imessage-for-windows/

[cle]

I know it's a long-shot, but do you plan on documenting the protocol?

[interactivecode]

Wasn’t it also that they had a great distributed way of doing messages but got sued to changes their implementation?

[hboon]

FaceTime was. Not sure if it happened to iMessage

[KoftaBob]

> We plan to make it work without the catch.

Interesting, how do you plan to do that?

[codetrotter]

> it requires an always on Mac (which can be in the cloud)

That’s surprising. How come?

[ArchOversight]

Likely because each iMessage receiving device has a private key to decrypt the messages, and senders encrypt the iMessage against all the keys in your key bag.

To access iCloud and or services you also need to have a device security key that is tied to genuine hardware.

[sumnole]

Anyone know of something similar for Android texts?

[mvkel]

any way to get around the reg/invite wall to try it out?