[CobrastanJorji]

This title really undersells the absolute insanity of the described solution. This is a beautiful example of "if it's stupid, but it works, it's not stupid." The justification is very convincing.

One thing I'm curious about: how did you build your corpus of meme images and videos?

[leokennis]

It reminds me a little of https://www.beeper.com/. It allows you to read iMessages on Android and other platforms. To make this work, they will ship you some old iPhone to act as a server "bridge": https://twitter.com/ericmigi/status/1351934418961661959

If it works, it works. But it also speaks volumes about Apple's disregard/inexperience with exposing their stuff via the web - https://www.icloud.com/ being the prime example: half the stuff the phone apps can do are not available (cannot create a reminder with a due date...) and the things that are there are slow and buggy.

[dewey]

I think I've seen a post from https://texts.com about that they. I don't think they ship you the iPhone though, the host it themselves.

[kaladin-jasnah]

I would imagine the only scalable way to run such a service is to run macOS virtual machines with multiple user accounts for each iMessage user.

[aemreunal]

(Not the author) Maybe they leveraged https://knowyourmeme.com/? But that can't possibly have all the random memes, could it?

[mandatory]

Author here: KnowYourMeme is one of many sites that memes are continually ingested from (any site that has memes I try to ingest regularly) :)

[aemreunal]

Amazing work! Also, thank you for making that feed on the main page, been laughing for a while here :D

[yojo]

Also lost 20 minutes doom scrolling that feed. Add an upvote button and some ML and you could destroy some lives.

[mandatory]

Thanks! Comment made my night.

[GistNoesis]

Nice IPhone cluster.

Have you tried something based on deep-learning that uses Transformers : https://github.com/roatienza/deep-text-recognition-benchmark (available weights are for tasks that seem similar to OCR so there is a good chance you can use it out of the box). With a good gpu it should process hundreds to thousands image per seconds, so you likely can build your index in less than a day. (Maybe you can even port it to your iphone stack :) )

https://github.com/microsoft/GenerativeImage2Text (You'll probably have to train on your custom dataset that you have constituted)

There are tons of other freely available solutions that you can get with a search for things with keywords like "image to text ocr" "transformers" "visual transformers"...

[astrange]

You can do better than a general image-to-text model reading memes, because they all use the same fonts - so you want something trained off synthetic data made with that font.

[generalizations]

Personally, I've been hunting for something that can extract both the text and the associated image. I've never seen anything that can do both.

[taneq]

All hail the memelord!

[spiffytech]

How do you ingest your social circle's in-group memes? Are they reliably posted to meme generator sites?

[counttheforks]

What about copyright?

[code_duck]

OP’s meme site lists where each image comes from. Looking through it I mainly see ifunny and 9gag.

[solarkraft]

Do you crawl telegram channels?

[bryanrasmussen]

yeah but lots of things that work are stupid because there are many other solutions that work better, the greatness of this crazy solution is it really seems like the best solution given price requirements.

[jychang]

I feel like I’m taking crazy pills in this thread. Am I the only one who talks to Gen Z kids who explore around their iPhone apps? This definitely isn’t the best option given price requirements. It’s not even the most convenient option.

I’m around age 30, not 13, so similar to the article, my first instinct was also to create a database and OCR the image. But by total coincidence, yesterday I had a conversation with my 14 year old cousin on the topic of saving memes. Her response was along the lines of “yeah, everyone nowadays just saves the image to your iPhone photos, and then just search for it later from the photos app”.

Yeah. This whole article is literally already built into iOS UI, not just a hidden API. And kids all seem to know about this, apparently.

This article uses an example meme with the text “Sorry young man But the armband (red) stays on during solo raids”. I saved it in my iPhone photos app… and found it again through the search function in the photos app.

https://imgur.com/a/BPICjOz

https://imgur.com/a/55el9uQ

This is a solved problem already, by teenager standards.

I felt extremely old yesterday when I was talking to my cousin. And I felt extremely old today, reading this article. This is because looking back, the past few decades of CS cultural intuition have established that text are text, and images are images. Strings and bitmaps don’t mix.

This seems sort of obvious to anyone in tech, but I realized that from a clueless grandma perspective, not being able to search up text in photos wasn’t really obvious. Well, the roles are reversed now. Ordinary people now have access to software that treats text as a first class citizen in photos by default.

[counttheforks]

The entire point is to find memes you don't already have.

[jychang]

No, it's not. The author even mentions private memes that are in-jokes. He built a service that can be used to explore memes, but people generally don't search for new memes in a search engine. They tend to use the search engine to find memes they already have.

[counttheforks]

knowing != having

[ZephyrBlu]

How would you solve sourcing and distribution using just iOS though? Sure, it's built into iPhones, but if you want to create a comprehensive globally accessible meme search I don't think you can do that by saving memes to your iPhone.

[jychang]

A kid doesn’t care about that. They just save the memes they like, including the custom ones that their friend made which doesn’t make any sense to anyone else. You don’t need a comprehensive global search engine, if you have a tool that will tell you exact personalized answers to images you’ve saved before. And kids these days save everything; it’s like how people use Gmail, no point in deleting if you just archive.

If you’re talking about kids without iPhones, then I don’t know, I’m assuming there’s probably some competitor apps on android now.

But I think you’re thinking too narrow. Don’t worry about a meme database. What about a searchable visual database of everything you’ve ever seen?

[TeMPOraL]

The use case for the meme database is slightly different: it's to find that meme you saw somewhere else. Local search isn't enough then.

Though I share your feelings somewhat - I was completely surprised by what you wrote in the comment higher up about iPhone gallery search. I didn't realize this is possible in a reliable fashion, much less off-line and deployed in a mass market device.

[rakoo]

> The use case for the meme database is slightly different: it's to find that meme you saw somewhere else.

The whole idea is that whenever you find a meme you like you save it, so all search is still local

[password1]

Your cousin and your friends don't care about that, not sure if it applies to all kids worldwide honestly. I'm sure "meme collecting" is a common practice among many teens, but I don't think it means that every teen saves all meme/images they encounter.

You know that some teens don't even save some images? They store them on specific instagram accounts they make for a specific category. My cousin had an instagram account for close friends (5 people) where she only shared bad things that happened to her during the day. Another one for nice things, etc. All those memories were recorded in app and never saved in the device, only stored as stories on the account. Guess what? She was sad a while ago because somehow she lost access to one of this accounts and so to all those pictures/videos.

Also the fact that some teens save a lot of pics on their devices doesn't mean much in the grand scheme of things. In 2008 I had folders upon folders of images downloaded from the internet. Now i'm not even sure where they are, probably in some hard drive in some closet. You can be sure that if i remember any of those contents I won't dig up my hard drive, but google them (use a comprehensive global search engine). We have no idea where all this collected data from teens will be in 15 years, it's not unlikely that it will be lost or archived in hard to reach places and forgotten. I've stumbled some times into "meme dumps" where they upload all their memes to a service to free space on their device/icloud.

For sure teens use technology in ways that might be unexpected and counter-intuitive to us, but I don't think that invalidates in the slightly the need of a global search engine for memes. It's a good idea if i want to find a meme that I saw, a need that I don't think will disappear anytime and that's also not a millennial+ only problem.

[karatinversion]

I only realised quite recently that I could now select text in images on my iPhone the same way I could if I was looking at a web page.

[jychang]

You can do this back in 2021!

https://www.macrumors.com/how-to/copy-paste-text-from-photos...

I really want to emphasize how insane this situation is, because I think most tech people won’t realize what’s happening unless it’s pointed out.

If you’re a typical tech person, you probably look at this, go “oh, iOS Photos now OCRs every photo. Cool, that’s 2000 or 2010 era fancy tech, boring these days. And then a search engine on those strings, yeah cool, nothing too mind blowing”. The sheer boring-ness of this by tech people standards meant that this iOS UI change went under the radar.

That’s not true for non tech people.

The people who discovered this, put this to use immediately. You can search up anything from an image now. Old memes? Sure. Forgot the name of a restaurant you went to, but remember that you took a picture of the menu and the beef dish was amazing? Search up the word “beef” and it’s probably in there. Took a screenshot of an article, remember 1 or 2 words from it, but can’t find it on Google? Search for those 1-2 words you remember to find the screenshot, then use the phrases in the screenshot to find the article on google. Trying to find a picture of a cat you saved? Type in ”cat” and search for it. Yes, the photos app can do that too.

Screenshots are cheap and instant. Kids never delete them. It’s like how Gmail “archive” feature in 2005 revolutionized email because you never had to delete an email. Well, iCloud Photos “optimize storage” means that you can effectively store infinite screenshots.

There’s another UX revolution happening in terms of saving information. It’s just that photos became easily instantly searchable, and nobody seemed to have really noticed the implications this has on storing memories, and boosting recollection. This can possibly be the equivalent of “you’ll always have a calculator in your pocket” but equivalent analogy to memory techniques like spaced repetition.

[TeMPOraL]

> I really want to emphasize how insane this situation is, because I think most tech people won’t realize what’s happening unless it’s pointed out.

Count me in. If you asked me about the OCR itself, I'd probably say "yeah, it's mostly been solved for a good decade for print books and articles, but it's unreliable enough". I somehow never considered OCR might have gone better - possibly because my main exposure was through badly OCRed book scans and a built-in OCR in some PDF reader I used at one point.

It definitely didn't occur to me that OCR works well enough on arbitrary images, and it's cheap enough compute-wise that you could do it locally in a casual fashion.

Nice thing you have there in the Apple garden. Over here in Android land, I have the opposite problem. You say:

> This can possibly be the equivalent of “you’ll always have a calculator in your pocket” but equivalent analogy to memory techniques like spaced repetition.

and all i can think of is how I recently became convinced that a Samsung flagship is losing my photos. There's been a couple cases over the past few months when I felt really damn sure I made a set of photos of something (e.g. remodeled kitchen), but when I checked on the phone, it turned out those photos don't exist, or there is maybe just one where I expected 5-10. They aren't in the gallery. They aren't in the filesystem. Poof, gone.

So either I'm getting senile in my 30s, or something is off with the way my phone stores photos. I did a web search for this the other day, there are relatively recent reports on-line complaining about the same thing, but no one has any evidence. I'm thinking about doing an experiment now (basically make extra photos every day and document them in a paper notebook, and check after half a year if the photos match the notes) - but the point of me sharing this is: I no longer trust new tech, smartphones in particular, to handle basics correctly. Much less do something advanced like reliable text search on images.

[duccinator]

There is a chance that your photos are being backed up by some cloud service and being removed from your gallery. The most likely suspect is Google Photos.

Note that Google photos not only OCRs, but it also does a visual search of objects, faces, scenery etc. and is extremely powerful.

[djmips]

Well maybe this isn't for teenagers with iphones