[raybb]

Don't have a link now but as I understand it the main problem is they were using websockets and it didn't scale to so many users. So they're switching to rest api calls and that should fix a lot of the issues. But it's a pretty big switch they've been working hard on.

[ehutch79]

Shouldn't rest (or graphql) be the default? with realtime updates being a progressive enhancement?

[shagie]

Note that what amounts to a single page app with all the links rendered after fetching some data from a service isn't going to be something that can be indexed by a search engine.

A google search for site:lemmy.ml with it rendered this way will never result in any data.

This may or may not be a good thing since it isn't necessarily assured that the data is stored for extended periods of time (compare: https://masto.host/mastodon-content-retention-settings/ ).

Part of the value that people have ascribed to Reddit is being able to do searches for a product review and limit it to just pages served by reddit. Lemmy appears to be ephemeral and not searchable.

Discussing the current Star Trek SNW episode may be ok, but reading a discussion from a few years ago about Emissary on DS9 ( https://www.google.com/search?q=deep+space+nine+emissary+sit... ) is something that isn't going to be doable.

[RobotToaster]

It looks like kbin is a more traditional php program, so should be indexable?

[shagie]

Yes, except that they've blocked it for crawlers that honor that.

https://pagespeed.web.dev/analysis/http-kbin-social/ckiwikap...

> CRAWLING AND INDEXING

> Page is blocked from indexing

[ehutch79]

I'm not sure what any of that has to do with using websockets by default vs traditional rest data fetching?

[shagie]

You can cache some REST requests which would reduce the load of serving them out of the web socket request each time.

More importantly though (with either of these) is that when Google looks at the page, there are no links for it to traverse or index.

This also means that the site/community isn't discoverable from a random person searching for it inhibiting organic growth.

For example, if you do https://www.google.com/search?rls=en&q=where+to+discuss+star... it will likely never return a link to Lemmy (it may return pages that link to lemmy - like the reddit page), but not lemmy itself.

[randomdata]

Choosing a random Lemmy instance, I see server rendered content and <a href> links that are followable. Searching for 'site:<instance domain>' on Google turns up plenty of results, very much identical to what I see when searching for 'site:reddit.com'. Google seems to have no trouble indexing the site.

I expect you won't see results on Google, save for when you explicitly ask for the site, because there isn't much in the way of reputation. You are highly unlikely to see my personal blog, which is just plain old boring as it gets HTML, come up in search results for the same reason.