[reitanqild]

SQLite backend

Because comments are not Big Data.

Yep, you got it.

Also MIT licensed.

Please go ahead and kill Facebook comments.

I strongly dislike realname policies in this day and time where a single remark can cost you your job.

[anigbrowl]

Because comments are not Big Data.

I'd like to understand the significance of this. I'm extremely interested in comments as big data (to try to understand social networking patterns better from studying the structure of nested comments) and I'd like to get a handle on the opposite point of view - what is it that you prefer about this approach?

[reitanqild]

> I'd like to understand the significance of this. I'm extremely interested in comments as big data (to try to understand social networking patterns better from studying the structure of nested comments) and I'd like to get a handle on the opposite point of view - what is it that you prefer about this approach?

While this might be interesting to you it is in the best interest of the site owners and commenters worldwide that not everything they write is instantly correlatable to anyone who happens to work for Facebook or be in a position where they can somehow buy or otherwise demand this data from them.

[eitland]

User data can be valuable for research or even for profit in ways that doesn't harm the users but I think it should be treated like this:

https://en.m.wikipedia.org/wiki/Treaty_on_the_Non-Proliferat...

I guess collection, processing and stockpiling of user data should be regulated for the benefit of consumers, moderators and world peace.

[anigbrowl]

You seem to have me confused with someone else.

[reitanqild]

Care to explain?

[anigbrowl]

I don't work for FB, nor am I in a position to demand big data flows from someone else. I'm just intensely curious about the dynamics of commented discussions and think it's an under-studied phenomenon. I'm not interested in making money out of it, I just want to understand how ideas promulgate and what factors drive virtual communities.

I was confused by the original post because a) I'm not sure why using SQLite makes comments into 'not big data' (is it extraordinarily hard to get statistical information out of SQLite or something?) and b) even if a blog or website is completely private, might not the owner want to analyze what's going on within the private social network they've created?

Also, I question your blanket assertion about what is or in't in the interest of other people. You seem to have taken my prior comment as an intent to exploit comment data for financial gain.

[reitanqild]

Aha.

Then you might get a license from me. And if you haven't tested with the HN dataset that is where I would start.

I was confused by the original post because a) I'm not sure why using SQLite makes comments into 'not big data' (is it extraordinarily hard to get statistical information out of SQLite or something?)

No, getting data out of SQLite is easy. But I have yet to hear anyone using SQLite for anything that would be called big data in a way that makes sense. (Yes, my previous employers file share where we could upload 5GB files doesn't qualify.)

I think the comment on the website is supposed to be funny in a bitter way.

and b) even if a blog or website is completely private, might not the owner want to analyze what's going on within the private social network they've created?

I have yet to manage to think of a private website that might contain enough non-spam comments to qualify as big data.

And for small and medium size data plain SQLite is wonderful : )

What I and others are skeptical to is REAL big data, where certain companies scoop up everything they can get their hands on. This is an often dirty, potentially really harmful raw material as we have seen in articles about the moderation teams that makes sure Facebook and others stays reasonably clean.

With a little bit of processing this "nuclear waste" is also possible to weaponize. Two ideas off the top of my head:

- Facebook and others can easily name thousands of people who waste their employers time during the day.

- or post a number of people who run pseudonymous accounts for various reasons but have failed to maintain proper opsec.

[lifthrasiir]

Comments for your website are not Big Data (in most cases). Comments aggregated throughout multiple websites can be Big Data, but making them available is not your responsibility.

[ultrahate]

Well because it's dealing with like comments on blog posts. There's a time and place for mass scale comment backends, but most blogs are not those places, and most people ain't got the time anyways.

[stevekemp]

I also wrote a self-hosted comment-server:

https://github.com/skx/e-comments/

Mine uses Redis or SQLite for storage, and uses markdown. The biggest difference is mine is flat. I keep thinking I should rework it to allow nested comments, and avatars, but at the same time it is nice and simple as-is..

[stevekemp]

Nested comments are now supported, FWIW.

Demo - https://tweaked.io/guide/demo/

[intopieces]

>I strongly dislike realname policies in this day and time where a single remark can cost you your job.

Can it though? Because I see some vile comments made by people who appear to be using their real name, photo, location information on their profile.

[lazaroclapp]

Can doesn't mean "necessarily will". It depends on how much attention the comment attracts a lot of times, even more than on its content. Things that anyone could say in anger on a bad day or as dark joke can come back to haunt you if they happen to raise a media circus. Yet, advocating genocide can have no effect in your career if it goes broadly unnoticed. That's part of the problem.

[reitanqild]

PyCon. Two decelopers on the floor lost their job because they were overhead by someone who posted it to twitter.

[asitdhal]

> Because comments are not Big Data.

Big data is mostly used when you want to monetize those comments.

[bidoof___]

Disqus and social sign-on comments are cancer. They must be plugged. Too much Ivan all over social media these days. Zuckerberg is equally disturbing a threat and should not be trusted with his bizarre utopian language.