Hacker News new | ask | show | jobs
by jsiarto 4883 days ago
This is a very interesting study--nice work! Our company does social research for all types of companies and we've found most automated sentiment analysis to be subpar (at best, 60% accurate). The problem with just looking at words is that there is no context of the whole Tweet and computers are generally bad at picking up sarcasm, innuendo and turns of phrase that may contain negative words in a positive manner (toward the brand or company).

I realize that this isn't the key focus of your paper, but we've found that sampling and human analysis/tagging is far more accurate at judging the sentiment around a brand, company or topic.

2 comments

Thank you.

In the context of this study, I found that it was impossible to accurately infer 'sentiment' of a single tweet or person (not just because of sarcasm and other nuances). However, when you take the average of a group (wisdom of the crowd) then the results are much more promising. A trends noticed across thousands of users is also more interesting than the potentially unreliable sentiment of a single person.

In this case, I suppose it is definitely just the words that are being analysed – not true 'sentiment.' I wouldn't rule it out as inaccurate though, it just depends what you're looking for and how you use the results. Compared to other sentiment data-sets, the ANEW approach seems more more detailed (the original scorings are created from human tagging).

I do agree though, that automated approaches can be inaccurate if you're looking for fine-level analysis.

Completely agree--we always start our analysis with basic questions the client wants answered. I also appreciate how in-depth your piece was--we need more research and case studies like this. If I have to listen to one more social monitoring tool salesperson tell me how amazing their sentiment analysis is without ever showing concrete examples and results...
I worked for a company that just asked people once a week. For a little extra effort, it was 88% accurate (12% of people were filtered out for just hitting random items).