Comments on: Social Media Sentiment Monitoring http://www.murraynewlands.com/2009/11/social-media-sentiment-monitoring/ Affiliate Marketing Blog, Email Marketing Blog and Social Media Marketing Blog Thu, 22 Dec 2011 10:46:13 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Steven Maimes http://www.murraynewlands.com/2009/11/social-media-sentiment-monitoring/comment-page-1/#comment-7121 Steven Maimes Thu, 03 Dec 2009 17:14:24 +0000 http://www.murraynewlands.com/?p=5533#comment-7121 I agree with Hannah’s opinions (above) and would like to add from my first-hand experience. I have done mainstream and social media human sentiment measurement work for ImpactWatch, Cymfony and Biz360. Quality human analysis accuracy should be over 90% and is most useful for key publications with major influencers. Automated analysis is probably around 60-70% accurate. However, Biz360’s automated sentiment wizard produces accuracy around 80%. One problem occurs when human sentiment analysis is outsourced to India or the Philippines. In several instances I know first-hand the accuracy drops to 70% or below – and in my opinion is no better than automated analysis. If accurate sentiment measurement is important, clients should ask: who will be doing the human sentiment analysis? Clients should also have some idea the criteria for positive/negative sentiment. I agree with Hannah’s opinions (above) and would like to add from my first-hand experience. I have done mainstream and social media human sentiment measurement work for ImpactWatch, Cymfony and Biz360. Quality human analysis accuracy should be over 90% and is most useful for key publications with major influencers. Automated analysis is probably around 60-70% accurate. However, Biz360’s automated sentiment wizard produces accuracy around 80%. One problem occurs when human sentiment analysis is outsourced to India or the Philippines. In several instances I know first-hand the accuracy drops to 70% or below – and in my opinion is no better than automated analysis. If accurate sentiment measurement is important, clients should ask: who will be doing the human sentiment analysis? Clients should also have some idea the criteria for positive/negative sentiment.

]]> By: Mark Evans http://www.murraynewlands.com/2009/11/social-media-sentiment-monitoring/comment-page-1/#comment-7119 Mark Evans Thu, 03 Dec 2009 16:54:03 +0000 http://www.murraynewlands.com/?p=5533#comment-7119 Murray, As you've highlighted, sentiment is a key part of the social media monitoring equation. Truth be told, it's challenging to have 100% accuracy when trying to use technology to tell whether something is positive, negative or neutral. That said, the social media technologies used to automatically determine sentiment do a really good job and, I suspect, will improve as the technology continues be improved and refined. cheers, Mark Mark Evans Director of Communications Sysomos.com Murray,

As you’ve highlighted, sentiment is a key part of the social media monitoring equation. Truth be told, it’s challenging to have 100% accuracy when trying to use technology to tell whether something is positive, negative or neutral.

That said, the social media technologies used to automatically determine sentiment do a really good job and, I suspect, will improve as the technology continues be improved and refined.

cheers, Mark

Mark Evans
Director of Communications
Sysomos.com

]]> By: Murray http://www.murraynewlands.com/2009/11/social-media-sentiment-monitoring/comment-page-1/#comment-7086 Murray Wed, 02 Dec 2009 18:40:22 +0000 http://www.murraynewlands.com/?p=5533#comment-7086 Hannah Thank you for the insight into the accuracy. Murray Hannah

Thank you for the insight into the accuracy.

Murray

]]> By: Hannah Del Porto http://www.murraynewlands.com/2009/11/social-media-sentiment-monitoring/comment-page-1/#comment-7080 Hannah Del Porto Wed, 02 Dec 2009 16:13:58 +0000 http://www.murraynewlands.com/?p=5533#comment-7080 Not to my knowledge. It would be extremely tedious as you would have to have large samples first analyzed by humans, then that sample and the automated sample again analyzed by humans for accuracy. I believe the statistics on the accuracy of automation are also estimates. I have most often seen providers of sentiment automation claim to have around 70% accuracy while human analysis of course varies with the human :) but can be expected in the 90th percentiles of accuracy. Not to my knowledge. It would be extremely tedious as you would have to have large samples first analyzed by humans, then that sample and the automated sample again analyzed by humans for accuracy.

I believe the statistics on the accuracy of automation are also estimates. I have most often seen providers of sentiment automation claim to have around 70% accuracy while human analysis of course varies with the human :) but can be expected in the 90th percentiles of accuracy.

]]> By: Murray http://www.murraynewlands.com/2009/11/social-media-sentiment-monitoring/comment-page-1/#comment-7043 Murray Tue, 01 Dec 2009 23:47:20 +0000 http://www.murraynewlands.com/?p=5533#comment-7043 Bran and Hannah Thank you both very much for your input. Much appreciated, truly. Has anyone ever done any comparative review work? Murray Bran and Hannah

Thank you both very much for your input. Much appreciated, truly. Has anyone ever done any comparative review work?

Murray

]]> By: Hannah Del Porto http://www.murraynewlands.com/2009/11/social-media-sentiment-monitoring/comment-page-1/#comment-7036 Hannah Del Porto Tue, 01 Dec 2009 20:25:18 +0000 http://www.murraynewlands.com/?p=5533#comment-7036 Human analysis is more accurate, but can be cost-prohibitive at high volumes or just not a good value if brands are not measuring individual mentions. Yes, the software will get it wrong 30% of the time, but will still be able to delineate the big picture. - Most of our clients prefer human analysts but for some who receive several thousand mentions per day, it's just not possible to read them all even with a team of analysts. So a second application is to use automation for the bulk of high volume mentions, leaving the most important sources or comments to be reviewed by hand. - Sadly, the debate over sentiment automation (and countless other media monitoring issues) continues to be a distraction as people look for a one-size-fits-all solution where none exists. Cheers, Hannah Del Porto ImpactWatch @hcdelp Human analysis is more accurate, but can be cost-prohibitive at high volumes or just not a good value if brands are not measuring individual mentions. Yes, the software will get it wrong 30% of the time, but will still be able to delineate the big picture.
-
Most of our clients prefer human analysts but for some who receive several thousand mentions per day, it’s just not possible to read them all even with a team of analysts. So a second application is to use automation for the bulk of high volume mentions, leaving the most important sources or comments to be reviewed by hand.
-
Sadly, the debate over sentiment automation (and countless other media monitoring issues) continues to be a distraction as people look for a one-size-fits-all solution where none exists.

Cheers,
Hannah Del Porto
ImpactWatch
@hcdelp

]]> By: Brad Brodigan http://www.murraynewlands.com/2009/11/social-media-sentiment-monitoring/comment-page-1/#comment-7026 Brad Brodigan Tue, 01 Dec 2009 14:57:38 +0000 http://www.murraynewlands.com/?p=5533#comment-7026 At Biz360 we have been providing automated sentiment tools for our clients for more than 6 years. In our social media platform (Community Insights) we provide clients both the ability to use the automated sentiment as well as to override or manually score the sentiment. Our experience is that it provides a very important layer of understanding the conversation even if there are occurrences when a single post is scored wrong due to sarcasm or some other issue. When looking at a sample of 20, 50, 100, or 1,000 posts, automated sentiment is a very valuable tool to be able to quickly understand trends in the conversation. We believe the combination of automated sentiment with a manual override provides the bets of both worlds and the most valuable experience. At Biz360 we have been providing automated sentiment tools for our clients for more than 6 years. In our social media platform (Community Insights) we provide clients both the ability to use the automated sentiment as well as to override or manually score the sentiment. Our experience is that it provides a very important layer of understanding the conversation even if there are occurrences when a single post is scored wrong due to sarcasm or some other issue. When looking at a sample of 20, 50, 100, or 1,000 posts, automated sentiment is a very valuable tool to be able to quickly understand trends in the conversation. We believe the combination of automated sentiment with a manual override provides the bets of both worlds and the most valuable experience.

]]> By: affdump (affdump) http://www.murraynewlands.com/2009/11/social-media-sentiment-monitoring/comment-page-1/#comment-6950 affdump (affdump) Sun, 29 Nov 2009 16:19:50 +0000 http://www.murraynewlands.com/?p=5533#comment-6950 <strong>Twitter Comment</strong> <a href="http://twitter.com/affdump" title="Twitter Comment"> <div class="ccimg1" title="affdump (affdump)" style="float:left;margin-right:10px;padding:0;width:60px;height:60px;background:url(http://www.murraynewlands.com/wp-content/plugins/chatcatcher/picbg.jpg) no-repeat top;cursor:hand;"> </div> <div class="ccimg2" title="affdump (affdump)" style="float:left;margin-left:-70px;margin-right:10px;padding:0;width:60px;height:60px;background:url(http://purl.org/net/spiurl/affdump) no-repeat top;cursor:hand;"> </div> </a> @murraynewlands Social Media Sentiment Monitoring [link to post] (via @affdump)<br /><br /> - <a href="http://chatcatcher.com" target="_blank">Posted using Chat Catcher</a> Twitter Comment


@murraynewlands Social Media Sentiment Monitoring [link to post] (via @affdump)

– Posted using Chat Catcher

]]>