Wesley83's Blog

Wesley Faulkner's personal blog. (wesley83 on twitter)

  • Home
  • About Me
    • Edit
    • Delete
    • Tags
    • Autopost

    Can you trust Klout's accuracy?

    For those of you not familiar with Klout, in short, it is a service that rates the influence and reach of someone in the social media space. Klout looks at Twitter and Facebook accounts, and will soon add LinkedIn to determine a user's Klout score.

    There are several companies out there trying to quantify people so that companies know who to target to get the best results for their marketing efforts. A PR friend of mine directed me to a recent post from Peter Shankman (of HARO fame), who praises Klout on providing "real information." Real? Really? I'm just not convinced. But maybe it's me.

    I knew that the system wasn't perfect, but the issue that really put things into focus for me was on October 29th when I went to Klout's homepage at http://klout.com and was presented the following image:

    First_page
    If you see the image in the square on the far right, with the name RYE_BREADY, it may look familiar. Anyone who has been on twitter for a while will recognize it as a spam bot.
    First_page_thumb
    I wanted to verify if this was indeed a spam bot, and how they were able to get such a high Klout score, so I clicked through. The profile link was here, but has been removed by Klout after they were made aware of its existence: http://klout.com/Rye_Bready I took some screenshots before it was removed.
    (download)
    Click here to download:
    can-you-trust-klout-s-accuracy-waDCclttrnDfsvFGpJfu.zip (442 KB)
    Her stats are on the thin side. She also only has one badge. This gets even stranger, and also adds weight to my argument that this might be a spammer. I checked out her twitter profile.
    Third_page
    That's right. It's suspended! Makes you kind of wonder about the scoring huh?

    From speaking with other members that have a Klout account there are more discrepancies that add doubt as to the accuracy of the service. The "influenced by" and "influencer of" sections. Many people have told me that they are very wrong, if never updated. I for one have person in the "influenced by" section that I haven't had contact with in months. When I contacted support for assistance they where not able to resolve this error after three tries.

    The mystery on how they get their scores and determine who influences who makes it impossible to verify or reproduce. The service and its algorithms are a black box. There is no way to tell where the service is, where it was, and if there are any improvements in the future.

    Through a personal contact at Klout I was introduced to Megan Berry who is a Marketing Manager there. First off, I would like to say that Megan was extremely helpful and very responsive. She informed me that the problems with spam accounts is something that Klout is very aware of, and is a problem they are still trying to solve.

    I am not sure exactly if the examples I have given above negates all the work that they have done to quantify clout, but it does show that they do have a way to go before their numbers can be fully trusted.
    I hope that the people at Klout take their accuracy seriously. Phil Hotchkiss, the Chief Product Officer,  has made accuracy his number one mission, but he has yet to reply to my inquires. If you have used Klout please share your experience in the comments below.

    Special thanks to the following for all their help with this post:
    @ckieff @aqiylah @MisoHungry @davidgiesberg @TheJenATX @luannsaid @gregarious

    • 14 November 2010
    • Views
    • 10 Comments
    • Permalink
    • Tweet
    • 10 responses
    • Like
    • Comment
    over 1 year ago John Taylor responded:
    John Taylor
    The larger issue is determining the best measure our social media efforts. Klout's system certainly has its shortcomings, which you've done a good (and fair) job of pointing out in this post, but I'm not sure what's much better. As I think about your post, I wonder if you couldn't write others with the same title: "Can you trust fill-in-the-blank's accuracy?"

    One of the challenges I face in my work is demonstrating to my internal customers the measure of our success in social media and in communications generally. Because few of these customers participate in social media, they don't have the basic vocabulary or understanding of how various social media channels work, much less the ability to understand if our efforts are effective.

    Because they do read newspapers and magazines (both print and online) they know if we're successful because they read it for themselves. That's not the case if we present to them an analysis of what we've done with various social media programs.

    The social media consultant who can offer a measurement program for businesses who are new to social media, but run established marketing and public relations programs stands to make a fortune if those measurements are solid enough to withstand scrutiny. So far, I've not met that consultant or seen that measurement program.

    I agree with you that Klout has some work to do to improve their scoring, but I think they've got a headstart on others. The question remains will competitors come along and surpass them before they can address the shortcomings you've done such a good job of pointing out.

    over 1 year ago lawrencekslive (Twitter) responded:
    Joshdavis300h_normal
    I think the Klout score is one of the better metrics that is available for determining influence. As such, I want to start including Klout scores as one of the metrics we present to clients. But I don't feel that is possible yet for a number of reasons.

    Large gaps in what is presented. I have a hard time believing that (in fact I know) that @AmericanExpress is influencing some accounts, but they currently have no "influencers of".

    Their Content Specialist area only draws from one small point in time and isn't representative of what they are doing now or even what they have done that last six months. See:

    Topic Summary

    Share
    #amexjovi
    #amexTravel
    Jon Bon
    jon bon jovi
    #amexgiftcard
    Jovi Soul Foundation
    --

    I know Klout is always working on this, but I believe they are extremely understaffed given how big the Klout name is becoming. Updates are rare.

    It makes sense to me that Klout would put a priority on making sure that Fortune 500 and Fortune 500 Global corporations that have Twitter accounts are updated regularly and attention is make to make sure their reports are correct.

    In the long run, everyone needs an accurate score, but these corporations and their agencies can help make Klout scores a key metric, and what Klout is presenting in their accounts now is so outdated, that I can't pitch them as a metric and feel confident.

    I have been in contact with individuals at Klout, and they have told me they are trying to put Fortune 500 companies on a more consistent update schedule, but that doesn't solve for having accurate scores for the millions of influencers who are the ones spreading messages and engaging in conversations.

    I have high hopes for Klout but they need some funding to hire more qualified people to make Klout scores truly relevant.

    over 1 year ago JoeFernandez (Twitter) responded:
    Yellow_face_normal
    Hey Wesley,

    I am the ceo and one of the co-founders here at Klout. You are totally right here, we've got to do better.

    As you mentioned in your article, quality is our number one goal here at Klout. We are putting a tremendous amount of work into both spam detection and data timeliness. In the next couple weeks we have a huge release coming to address timeliness. The goal here is to get to as close to real time as possible (this is going to take us a while but we are up for the challenge).

    I am surprised a spam acct got a score that high. It's too bad twitter killed the account so we couldn't investigate more. We are constantly looking at how spam behavior morphs and what the impact is on our score. This is another area we are putting a lot of effort in on. If you see any other accts like this don't hesitate to send them our way.

    We are a small team trying to tackle a huge challenge. It's early and we've got a long way to go but I do hope we can win your support as key metric you feel confident pitching to your clients.

    Me and the rest of the team are always available to chat.

    Thanks!

    over 1 year ago Tommy Landry responded:
    Tommy Landry
    I've been paying attention to Klout for some time now. Various other contacts have expressed similar concerns, but I for one, am cheering them on in hopes that they can reach their vision.

    This is not a simply problem, and the answer will be equally evasive to nail down. I do like that HootSuite began including Klout scores on profiles, and I've put that to use frequently since it was added. It helps stratify between the "serial tweeters" who never interact but simply post quotes or links, and the real people.

    Is it 100% accurate? Even the founder above knows it is not.

    Is it 100% better than having NO metric at all? Simple answer to me. Yes yes yes!

    Thanks to Klout for trying, and best of luck finding the end of the rainbow. I believe a true metric will make social media all the more manageable and rewarding.

    over 1 year ago Wesley Faulkner responded:
    Wesley Faulkner
    @joefernandez Joe, thank you for commenting on my post. I am happy to learn of your revamped version roll-out. As for the twitter account, maybe you could walk over and talk to them since y'all are in the same building. I'm sure they must have an achieve of that account.

    As for my support, you don't have to win that, you already have it. That's why I gladly sported your shirt in this post here: http://wesleyfaulkner.com/thank-you-klout-for-the-shirt

    What I would like help with is confidence in your service. I feel that you are on the right track, but are not there yet. I am more than willing to help you get there too. I was exposed to a twitter "bug" last week that showed more issues with your service an would love to share them with you. What is the best means for me to share this information with you and your company?

    over 1 year ago Albert Maruggi responded:
    Albert Maruggi
    while you all argue about al gore rhythms and calculation, etc etc, let me step back and say this about that. The idea of determine who is most influential is the antithesis of the social web's greatest attribute - community.

    We can argue whether the social web is leading a last century hierarchy model, or whether that model will not change. That model may simply be the way humans will associate for all eternity and therefore the social web as made it more complex to determine where individuals fit in the hierarchy.

    Or we can say, over time the idea of being influenced or being influential is no longer important in our formation of communicating. Look, the most important reason to rank individuals is for the commercial aspect of funding the social web. Either through advertising or allowing those highest on the rating scale to receive compensation for that influence.

    If that financial model changes then the need for such a system may well be reduced. I did a podcast with Meghan from Klout a while back and she explains how Klout goes about determining scores.

    PS: the people whom I supposedly influence have not changed in months regardless of whether I update my score or not.

    over 1 year ago aafromaa (Twitter) responded:
    Anne2011_normal
    I've have questions about the accuracy of how the Influenced by and Influenced of lists are generated. My Influenced of list has not changed in a long time---maybe never. It seems to me that this list was created when I first logged into Klout and it has not changed since. @pusser has tweeted only 8 times in the last year and did not mention me in any of them. @pusser has a Klout score of 11. @edoyle6 has not tweeted since 2008. Does Klout think that I have "influenced" her because I am one of 4 people she follows? How is it that her Klout score is 10.

    Interesting too, my score moved from 61, to 56, to 55 and stayed on 55 for the two weeks following the two weeks that I was quiet on Twitter. But, it jumped to 60 after a few days when I resumed more activity. The score is not immediate with increased RT and @s.

    While I have wonder about its accuracy, there is enough value in what it measures that I use Klout scores as a relative measurement. If Klout can clean up the dangling inaccuracies, the measurement becomes one tool of many that I will use, considering each one skeptically, within range of what it can actually measure, and what I think is important to be measured, and in comparison of others.

    Because of these inaccuracies of the listing of those being influenced shown on a Klout page, Klout has some credibility to build in cleaning those lists up.

    over 1 year ago Wesley Faulkner responded:
    Wesley Faulkner
    Thanks everyone for your input. Joe, the CEO of Klout, has been kind enough to grant me an interview. We did it on Monday and he had a lot of good information. It will take me a while to parse it out. It might even span multiple posts.

    That being said I found the following post about Klout and bots. It's a good read.

    http://www.wewillraakyou.com/2010/12/klout-is-broken/

    over 1 year ago Tommy Landry responded:
    Tommy Landry
    Just saw the following article on this topic and thought you might be interested: http://www.buzzstream.com/blog/problem-with-influence-scoring.html
    11 months ago Jeroen Fossaert liked this post.
  • Wesley Faulkner's Space

    Wesley’s experience spans multiple facets of the technology industry, from manufacturing to product development. His passion for technology was fostered through over thirteen years of experience in technical and customer service support, field engineering, and digital media entertainment development.

    Archive

    2012 (6)
    May (1)
    April (3)
    March (1)
    February (1)
    2011 (58)
    December (1)
    November (5)
    October (6)
    September (5)
    August (5)
    July (2)
    June (2)
    May (4)
    April (1)
    March (6)
    February (7)
    January (14)
    2010 (336)
    December (3)
    November (8)
    October (20)
    September (28)
    August (38)
    July (36)
    June (44)
    May (24)
    April (27)
    March (45)
    February (35)
    January (28)
    2009 (169)
    December (31)
    November (52)
    October (31)
    September (28)
    August (23)
    July (4)
  • About Wesley Faulkner

    Wesley’s experience spans multiple facets of the technology industry, from manufacturing to product development. His passion for technology was fostered through over thirteen years of experience in technical and customer service support, field engineering, and digital media entertainment development.

  • Subscribe via RSS
  • Sites I Like

    • Wesley Faulkner (@wesley83) on Twitter
    • Wesley Faulkner | LinkedIn
    • Wesley Faulkner - Google+
    • Wesley Faulkner (wesley83) on Pinterest

    Follow Me

      TwitterFacebookLaconi.ca/Identi.caLinkedInPlurkFriendfeedBuzzPicasaFlickrViddlerYoutubeVimeoTumblrDeliciousLivejournalBloggermetaweblog

Theme created for Posterous by Obox