Sunday, May 4, 2014

What Mark Higginson's Sparksheet Article Got Wrong

Coca-Cola Journey homepage
Last February, Mark Higginson, a social media manager at a university in the UK, wrote an article for the marketing publication Sparksheet challenging Coca-Cola’s content marketing strategy. I lead the group that publishes Coca-Cola Journey, the subject of Mark’s piece.

For his analysis, Mark counted the social shares that are visible on every page of Coca-Cola Journey, and concluded that because social sharing was (in his opinion) low, “engagement” was also low, and that Coke's content marketing program is pointless. 

To prove his point, Mark looked at more than 80 randomly-selected posts. He found that some of Coke's content is a hit with readers, a few pieces are mega-hits, and that many posts aren't shared that often. In his view, this is proof that Coca-Cola's investment in real content is a large-scale, expensive failure.

Unfortunately for the reader, there wasn't any comparison to other publications to validate this hypothesis. Instead, readers were left to assume that the data proved the point.

So I decided to do the comparison myself, using Coca-Cola Journey, Advertising Age, the largest trade marketing newspaper in the United States, and Sparksheet. I selected May 4, 2014 as my research date, and used the "top" stories on each magazine's website as my sample set. I did this because I believe it's fairer to compare the best content available on a given date than to use randomly selected articles (this also avoids selection bias; I've let each site select their content for me).

First, social shares on Coca-Cola Journey, and its sister site Coca-Cola Unbottled, compare quite favorably to other marketing publications. The ten most popular posts have an average of 448 social shares per article. Unlike Mark, I also found the spread of shares across the articles to be fairly even: only one article has less than 100 shares, five have between 100-500, and four have more than 700.

Then I examined the top six posts on Advertising Age (they only list their top six articles). The average for these top posts is a massive 9,288 shares. But a closer examination reveals that this article (with 53,000 social shares) is the reason for the exceptionally high average. If we exclude this outlier, the Ad Age average is about 398 social shares per article. Like Coca-Cola Journey, shares on these posts are fairly evenly spread: one has less than 100 shares, one has more than 1,000 shares, and the remaining four average under 500.

Sparksheet doesn’t have a “Top Ten”, so I took a look at the nine “Editor’s Picks" on the front page. Together, these nine articles average about 110 social shares per article, well below the average for both Coca-Cola Journey and Advertising Age. 

In addition, Sparksheet also suffers from the "feast or famine" social sharing Mark finds problematic. All but two articles have less than 100 shares. The top piece has 334 shares and the second-highest shared piece has 267, well over half of all shares in the editor's top nine. In fact, all but two of the Coke Journey and Ad Age top pieces score higher than seven of the nine Sparksheet articles.

By Mark's reasoning, Sparksheet is talking, but no one is listening.

Distribution of Social Shares, Top 5 Articles

Based on the high level of interest in Mark's post, this isn't the case. Clearly, social shares alone are not a good marker of engagement or influence. 

All publications, even top newspapers, have some content that wins, some that's just average, and some that's not as popular. Coca-Cola Journey, Advertising Age, and Sparksheet are not exceptions, and that's why high-performing content strategies aren't based on virality. After all, Mark's article has been shared 519 times - a good performance for Sparksheet but only about average for a top trade magazine. Yet it generated thirty thoughtful comments and fifteen trackbacks, surely signs of true engagement.

Clearly there's something here worth exploring: what are good markers of ROI for brand journalism?

In the next installment, I'll explore what I believe are a few excellent standards for measurement. Until then, feel free to share.

UPDATE: A reader correctly pointed out that Coca-Cola Journey also features "Editor's Picks". To keep the playing field level, I have tallied their shares as of May 5, 2014. The average number of shares for this group is 274 social shares per article, still well above the average for the similar set of articles on Sparksheet. 

This is the first installment of a series on measuring the ROI of content marketing.

11 comments:

  1. Dear @ashbrown77

    Thanks so much for this blog entry and your great feedback on our blog entry regarding Coca-Cola's social media efforts: http://commetrics.com/articles/how-to-create-better-engagement-1/ (click to read)

    For me sharing stories posted online in a blog on Facebook, LinkedIn or Twitter is fine. Nevertheless, we know that people can share things automatically - and they do - without having read the content prior to sharing.

    Accordingly, they just broadcast your content without having it studied before they shared it (i.e. with some programs we can easily make this happen automatically, while one sleeps :-) ).

    However, the bottom line is if people really engage with your content. by commenting. This requires one to have read the entry first before commenting.

    === http://blog.drkpi.com/blog-influence-1/ click to view

    However, writing a comment that adds substance to a blog entry requires that you have read the blog entry beforehand.

    If we check with Coca-Cola's online magazine or the blog (actually, I am not sure why it is called a blog), both get few reader comments. As well, authors do not reply to reader comments.

    Hence, there is really no conversation happening and reader engagement is very low.

    I am not sure if Ad Age is a good example to compare to. Maybe another corporate blog would be better for comparing oneself against?

    Ashley thanks so much for sharing.

    Cordially

    Urs

    PS. Here you find some data about your blog: http://DrKPI.com/rank/*/R/www.ashbrown.org

    ReplyDelete
  2. Urs,


    Thanks for your comment. We've had more than 20,000 user comments on Journey in the first year, which we're very happy about. We can't respond to them all (for resource reasons), but think of comments as an opportunity for fans to engage each other. Coke doesn't believe that it has to participate in every conversation. We're happy providing the forum.


    All of this is, of course, not directly related to Mark's thesis - which was that our content isn't shared. I've proven that it is.

    ReplyDelete
  3. Dear @ashbrown77

    Thanks for replying again. You state:
    which was that our content isn't shared. I've proven that it is.

    I am not sure if proven is the right word, you may have shown that some content is shared. But how much this is in relationship to blogs from other corporations is not clear.
    Put differently, our stats from FT Global 500 shows that it is not.... about average. That is probably good enough. So keep on plugging.
    Cordially
    Urs
    @drkpicom

    ReplyDelete
  4. Ashley,

    I love this piece. But I may be even more excited that AdAge’s most shared post features the splits, Volkswagen and JCVD – at the very least you’ve proven the internet has no immunity to this trifecta of awesome ;)

    Sparksheet is a very small media brand. Compared to the budgets and reach of Adage and Coca-Cola it only makes sense that our numbers would reflect our size (though there are plenty of counter examples showing size isn’t universally restrictive).

    But you didn’t write this post to out our numbers. You’re speaking to the much more interesting and important question of measurement – what metrics or standards should content marketers use to justify the value of their strategies and their content? I’m not convinced we’ll ever arrive at a universal law, but it’s why I find this conversation so compelling. It’s poking at content marketing’s sensitive bits. It’s getting to the heart of what branding is all about.

    At Sparksheet, we try to publish pieces that contain our own
    trifecta of awesome: Thoughtful, provocative and novel ideas. That’s our benchmark. It’s completely subjective, it’s difficult to measure, but it suits us.

    Looking forward to reading about your standards – and thanks
    again for all of your insightful comments over on our humble blog. Conversation at its best.

    ReplyDelete
  5. Thanks for commenting Sophie.


    As you point out, i wasn't taking a jab at Sprksheet. Actually the reverse - the high levels of engagement on your publication prove the opposite of Mark's thesis. Sparksheet is proof that social sharing alone is not an indicator of true reach, engagement, and influence (and I can assure you, Sparksheet is influential - at least within Coca-Cola).


    I wrote this piece because as marketers in a fast changing space, we spend an extraordinary amount of time and energy beating back shiny objects. Before we let false narratives like Mark's take root, I think it's important for subject matter experts to set the record straight with actual, real data.


    Thanks again for commenting on my blog. I love your magazine, and am a big fan of your work. Keep it up.

    ReplyDelete
  6. Hi Ashley! I didn't say that 'Journey' was unsuccessful compared to sites for whom publishing is their core business. I said: "what’s true for publishers is true for one of the biggest brands in the world..." that is to say we are all at the mercy of the power law effect that governs the complex network that is the web. EVERY site suffers from 'feast or famine'. Your statement "all publications, even top newspapers, have some content that wins, some that's just average, and some that's not as popular" is incorrect. It's all or nothing. As I commented on a recent piece by Joe McCambley you need to understand the Barab├ísi–Albert model.

    Have a look at this chart: http://imgur.com/i0tJEXo,rgDs1Pi

    It compares Facebook shares from 90 Journey posts on the primary axis and Facebook shares from 53 posts for Imperial College's news items on the y-axis. Two completely different sets of content. Two completely different levels of sharing. But the pattern when overlaid is identical. If you understand the implications of this you understand my argument.

    I also looked at AdAge as you used this as an example: http://imgur.com/i0tJEXo,rgDs1Pi#1

    It compares the same 90 posts from Journey against the total shares from 76 posts from AdAge occurring over the last four day period. Again, the same pattern.

    Your decision to look at the 'top' stories doesn't work for this exercise as the sample set is too small and is only selected from the 'head' of content being produced at a given time. This is not avoiding 'selection bias'; that's not what the term means.

    Based on what you've said I imagine your comment will be that Journey is performing 'as well as' AdAge. This is of course not the case. Publishing is AdAge's business. They produce many times the number of posts as Journey, thus overall sharing will be much higher if all the posts were counted. Additionally they 'speak' to a specific industry. However Coca-Cola, who do not derive their revenue from publishing and instead serve 1.8 billion beverages PER DAY don't see this colossal reach turn into significant interaction online.

    Same goes for Sparksheet, they may be smaller but they are tightly focused. I'm not the world's biggest drinks company yet my post on Sparksheet would have been the 12th most shared out of the 90 Journey posts I looked at in my original piece. My wider point was that while Sparksheet gets this for little overhead the cost to Coca-Cola of running Journey must by huge, especially with a plan to roll-out this format in 30 countries. That's why I don't see a point to it. What on earth is the benefit when set against the cost?

    I hope these charts make the effect I'm outlining clearer. It applies across the board, be it page views, incoming links, social shares, etc. and is a fundamental rule. What I describe is not a "false narrative" but is actually one of the most widely cited academic papers in the field of network science. The fact it invalidates the cost / benefit equation of the entire content marketing industry and most of web publishing is an inconvenience marketers ignore; most aren't even aware of it: https://twitter.com/KirkCheyfitz/status/459070028730466304

    ReplyDelete
  7. P.S. Please do set the record straight by publishing a full set of data from Journey and we'll see who the subject matter expert is!

    ReplyDelete
  8. Mark,


    Your argument is actually becoming less clear.


    The hypothesis that social shares are a marker for engagement is flat out wrong. You didn't touch on this above, so I assume you now accept that to be true. This was the point of your Sparksheet post.


    The crux of your comment here is that the investment we are making in storytelling and content can't possibly be worth it. But that's an assumption you make without knowing any of the inputs required to make the case:


    a) you don't know how much we're spending;
    b) you don't know how much we spent on our "old" static corporate website (to determine whether the upgrade is worth it);
    c) you don't know our content strategy;
    d) you don't assign any value to search and the long tail; and
    e) you don't know much we would spend to reach these same people via other types of media (be it TV, print, social, etc.).


    The last two are especially important, along with an understanding of how much we value and invest in PR (which is key to storytelling).


    Even if you posit that content marketing is just advertising (which would be a total misunderstanding of the point), without knowing (e) you can't determine if it's as efficient as other forms of paid media (hint: it is).


    You keep mentioning the Barabasi-Albert model - which has nothing really to do with the argument that started this all. The model, as I understand it, concerns the size of nodes and the preference for new entrants to favor large, existing nodes. The relevance or value of those nodes are not examined. In this discussion, the value of nodes is *everything*.


    Finally, Coca-Cola Journey is a destination, and an increasingly popular one. Our traffic is up more than 30% YOY. As social networks throttle organic views, my prediction is that owned media will feature more prominently in brand marketing strategies. But, hey, what do I know ;-)?

    ReplyDelete
  9. Mark @Mark Higginson

    I love your ideas and have used some of them in one of my blog posts. Nevertheless, after having read the above, I have to agree with @ashbrown77 ==> your argument is becoming less clear to me.

    If you go to http://liorpachter.wordpress.com/2014/02/10/the-network-nonsense-of-albert-laszlo-barabasi/ you get some links to various papers that clearly refute the models applicability to all types of networks, including social networks, loose ties, biological applications and, yes, publishing.

    So my question is where you tested with your data that the Barabasi model applies. That would, of course, suggest you share the adjustments you made to these equations. Until then, it remains a mystery to me.

    Thanks so much for sharing.

    Urs
    @drkpicom

    PS. the original model is described in this paper available for free download: http://arxiv.org/abs/cond-mat/9910332

    ReplyDelete
  10. P.S. One more thing you did not control for - age. It's unfair to compare articles published yesterday, with one that is 3 years old.


    This is one reason why I selected only the "most popular" content on each site. AdAge and Sparksheet have been publishing for years; Journey isn't even a year and a half old.


    The beauty of search is that older content is infinitely findable and shareable. Some of our most popular content today was published months ago.

    ReplyDelete
  11. That model is the basis of prior posts that led to the invitation to write for Sparksheet so is integral to my view of how attention flows on the web. I think we're just going back-and-forth here, in part due to us being at the mercy of the comment format.


    It's probably more likely you'll be in London at some point than I'll be in Austin so how about you drop me a DM if you're ever in town and we can meet and discuss for an hour over coffee? I can explain a little more about how I'm applying this in the context of the university community I work with and you can tell me more about how you are getting 'buy-in' within your own large organisation as I'm sure this is a challenge we both share even if we have differing viewpoints.

    ReplyDelete

Read Another Post