What to Measure? The Question Journalism Has to Answer

How should we measure articles' success?

offthemedia-1A few months ago I wrote an article for a website where the standard agreement for writers is a bonus incentive on social shares for the article. This was both usual, and unusual for a lot of reasons.

Sign Up For Our Daily Newsletter

By clicking submit, you agree to our <a rel="noreferrer" href="http://observermedia.com/terms">terms of service</a> and acknowledge we may use your information to send you emails, product samples, and promotions on this website and other properties. You can opt out anytime.

See all of our newsletters

First, most websites don’t pay writers anything. A good portion of writing online is done for exposure (which a lot of people laugh at but content marketing can be hugely lucrative and I encourage my clients to do it). So that was slightly unusual. What was more standard was the fact that for a site that did pay, the payment was partially contingent on page views (there was a bonus for how many social shares the article got).

Slightly more unique, I am in a financial position where because of my books and other income, I did not need to bother with such a bonus and could refuse—the standard, token payment for the article itself was plenty fine and plenty motivating. Why would I write something if I didn’t want lots of people to see it and like it?

That article did something like 5,000 Facebook shares—a good portion of them because I worked hard to promote it. Now that I am working with Betabeat, I’ve been thinking about all this quite a bit. Because it’s a dilemma every writer is thinking about—as well as every editor tasked with growing an online property.

Though I could paint it otherwise, the site was not trying to corrupt me with some compromising offer. Nor was I some moral paragon for refusing it. After all, my posts at Forbes resulted in some nice little checks due to the per-pageview rate they charged. My contract with Betabeat also includes traffic goals and rewards tied to them.

 When I see Marc Andreeseen’s intelligent critiques of journalism, particularly the bad metrics of current journalism, l agree. But the other side of me thinks: well, what the hell else are people supposed to do?

What am I—or any publisher—supposed to measure then? How should media outlets in 2014 determine what “success” looks like and create employee compensation parameters around it?

If pageviews or uniques aren’t it, what should it be? Gut instincts? Length? Comments? What?

Just about every metric you can think of has a downside. Longreads, as other writers have pointed out, have raced past reasonableness and into self-indulgence. Comments—the lack of or the existence of—don’t prove anything as Warnock’s dilemma states. Social shares? That creates Upworthy. Subscribers? I’m not sure if the causal relationship is clear enough–plus I know a lot of crappy creators with big email lists.

It’s the lack of a clear, easy answer to these questions that has us in the state we are currently in. As much as I’ve criticized it, I empathize with the quandary. Page views are easy to track. Quality is harder. It’s not only relative and subjective, it takes time and resources that many people don’t have. It’s easier to fall back on “whatever everyone else is doing.”

I’ve used this analogy before but here it goes again: We can all admit that “bodycount” is a pretty terrible metric to use in a war–one responsible for all sorts of calamities–but what’s a better one? What’s one that’s not only better but more practical than this obvious and straightforward heuristic.

Because it’s real easy to criticize, but hard to come up with a genuine alternative

Maybe step one is to start having this discussion publicly–to start figuring out and hearing from readers about how they want it to go. I’m not talking about what they’re clicking, that is notoriously inaccurate (as Henry Ford said, if he’d listened to customers he’d have invented a faster horse). I’m talking about eliminating the opaque veil that hides how the business works.

When the public understands the economics of the information, they can begin to understand why the information is how it is. They can also make decisions about who they choose to read and why, just as they might pick the ad-free Consumer Reports over the potentially biased Car and Driver. Right now it’s the decisions being made at BuzzFeed and Upworthy and Gawker that are driving how our information is produced, instead of the values of readers and the public.

I think publishers and bloggers are ready for an alternative. They are tired of being the shitheels of society again, just as their colleagues a hundred years ago were. And I wonder if that is not the ultimate solution: principle and shame.

Andreessen posited that quality came to the news industry as it was taken over by “oligarchies.” Rich families care about money, but they also care about how they look to people in their social circles. They have connections and relationships that they value in addition to the raw bottom line. I passed on that bonus for that exact reason. That came from a privileged position–which not everyone is able to do.

Maybe we need more of that. In a hyper-connected world where relationships and networks matter more than ever, we should consider incentives—and a culture—that makes people think twice before making decisions that externalize bad outcomes on the consumer. 

What to Measure? The Question Journalism Has to Answer