Who's Checking the Numbers?

Who's Checking the Numbers?
by Dave Morgan, Tuesday, Jun 7, 2005 6:00 AM EST
BUSINESS WEEK BLOGSPOTTING'S STEVE BAKER made a great point last week that blogs are bringing math into journalism:

He opines that the open publishing platform is giving media voices to non-traditional journalists, like technologists and mathematicians who have very strong math skills and are bringing with them some much-needed quantitative discipline to the profession.


As someone who works in online advertising, an industry that lives and dies by numbers, Steve's point really hit home with me.


The online advertising world is awash with numbers, and we need more people covering that world who can make sense of them.


An enormous advantage presented by the growth of digital computing is their acute measurability. You can generate, analyze, and report numbers on almost anything related to digital media and marketing these days. Unfortunately, while this wealth of numerical data in the right hands can be powerful and liberating, it can also be overwhelming and quite misleading when misused.


It is virtually impossible to consume industry information and communication these days without encountering numbers, whether it is a news article in one of the trades, or a corporate press release, or a presentation at an industry conference. Numbers are everywhere, and usually, by their sheer specificity, they tend to carry a lot of weight. But should they?


All too often, numbers are released, reported, and accepted as gospel because no one in the audience is either knowledgeable or comfortable enough to ask the right questions. During the Internet Bubble years, everyone was making pronouncements about the size of the online ad marketplace. Every forecast got bigger, and no one questioned them because everyone wanted to believe. As I recall, one of the first attempts to call the numbers into question occurred in 1998 and was led by BURST! Media's Jarvis Coffin.


Uncomfortable with seemingly inflated numbers coming out of the portals, ad tracking services, and trade organizations, BURST! sponsored original research that analyzed that actual financial discloses of advertisers and found that their online ad spend was being inflated by more than 50 percent. While many in the industry dismissed his research as wrong and "contrarian" at that time, two years later, once the billion-plus dollars in "roundtrips" and fraudulent bookings at places like AOL and Homestore.com and dozens of others came to light, it became clear that he was right, and that the industry had been living with bad numbers. In the end, we lost a lot of credibility that it has taken the better part of 5 years to rebuild.


We need to be better prepared this time to have our numbers scrutinized by experts. For example, I, like others, was quite happy to see TNS Media Intelligence recently release their list of Top Internet Advertisers for the month of April together with the estimated monthly spend for each advertiser. Having data like this available in the market is a true benefit to all.


Once I looked at the actual numbers, however, I was not so sure that they were doing anyone a real service. The number one reported advertiser, "Tickle by Emode," was estimated to have spent $23,770,000 on online advertising in the month of April. Number two was Vonage at $21,829,000 for the month. Number three was South Beach Diet at $18,607,000. Number four was Lowermybills.com at $14,124,000. Netflix.com was number six at $12,508,000 and Classmates.com was number seven at $10,137,000. Basically, this report says that these companies had online ad spend in the month of April at a $100,000,000 to $250,000,000 annualized rate each.


I don't believe it. Does anybody else?


I don't believe that any of these companies spent anything close to those numbers in actual dollars in the month of April. I think that all TNS did was take some gross ad delivery numbers and multiply them by some estimated generic rate card numbers (which, as anyone in the industry knows, does not exist) and published the product of those numbers. They totally missed the fact that a large portion, and probably the majority, of all online ads are sold on a cost-per-click or cost-per-action basis. Their disclaimer (presented with the numbers) that "Media expenditures do not always take into account special considerations including publisher discounts, barter agreements, co-sponsorship, affiliate relationships, etc." misses all of that.


It is not my intention to single out TNS here. My concern is that numbers like these need to have their methodology called into question. They carry a lot of legitimacy because of their sponsor and their specificity. We need people that know the questions to ask and who can analyze and interpret the results. I hope that Steve is right. I hope that more mathematicians are coming our way.


Dave Morgan is CEO of Tacoda Systems.