Updated: 4/1/2005; 9:35:31 AM.
Berlind's Media Transparency Channel
If you're looking for my podcasts, please read What to do if you're looking for my series of podcasts on IT Matters. Otherwise, read on.

This blog is now a part of my experiment in media transparency. The premise is that if the media can broadcast polished edited content through one channel like ZDNet, then why can't it also broadcast a parallel channel that's full of the raw materials (thus, this "channel"). For a much more detailed explanation, be sure to check out the following:In case you're interested, maintaining a simplistic transparency channel like this one has so far involved a significant amount of heavy lifting. The core technology may exist, but it's my opinion that a decent UI for publishing a transparency channel does not. So, one outgrowth of this experiment might be a complete specification for such a system -- Something I call JOTS. Finally, as a student of media, convergence, and technology monoculture (three very inter-related issues, if you ask me), I'll be blogging any news that comes my way that I think is relevant to the media revolution that's upon us (the one that many media executives are in obvious denial about).

Friday, March 04, 2005

So, I think we're in agreement that the media needs some transparency. And based on what I see being written elsewhere, some PR transparency appears to be on order as well. So, what about research? In our industry -- the tech industry -- if there's a part the business that desperately needs more transparency, it's the research part. I was reminded of this today when I was forwarded an e-mail newsletter known as the InternetAcceleration Newsletter from InternetAcceleration.com. In the February 22, 2005 issue is a segment called There they go again. In saying "We think that the current quality of output in its Magic Quadrants are both potentially misleading to IT buyers and an abuse of its brand," the report has some harsh words for tech research outfit Gartner who owns the Magic Quandrant brand. In fact, so harsh is the newsletter in levying accusations of impropriety (could this end up being a transparency test), that I can't help but wonder how Gartner cannot react. If Gartner sues for libel, then we get to watch Gartner's image go on trial. If Gartner does nothing, what are we to think then? The report goes on to say:

"Here's a shocking example of what one vendor told us: - "we spent about a half hour with ... [Gartner Research Vice President Name Deleted] ...of Gartner almost exactly one year ago. We gave him a brief overview of our company and exchanged pleasantries. As I recall the subject of a "paid relationship with Gartner" was raised more than once." That wasn't the only response like that. This situation turns out to be worse than we thought. It appears to us that not only is Gartner clearly doing insufficient research (hadn't even spoken to a vendor listed in the MQ for a year!), but that Gartner Vice Presidents are using vendor briefings as thinly disguised sales calls. It's pretty intimidating for a vendor to be asked to sign up for research services IN THE SAME discussion as they're providing input used in a MQ ranking and some may say unethical."

Whether or not these stories about Gartner have any merit remains to be seen (or perhaps we'll never know). But there are many research outfits in the tech business and in my discussions with certain marcomm pros that I've know for years and trust, there is no question in my mind that these sorts of shennanigans are taking place behind the scenes. While I'm not going to call anybody out for a bar room brawl, the newsletter reminded me that research transparency is definitely a discussion that needs to be had. For example, when presenting scoreboard like research like Gartner's Magic Quandrants, shouldn't the charts say which of the companies listed in the chart are also Gartner clients? Or how about when the press gets pitched on "new, earthshattering" results as a proofpoint of some vendor's leadership?

Case in point. Recently via e-mail, I received a copy of a press release from Check Point Software Technologies that says:

Check Point Software Technologies Ltd. (NASDAQ: CHKP), the worldwide leader in securing the Internet, today announced that recent independent tests conducted by The Tolly Group confirm that Check Point provides the broadest breadth of coverage and the lowest Total Cost of Ownership (TCO) for today’s complex security vulnerabilities in comparison to Cisco Systems, Inc. (NASDAQ: CSCO) and Juniper Networks (NASDAQ: JNPR).

First, if you're a public relations professional, please listen to what I'm about to say: I don't know about other journalists and I know you're just trying to be helpful, but when you pitch me on customer success stories and research-based proof-points, my spider senses start tingling. Call me fickle, but the last people I'm going to trust to provide me with objective information about your products or services are the people you're recommending to me. Think about it. What are the odds that you're going to furnish me with a contact that has something really bad (read: objective) to say about the product your pitching. Would I really doing my readers a favor by relying on these sources as authorities?

Second, this was one of the first research-touting e-mails that I've received since starting this transparency channel and as I read it, I realized that, in the name of transparency, it's time to start asking questions. And I did. In this case I asked the following question "Was the study by the Tolly Group commissioned by Checkpoint?" Answer? "Yes, the Tolly Group study was commissioned by Check Point."

I'm glad the public relations professional was honest. That was a good move. But, for the research community, this puts the transparency issue at front and center. Although I'm an ex-testing lab director, I can't for a minute make a qualitative assessment about the Tolly Group's research methodologies. I haven't seen them. But I can say its time to insist on several layers of transparency.

First, any pitches by vendors or public relations personnel to the press, analysts, or customers that cite research must absolutely disclose any relationship that those entities have with the provider(s) of the research being cited. Second, there needs to be a review and consensus of what language can be used in these pitches. In the aforementioned pull-quote from the press release, the Tolly Group's tests are characterized as "independent." Now, perhaps everyone who signed off on the press release including the folks at the Tolly Group have their own definition of independent and they're certainly entitled to believe that. But, in my book, when the cited research is commissioned by the vendor that's pitching me, it doesn't pass my test for indepedence. It doesn't even come close. Third, as I said earlier, all published research should be accompanied by disclosure of the client relationships that are relevant to that research. If such relationships exist, then I think that calls for an additional layer of transparency -- one that discloses whether the vendor was exposed in anyway to the research methodologies prior to the start of the testing or research. If so, then the nature of that exposure must be detailed. Was the vendor allowed to provide consultative input? Was the vendor given any veto power over certain evaluation criteria? If a comparison was involved, did the vendor have any influence over the competitive set? I'm sure there are other questions to ask -- questions that make it possible for consumers of any research to make their own judgements about the results being shown to them (very often to help turn a prospect into a customer).

I think you get the drift. Today, you'd be hard pressed to find any of these practices in use and there's a reason for that. It's one of this industry's ugly, dirty little secrets and it's time for a change.

Update: Be sure to read the comments on this blog entry as well my interactions with Elizabeth Albrycht.   How many more "cathardic" postings from insiders (former or present) will it take before the issue snowballs into something bigger?

Finally, I know it has been a while since I last posted, but it has not been a while since I've been working on this transparency channel. As you can see by my expandable blog roll on the right side, I've been experimenting with OPML. I've also been spending a lot of time under the hood of Radio Userland, trying to figure out the best ways to connect my e-mail system to it in a way that turns Radio into a transparency content management system, something that would serve as the underpinnings of my evolving JOTS specification.

10:55:27 PM    comment [] RadioEdit

© Copyright 2005 David Berlind.
March 2005
Sun Mon Tue Wed Thu Fri Sat
    1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31    
Feb   Apr

 New Media Thinkers
 Media & PR Transparency
 Vendor Blogs
 Tech Guru Blogs
 Tech News Sites
 ZDNet Blogs
 Other Cool Peeps

Click here to visit the Radio UserLand website.

Subscribe to "Berlind's Media Transparency Channel" in Radio UserLand.

Click to see the XML version of this web page.

Click here to send an email to the editor of this weblog.

Technorati search

Top 10 hits for media transparency on..

Help link
 4/1/2005; 9:34:40 AM.

Categories and Current Editorial Projects*