Monday, March 07, 2005

Email from Elizabeth Albrycht regarding research transparency I think we have to distinguish here too between the analyst him/herself and the organization they work for.  I can't tell you how many times I have been on the phone with analyst/executive and the analyst rather sheepishly says - "Oh, I really need to give you the sales pitch now." Or something along those lines, intimating they are forced to it by "management." I question whether  research associates are the best people to give sales pitches.  Those two skills don't always go together! Many times, for clients, our meetings with analysts are two-fold.  To get on their radar screen, and to evaluate whether we want to pay them for future engagements.  The sales discussion has always been some part of the negotiation.  But, over the past years, the balance of power has shifted to the sales side.   I agree that analysts should not be seen as objective or independent, when you have to pay to play. Aside:  I work with Gene Signorini of Yankee Group, and now Nick McQuire, through the Mobile Enterprise Alliance and I have been impressed by both of their smarts.  This has been mainly true over time of Yankee analysts as a whole.  My original rant referenced above comes from Oct. 2003.  Maybe their scheduling system has changed since then; I haven't used it in awhile.  Gene and Nick are members of the MEA's advisory board and I don't need to go though the scheduling system to talk to them. 11:10:26 AM    comment [] RadioEdit

Research outfits outted? If you've been following this transparency channel, then you may have seen how I just took the IT research community task over its lack of transparency. For obvious reasons, I wrote this from a reporter's perspective. I just got done reading Elizabeth Albrycht's perspective from the public relations side of the equation and all I can say is "Someone get me the Pepto please." Give it a read and you'll see why it just makes you ill. The most disturbing quote from her posting -- one that may corroborate the tainted process that I read about in the InternetAcceleration newsletter -- talks about how analyst briefing requests are held hostage for a ransom of a sales pitch. Albrycht has no qualms about identifying who engages in what practices: Then there is Yankee. You now have to talk to a sales person before you are allowed to request a briefing. You have to listen to their pitch before you get to climb Olympus and talk to the ever-quoted Zeus. The implication from Albyrcht's account is that if you're a vendor that's not a customer of a research company, then to get on an analyst's radar (an analyst that the rest of the world trusts to have objectively surveyed the entire landscape by the way) requires a sales pitch first. Imagine if this was standard practice in the press too? Holy cow! Media company's would be making money hand over fist. Thankfully, it's not. At least not where I've worked. All vendors have an equal shot at me. I'm not saying that I'll write about every one, or even take every briefing that's tossed my way. That's physically impossible. But if my sole purpose of being was to offer exhaustive research on some vertical category in order to empower buyers to make informed decisions, then I'd want to hear from every single player in that category on a regular basis and I woudn't want my sales department standing in the way of those briefings. But instead, we have eyewitness accounts from outfits like the InternetAcceration newsletter that are disturbing at best when it comes to the implications to a research reports objectivity and thoroughness: We asked a few of the vendors listed in the [Magic Quadrant] how much effort Gartner put into its evaluation (did Gartner contact them, interview reference accounts, talk to investors, business partners, review "bake off" findings...) and we were basically told that, from these vendors' perspective, Gartner had done very little. Here's a shocking example of what one vendor told us: - "we spent about a half hour with ... [Gartner Research Vice President Name Deleted] ...of Gartner almost exactly one year ago. We gave him a brief overview of our company and exchanged pleasantries. As I recall the subject of a 'paid relationship with Gartner' was raised more than once." Now, before we go casting the whole lot as a bunch of crooks on the take, I'd like to point out a few things. First, research companies, like all other companies, can't exist without revenues. And given that their research is useful to the buyers of IT as well as the vendors who are featured in it (eg: for competitive analysis), it makes perfect sense for research outfits to pitch both communities on research provision. But, like the press, the #1 asset that a research outfit has is it's credibility. If it loses that, it loses everything. So, in the name of credibilty, great care must be taken not to poison the well. As a researcher, I'd want to spend a few hours per quarter (or more) with each of the companies in my beat and, in the name of revenue, I think it would even be fair to let them know that my research results are available to all parties on the same terms and who the contacts are if they're interested. But I would also engage in complete disclosure. First, I'd have a publicly available disclosure that describes how I go about guaranteeing objectivity and comprehensiveness. In it, I'd mention what my company's business model is and how I occasionally discuss the availability of my research with all potential consumers but that it's more of an awareness thing than a hard sell and that it in no way impacts the outcome of my reports. Then, in my reports and any summaries of them that get republished by other entities (charts for example), I would be very clear about who in the report is a paying client of the company, and who is not. Playing a role in sales, making money, and having a financial relationship with the companies or people that a researcher researches isn't the mistake. The mistake is in not disclosing everything there is to disclose about how the process works and where credibility could be called into question should the nature of certain relationships that aren't otherwise disclosed become public information. With the blogosphere, it's only a matter of time before more of these relationships and processes are outted and certain research outfits are put on the defense (never a good position to be in). Better to go on offense so that by the time a question comes up, the world has pretty much already established a comfort level with how things work. The bonus is that, just like with journalists, when researchers know that full transparency is in effect, they have no choice but to work extra hard to be objective and avoid any misperception of bias. The resulting research can only enhance a research outfit's credibility and revenue prospects. Update: A person claiming to be David Scott Lewis has responded to my original post on researcher transparency with his own first hand accounts and uses the disturbing verb "Aberdeened." I've never met Mr. Lewis, can't guarantee that it was him that made this post (if you are him and want to "authenticate," please e-mail me), and I cannot vouch for the authenticity of his comments. But, if for no other reason than the fact that his storytelling shares some commonalities with other accounts of shameful behavior, it's worth a read. 8:10:06 AM    comment [] RadioEdit