|
Tuesday, January 13, 2004 |
Fear Factor
Fred Langa, in InformationWeek, tries to run a test of email reliability and claims that 40% of his email did not get through. He emphasizes how he had thousands of volunteers participating in the test. Sounds good, yesno?When you read through to the mean of his article, though, it turns out that up to 40% of his eager volunteers did not successfully respond to an email from an email address they didn't know. Right off the bat, this is flawed: he's not actually testing if email gets delivered, he's testing if a random group reads unsolicited emails from unknown addresses, and if they then actually successfully respond to it.
Additional problems: Langa claims that "The subject line also would be plain and general, neither designed to trigger nor avoid spam filters. " but then goes on to say that "The E-mails carried a totally generic subject line of 'Hello'."
Many people routinely delete email with subject lines like this - I've got six emails in this morning's "Junk email" folder with the subject of "Hi" or "Hello" out of the hundred or so pieces of spam I got overnight, and I deleted all of them unread. If I don't recognize your name, you're probably a spammer. Then, to reply, you must follow his directions:
Please do not--repeat do not--simply reply to this E-mail! Instead, please create (or forward) a NEW E-mail to [target address was placed here]. Again, please do not--repeat do not--hit "reply." That will not work.
So now, we've got a case where he's introducing a new source of failure into the design, without accounting for it (for example, by trying to send an email from his Fred Langa address and seeing the comparable compliance rate with his directions).
Years ago, I TA'ed experimental psych for three semesters. Anyone who brought this design and methodology into class as a proposal would have received a very poor grade. Why Information Week chose to publish it is beyond me (except the obvious reason that many publishers believe that "Fear Sells", and don't care about the actual truth of what they publish.
