Starting this week I'll be cutting down on blogging. I won't blog off completely, but for a while I'm going to need to devote fuller attention to the problem of, as they say, "securing means of livelihood". So I'll be taking a step back from this infinite game and try to win a couple of finite games for a change. Expect updates once a week, either on Saturdays or Sundays.
At the same time this will let me find out just how much I'm addicted to the stuff. Let's hope the withdrawal symptoms won't be too severe.
In the last post I used googlefight to decide whether to write "traveling" with one or two L's. Single-L wins by a small margin. I often look up spelling like that manually, so it's nice to get an automated version. (Of course, if the majority of pages make the same spelling mistake I'll do it too, but who'll notice?) What do you think? [] links to this post 10:31:06 PM
Making communities of practice fly. Diane Le Moult has written an excellent summary on how to make CoPs work. This is written from her direct experience, and highlights a number of very useful guidelines: 10 fundamental questions you need to ask before starting a CoP: [...]
[...] Finally, we identified 10 classic pitfalls you have to be aware of:
Ignore moods and demands of members: People participate primarily for themselves, not for you or an executive demanding certain results. Therefore always have an open ear for the members, motivate them to shape their community.
Not enough content: You have to reach a critical mass which differs. If there is not enough interesting content, people will work less in the community, contribute less. That´s a real vicious circle!
Too strict or too loose: People need leadership but don´t want to be cramped. Due to the voluntary character of communities, finding the right way is a challenging task.
No scope: People need room for innovation and creation but also noticeable landmarks for orientation.
No aims: Communicate your estimated aims and outputs and be open to discuss.
Only technical platform: „First invest in travel and in beer, then in information technology“ (from EFQM Benchmark KM).
No Admin response: Assure that people are heard when they have problems and get useful answers.
No support (help and training): Effectiveness needs constant support and trainings - especially for the key members.
Only extrinsic motivation: It is impossible to achieve quality results when the members don´t have a natural interest and need for these.
Bad moderation: Even the best experts need qualified moderation and facilitation.
This article underscores several important issues about the interplay between research and publication.
On the web newsgroup, Greg Kuperberg, a math professor at the University of California, Davis, says: "Academia is playing a double game with journals. On the one hand, serious researchers know it doesn't necessarily mean anything, if a paper gets published. On the other hand, the promotion system and even the job market treat refereed publications as tangible achievements."
As a result, the career prospects of a scientist with scores of hack publications are far better than those of one who publishes a few quality papers.
To make matters worse, refereeing is anonymous, unpaid work, and busy scientists often don't have time to do more than glance at the papers they've been asked to review. If the authors seem to know what they are talking about, the papers are okayed; and if the topic is trendy or the author is a star, approval can be almost automatic.
The most valuable insight comes at the end from Bob Wolkow, who describes the natural process of weeding out useless results as happening after peer review:
" [...] peer- review is only the first step -- the chewing up. The digestion is the work meandering around the circuit, being discussed at conferences and eventually gaining credence or not." [...]
"The system can't be cheated for very long, because claims of fact must be verified eventually," says Wolkow. "You can't build on something that's not true. You can't take it the next step, because the foundation won't be there.
"So, there is a natural check and balance on all these things, though it might take time."
Now there's the rub: to a first approximation, scholars get rewarded for outputting knowledge that makes it through peer review, and not for knowledge that goes the full distance to reuse, providing a proper foundation for further work. And that is clearly a problem, because it favors short-term tactics. Run, and make it past the first hurdle; why should you worry about the next nine, if your promotion committees don't watch that part of the race?
There's another valid observation in there, between the lines: the people who will best assess the quality of your work are those who need to use it. Notice how similar this is to how things work in open-source software development. Also notice how similar this is to how the blogosphere operates. Users/reusers and readers are the real quality checkers, not a disinterested, busy, and anonymous third party.
So perhaps this would be a way to improve the usefulness and reliability of peer review: to ask people to review papers after they have themselves attempted to use the results. Post-publication, post-reuse peer review by self-selected reviewers. Is self-selection of reviewers a problem? And how would papers that are not actually followed upon by anyone else than the author get a review? Hmm. I'll have to think about this again sometime.
Actually, something like this, called "quick reviews", was attempted by Daniel Gottesman a couple years ago, but the effort somehow didn't mushroom. I'm not sure why, but I suspect that the volunteering reviewers did not get enough feedback or positive reinforcement.
Eric Hanson reflects on how hard it is to turn conversation into documentation. A very important issue if you want to go beyond group discussion systems or klognets, towards knowledge architectures worthy of the name.
The Whole in Our Heads. We're quite familiar with the mechanics of casual conversation here in Internet land. It's a metaphor made popular by sites like Slashdot in which some sort of post, usually called a "story", acts as a conversation starter. Then there's a flurry of conversation that follows as everybody puts in their two cents. I've never liked this. It's great for conversation, and the Slashdot community is inequivocably one of the most successful Internet communities. What bothers me is that the story/conversation metaphor produces, well, a story and a conversation. The resultant collaboratively produced informational resource is neither well organized nor a coherent whole, and often times not even related to the topic under which it was created. [group-forming]
Now I can see how Willinsky uses the open-access movement to argue his thesis. After mentioning the PLoS, PMC, OAI, BOAI, OKI, and PKP, he makes this point: "This emerging commitment among scholars to make the knowledge they create freely available is at the heart of my own call to the readers and editors of this journal to consider how turning educational research into a more accessible public resource can further the connection between democracy and education. While offering open access to all forms of scholarly research is certainly a global boon to students and faculty and to curious minds everywhere, it has a special political significance for the social sciences, as this work bears directly on social policies, programs, and practices. If open access to research in the life sciences can create a more democratic and educational dynamic in doctor-patient relationships, then, as I have argued elsewhere (Willinsky, 2000a), it is worth exploring across the social sciences. Here I am specifically asking researchers in the field of education to consider how greater public access to educational research is consistent with our understanding of what we do to foster education and further democratic participation, just as it speaks to the love of learning and pursuit of knowledge that has driven so many of us in this line of work." [FOS News]
Eric Dobbs sent me a link to another recent article on democracy and education a while ago. I haven't had time to read it in full yet, but the beginning looked very good. Here it is: Literacy, Democracy, Jefferson, and Wellstone by Linda Miller Cleary.
(I hope Lyn is reading this. This is really his kind of stuff.)
