The Haversian Canal

« Starting Break | Main | A&M Rideshare System »

December 18, 2005

Blog Reliability: One vs Many

Dmitriy recently commented (Credibility Spat: Blogs vs. Peer-Reviewed Pubs) on George D. Lundberg's opinion of mass media's opinion of blogging. Dr Lundberg is the general editor of Medscape General Medicine. He starts by using a lot of loaded phrases (possibly designed to encourage blogging about it?), but I think Lundberg's central point, that blogs represent nothing more than the opinions of the bloggers who write them, is correct. However, the whole piece misses the larger issues: the real blogging community and the role of blogging in society.

Blogs are not peer-reviewed journals of the scientific community. Like any uncontrolled form of communication, blogs are a superb source of vast amounts of poor information. I didn't say bad information, just poor information. Looking down from thirty thousand feet, the blogs don't have much information. On average. Within the 70 million or so blogs, I'm fairly sure there are fewer bloggers than blogs. There are relatively few multi-author blogs, while there are many authors with multiple blogs. Among these authors, there are some who write poorly, inconsistently, and are factually wrong. Some write well, consistently, and are factually accurate. And there's everything in between.

Writing improves over time. We know that. The brain gets better at what it does repititiously. Einstein, Richard Feynman, Leonardo Da Vinci, Walt Whitman, Herman Melville. They weren't just good, they did a lot of what they did (ever read Moby Dick?). So, after allowing for a certain degree of talent, intelligence, whatever, writing consistently leads to writing consistently well, or at least better.

The third criteria one wants from a blogger is good fact-checking. Problem-solvers are less prone to factual error. Such things as solutions to real problems require facts. So the ability to appraise evidence is at a premium in problem-solving. There are classes dedicated to it. There are professors that spend their careers teaching the fine art of analyzing evidence. The fear of public embarrasment, more positively the pride of reputation, is an independent motivator for fact-checking.

Out of those 70 million blogs, a large number of those appear and disappear very frequently. Genuflecting on virtually any graph of internet traffic, with the high, sharp spike at 0 and a long right tail, it's easy enough to understand that most people don't stick with one thing on the internet for very long. So the longer from inception, the less likely it is that a blog is still posting new content. The only question is how many. My guess is that fewer than 1 million see the first six months. Why do I say that? Right now I'm ranked 15907 in the TTLB ecosystem, and 146,340 in Technorati. I've been blogging for six months, one week, and two days.

So how much more reliable are journals than blogs? The work of John Ionnidis is helpful here. His recent article in PLoS, Why Most Published Research Findings Are False, is representative. That article in particular indicates that any single work in a collected body of works (all journals, all textbooks, all blogs) has a certain probability of being wrong, and that, for traditional journals, this probability is high in an absolute sense. Picking up a random journal article, the odds of a correct conclusion are slim.

This, however, has to do with the medium's highly specific hypotheses. Textbooks, on the other hand, are amazingly reliable for general reasoning, though the more specific the clinician attempts to get, the closer to journal articles one has to go. It may be that the clincian would actually be able to get even more specific about a particular patient, and occasionally about a particular issue, through blogs than through journal articles, much as the textbooks are amazingly accurate at 30,000 feet, yet desperately myopic when compared the the truth at ground level. By honing in on the recorded thoughts and actions of one individual, the clinician may gain considerable insight into clinically important things, like the person's values, models of disease, and decision-making processes.

On a different tack, an expert blogger might post some exceptionally specific and useful information about their particular research interests in a blog. Unlike the medical column of the typical newspaper, where it is extremely difficult to assess the background, decision-making, and values of the once-weekly 500 word columnist, a blog reader has instant access to the blogger's archives and can rapidly assess their reliability and degree of expertise.

A third value of blogs is penetration. Blogs are easier to read than journals so more people read them. At least, I'm willing to bet less than 30% of America's population read a scientific journal article in the first three months of 2005. However, blog readers may value the bloggers that do read the literature. Personally, I got my subscription ot the New England Journal of Medicine in April 2003 and I currently have subscriptions to the New England journal, Nature, JAMA, and The Medical Letter (a newsletter on medications supported only by subscriber dues, no advertising). As a student, a student, I'm paying over $250 a year for those four subscriptions, and the librarians assure me that I'm getting off cheap. Just like internet usage, most doctors get their information from similar sources. Most medbloggers, I'll bet, have a lot of similar subscriptions. And those subscriptions are expensive.

Note that I didn't make originality a value among blogs. While the fiction of medbloggers is particularly good, IMHO, I should hope there's not to much original research coming out in the blogs. Similarly to political blogs that ultimately rely on foot-leather journalists, medical blogs rely on scientists in the clinic and in the lab for at least some of their content.

I predict that if there are 300 medical blogs, 60 have been published for more than six months and 50 of those authors have subscriptions to at least either the New England Journal or JAMA. Maybe somebody could check my prediction on medlogs.com, technorati, or some other aggregator. More generally, I think bloggers and medical journals have similar philosophical goal of distributing knowledge. In the war against myth and pseudoscience, bloggers are more valuable not as original sources but as more diffuse translational resources for the general public. For the general public, blogs are more useful than journals because they are linguistically and economically more accessable.

Posted by Niels Olson at December 18, 2005 9:46 PM

Comments

Posted by: Hippocrates at December 19, 2005 12:29 AM