The Haversian Canal

« USMLE Step 1 Score Calculator | Main | Shop Class as Soulcraft »

October 1, 2006

What is the framework for USMLE Multiple Choice Questions?

Why, the National Board of Medical Examiners answered that question their spring newsletter. The rather drab article is about the flow of questions through the editorial process, but from a student's point of view, the real thing this article reveals is a hint at the general framework the editors use for questions. To explore this, examine the sample question at the bottom of the article and see what the editor does to it. What you see is the editors are *editing*. This may sound like a no-brainer, but it wasn't entirely obvious to me until after I found the article, read it, and asked the question. Editors everywhere, for Conde Nast, the Times of London, and Oxford Press, all baseline their work by seeking specificity (who's in the room, why are they there, what, when, where, how), consistent grammar and syntax (six year old becomes 6-year-old), and consistent presentation (age, sex, chief complaint, info from the intake, physical exam, follow-up tests), and, in the case of MCQs, specific diagnoses. Also,notice how some of the tip-off verbage that may have been au natural in the first draft (trident hand), has become, perhaps more generally correct use of language, but also has lost some of the visceral significance (tridentate appearance of fingers on extension).

The article also links to a prior article, Who Writes Those Questions, reveals another thing worth knowing. Try as they may, the professors just can't keep a hint of condescension out of their communications to students:

Many students and some faculty members at medical schools have the perception that the USMLE Step examinations are designed and constructed by a group of anonymous individuals who have little connection with medical schools or current medical school curricula.

If you're a medical student and thought that people with little connection with medical school wrote the USMLE questions, please raise your hand (preferably in the comments section). Notice the cherry-picked questions (how many times were these questions asked, out of how many total questions? Out of how many unique questions?)

Comments like the following reflect this point of view:

"What a stupid question!"
"We never were taught that!"
"Our curriculum doesn't require us to memorize facts!"
"They must have an army of trolls sitting in a cave writing these questions!!"

In fact, however, designing the Step examinations, developing the examination materials (including the determination of "the correct answer"), and setting the minimum passing scores are responsibilities of examination committees composed of medical educators and clinicians.

There is also a certain editorial lassitude in the writing that these folks don't use when writing for their colleagues: "In fact, however," emphasis-yours quotation marks around "the correct answer", and a weak introductory clause leading into the list. If these people can edit MCQs for medical licensing exams, they surely appreciate the value of introducing a list with an independent clause. I'm not saying they're aren't the sharpest blades, they quite probably *are* the sharpest blades. It is the lassitude of condescension that's coming out, not any lassitude in their peer-to-peer writing. What's it indicate? A glass ceiling. There's only so far a student is going to get with these people, and they'll actually be oblivious to the fact that you might be trying to get through to them. I have little doubt that the USMLE tests will prove to be some of the best, most challenging tests ever written, but I wouldn't waste to much effort on contacting the NBME about anything. Just do what they say and study.

The NBME advises their question writers to write mainly what they call hinge questions, that is, the you are given some information, and you have to know the diagnosis in order to then answer the question.

Posted by Niels Olson at October 1, 2006 8:21 PM

Comments

Post a comment