Why do we think about numbers and graphs differently?
A visitor to Edward Tufte's forum, Angela Morelli, asked me by email why we understand numbers and graphs differently. A revised and sources-cited version of my response is below.
People interpret numbers and graphs differently because they are handled differently in the brain. Numbers are generally handled by the verbal linguistic system and graphs are handled by both the non-verbal linguistic system and the limbic system. The bit rate of the visual system is about 10 million bits/second (see Tufte's original thread). The rate of reading, listening, braille, typing, maxes out at around 150-400 words per minute. To understand how this works, and provide a foundation for further reading, a *very* brief review of the relevant neuroscience seems in order.
Visual processing begins in the retina with some very simple edge definition. Further edge definition occurs where the neurons of the optic nerve enter the brain at the lateral geniculate nucleus. The neurons synapse and new neurons run in the optic radiations from the LGN to the banks of the calcarine sulcus, where further edge definition and integration occurs. The calcarine sulcus is at the very posterior part of the cerebrum and represents the first time the visual information enters the cerebral grey matter. From there, the final elements of subconscious analysis and pattern recognition occur in the lingual gyrus and cuneus, which sort of wrap around the banks of the calcarine sulcus like concentric rings (heavily folded, of course). Everything up to and including this point is essentially image processing.
Conscious recognition starts to occur in the inferotemporal region, Brodmann's areas 37 and 7a. Lesions to these areas lead to what are called agnosias. Oliver Sacks' The Man Who Mistook His Wife for a Hat has a good example of an agnosia. In fact, most of the back half of the cerebrum (abaft your ears) that isn't involved in basic visual perception is involved in this kind of unimodal association. The other major exception is the angular gyrus and Wernicke's area.
Once the basic conversion to symbolic information occurs, information is routed based on type. Basic numeracy is handled by Brodmann's area 39 in the angular gyrus of the parietal lobe, just slightly above Brodmann's area 37 where the numbers were recognized as numbers. The syntactic region of the brain, Wernicke's area, exists very close by, and can be considered to involve the angular gyrus. A stroke to Wernicke's area leads to expressive aphasia. The patient has access to their entire vocabulary and will speak words clearly, but can't understand and can't compose syntactically correct thoughts. This is commonly described as a "word salad". Wernicke's area is where mathematical training or computer science training trains new syntactic structures. A physicist, in some ways, can think thoughts a non-physicist can't. If Noam Chomsky's syntactic structures exist, they are mainly constructed in Wernicke's area. Now, Wernicke's area is still in the back half of the brain. It is evolutionarily older than Homo sapiens. And, indeed, apes and even dogs, rats, cats, and insects can construct syntacticly different sequences of sounds.
Where we really start to diverge from other species is in Broca's area, which structurally lies in the relatively new prefrontal cortex and consists of the Pars triangularis and Pars opercularis. Functionally, Broca's area contains the dictionary. A person with a stroke in Broca's area can, with great difficulty, construct sentences, but they have profound word-finding problems which get worse with stress. And they're always stressed out because they can't find the word! Between Wernicke's area and Broca's area is the arcuate tract, a superhighway of axons committed to carrying information between the neurons of Wernicke's area and Broca's area. Damage to the arcuate tract results in a person who can understand and can speak, but can't hear what you say and then formulate a reply. In higher math, it is the left-sided verbal linguistic system that is involved in equations. In basic number recognition, it is simply the angular gyrus that is involved. This whole system is essentially verbal and exists on the left side. A non-verbal, musical, spatial, temporal, inflection-oriented corollary system exists on the right side.
Someone with a right-sided stroke can communicate, but may have inappropriate responses because they can't understand, interpret, or compose the "how you say it". They also have difficulties interpreting space and, perhaps, time, as they tend to have attention disorders. Interestingly, mathematicians are commonly interested in music and often find spatial expressions of their mathematics particularly appealing, suggesting a high degree of integration between their verbal and non-verbal language centers. And, similar to the syntactic function of Wernicke's area, a musician can compose syntactic structures a non-musician may have a hard time understanding. |
As an aside, I am inclined to wonder if general intelligence is an emergent property of our neurons in a way that is similar to how a Turing complete programming language can emerge from Church numerals and lambda calculus. One could think of a neuron as an atom, two neurons connected by a synapse as a list, and every neural synapse is an additive function activated by an action potential. |
Regardless of how many bits go between Broca's area and Wernicke's area over the arcuate tract, that word-per-minute limit is still representative of how fast people can cogitate about abstract things like numbers. Obviously 150-400 words per minute is much slower that 10 million bits/second. However, the trained mind can handle numbers faster than the untrained mind, and scanning a table of numbers for the high and low and getting a feel for the median and the nature of the distributions and the relationships between variables can be done quite quickly without cogitating too explicitly about each number. One need not advance every number to the level of free will to appreciate that some are bigger than others.
Outwardly the difference between computer graphics processors and general purpose central processors is very similar to the difference between the visual system and the verbal or non-verbal linguistic systems. At the nVidia08 conference, Mythbusters provided a very nice demonstration of the difference between manipulating a single-threaded general processor (analogous to the linguistic system composed of Broca's area, Wernicke's area, and the arcuate tract) to render an image, and a function-specific system like a graphics processor or the visual system. While faster general processors are desirable, their power is not in their speed. Their power is in their generality, their ability to deal with any abstract issue and, potentially, make value judgments and exert free will. The mythbusters's CPU illustration, for example, could also be used to pick things up, or whatever else a robotic arm can be made to do.
Judgment, free will, and risk analysis occur on two levels. Judgment and free will occur in the very newest part of the cerebrum, the anterior cingulate gyrus, roughly Brodmann area 24. Recent findings suggest this is the last part of the brain to mature, at about the age of 25. I guess the car insurance companies know what they're talking about :-) Risk analysis can occur in this area, but, as we all know, the average bee can also conduct risk analysis. This reflexive risk analysis occurs in the limbic system, part of the reptilian brain that underlies our newer cerebrum. marketeers like graphs with a positive slope because society teaches us over and over the a positive slope is good, income-positive, growing, whatever. It's the repetitive association with primal desires that causes the positive slope to be imprinted on the reptilian brain. Me like up and right! Can I has cookie? Valuations that are repeated, over and over, are more likely to be imprinted, branded in the limbic system's primordial type of memory. Commercial television really is all about holding your eyes still while they spray your brain with advertising. Imprinting those brands on the limbic system. Over and over. Until the reptilian brain learns. The great thing about this, for marketeers, is that the reptilian brain keeps us alive in a lot of situations so it tends to get privileged access to information so it can respond very quickly, when necessary, or at least when certain preconditions are met. Marketeers have leveraged those preconditions to train the imprinted, branded, conditioned reptile to pick things up off the shelf before the human brain intervenes.
What we have then is an input system, the visual system, that provides input to multiple analytical systems. We are, at a minimum, a multi-core processor. Understanding numbers and thinking about math and interpreting graphics and making value decisions requires the newest and most complex parts of our brain, but there is a very real possibility of sending the information to the wrong system, the reptilian system. The association (positive slope)==(good), in any human who grew up in modern society, can be safely assumed to be imprinted in the reptilian brain. Unfortunately, even very educated and successful people
So, making graphics to represent numbers can be tricky work, and society has rewarded ET and others for grappling with the problem. There are some books on how to think about these problems :-) If you want to evoke things that the reptile values, like hunger and fear, then activate the reptile. If you want things that are assigned by prefrontal centers, like credibility, reputation, and respect, then you should try to activate the prefrontal centers, and providing numerical information is one way to do that. If you need graphs because there are too many numbers, then you should make sure those graphs activate the non-verbal linguistic system: they should carry a fair bit of information, describe multiple variables,, prompt further decision-making by the prefrontal cortex, cite your sources, etc.
1The neuroanatomy for this post was checked against DE Haines, Fundamental Neuroscience for Basic and Clinical Applications, 3rd Ed, Elsevier 2006, pp 518-522.
