The Creativity Post | Intelligent Testing

A generation ago, Alan Kaufman introduced the notion of intelligent testing, which represented his philosophy of how IQ tests should be used. According to this approach:

"The focus is on the child, with...communication of the test results in the context of the child's particular background, behaviors, and approach to the test items as the main goals. Global scores are deemphasized, flexibility and insight on the part of the examiner are demanded, and the test is perceived as a dynamic helping agent rather than an instrument for placement, labeling, or other types of academic oppression. In short, intelligent testing is the key."

In a recent edited volume in honor of Alan Kaufman (Intelligent Testing: Integrating Psychological Theory and Clinical Practice), the neuropsychologist Elaine Fletcher-Janzen noted how Kaufman's Intelligent Testing approach "became the gold standard for psychometric test interpretation and clinical assessment."

Unfortunately, not all clinicians apply the gold standard when testing a child. When I was in high school, I had a bad experience with a school psychologist who provided the bronze standard.

IQ tests have received much criticism over the years, some warranted and some just plain silly. Criticisms have ranged from the claim that IQ tests are unfair to those who are disadvantaged, to the claim that the test items have changed little over the years, to the charge that IQ tests minimize the importance of creativity, practical intelligence, character, virtue, and morality, to the claim that all IQ test makers and theorists believe that intelligence is an immutable property of the brain.

My own views on the utility of IQ testing have softened over the years, even though I still have a healthy dose of skepticism for the tests. I'd like to see creativity better measured and appreciated. Same for unconscious, intuitive modes of thought. I'd like the tests to only help, and never limit. I'm not convinced the type of problems on the tests are as exciting as can be to motivate students to want to solve them. I have many other criticisms but that's not what this post is about.

After spending some years conducting research on intelligence, I have come to realize a few things. There is no doubt that there are some shoddy test administrators and intelligence researchers who are biased by their prior beliefs and are out of touch with the latest research. But to the credit of contemporary IQ test makers-- the field is rapidly evolving. IQ test constructors have tried to address the critics on many key points. On the whole, they are moving away from a focus on a single IQ score. Instead, they are relying on updated models of intelligence, such as the Cattell-Horn-Carrol (CHC) theory that emphasizes multiple cognitive abilities. In fact, there is a trend now to not even refer to IQ tests as measures of "intelligence" but instead refer to them as tests of "cognitive ability". Some tests are even based on neurological functioning.

Most (but not all) researchers fully admit that IQ tests only measure a piece of the intelligence pie (although they argue it is an important piece). Most intelligence researchers are actually pretty nice, sensitive individuals who sincerely want to help the welfare of children. They want to create tests that help teachers identify students with learning difficulties and then select the right intervention for those students. They are scoring tests using sophisticated statistics and encouraging the use of the intelligent testing approach and the use of cross-battery techniques to increase fairness in measuring the cognitive abilities of those from diverse backgrounds.

So is there any utility to modern IQ tests? This is an important question still open to discussion, but I think past American Psychological Association President Diane Halpern put it quite reasonably when she said:

"We will always need some way of making intelligent decisions about people. We're not all the same; we have different skills and abilities. What's wrong is thinking of intelligence as a fixed, innate ability, instead of something that develops in a context."

I have particular respect for Alan and Nadeen Kaufman (they are no relation to me), whose tests have been on the forefront of the evolving nature of IQ tests. Alan's intelligent testing approach makes sense to me. It elevates the clinician above the test and enables the test administrator to treat each child as an individual. If we have to use a test, it allows us to use that test usefully-- to go beyond a single IQ score as the clinician observes the child's profile of strengths and weaknesses with the goal of custom tailoring a program to help that particular child.

In honor of Alan's recent book IQ Testing 101, which provides a nice summary of the current state of the field of intelligence testing, I conducted a short interview with him.

Thus, what follows are words from an actual IQ test maker.

What is the main tenet of the intelligent testing approach?

To me, the main tenet is that there is a hierarchy between the IQ test and the clinician who administers and interprets the test--and it is the clinician and not the test that is at the top of the hierarchy. Yes, it is important to use carefully-developed, well-standardized, theory-based tests of intellectual development. But the scores on such tests are meaningless unless they are interpreted by sophisticated clinicians who are totally knowledgeable in pertinent research on child and adult development, cognitive psychology, and neuropsychology, and who are astute observers and interpreters of behaviors (such as the person's approach to problem solving or the degree to which anxiety or distractibility might have compromised the person's test scores). Intelligent testers also must have a healthy respect for what tests cannot do, such as measure two-thirds of Sternberg's triarchic theory of successful intelligence (i.e., practical intelligence and creativity).

In what ways is modern IQ testing misunderstood by the general public?

The general public tends to misunderstand the kinds of IQ tests that are administered by psychologists to children or adults who are referred for evaluation. Such referrals are for a diversity of reasons such as suspected brain damage, learning disabilities, intellectual disabilities (mental retardation), behavioral disorders or intellectual giftedness. The tests are clinical instruments that are administered one-on-one for an hour and a half or two and can only be given by professionals (usually psychologists) who have a high degree of supervised training in intellectual assessment. I believe that most people tend to hear the words IQ test and think of the kind of paper-and-pencil IQ tests that they took in school. Such group-administered IQ tests are still commonly used, but not for making important real-life decisions. There is also a second level of misunderstanding by the more sophisticated lay public--and sometimes by psychologists whose field does not include clinical assessment, or by special educators who generally are not qualified to administer clinical tests of intelligence. And that is the notion that IQ test = Wechsler scale (or sometimes the old Stanford-Binet) and that the most important scores to interpret are the Verbal IQ, Performance IQ, and Full Scale IQ. In fact, Wechsler's scales remain the most popular, but there are many other widely-used tests to choose from, and these tests uniformly are based on theories of intelligence and neuropsychological processing. Furthermore, these tests emphasize a person's profile of theory-based cognitive abilities, not their global intelligence. And even the latest versions of the Wechsler scales are based on theories and research in cognitive neuroscience and have eliminated Verbal and Performance IQs in favor of standard scores on four theory-driven indexes.

Individually-administered IQ tests are still used to make important real-life decisions though, right?

Individually-administered tests are ordinarily used for important decisions such as diagnosis or placement in special education, but some people still take short cuts. Group IQ tests such as the Otis-Lennon are sometimes used to identify gifted students. In fact, New York city uses the Otis-Lennon along with an individual screening test for gifted assessment. My own opinion is that NO major decision about a person's life--child or adult--should be based on a group-administered IQ test.

Is it ever appropriate for a global-IQ score or specific cognitive ability score to be used as a cut-off to get into gifted education or to make any other important real-decisions?

Cut-off scores of any sort are a violation of what we have known for years about errors of measurement. The only situations where cut-off scores make any sense are for the diagnosis of intellectual disability (which used to be known as mental retardation). That diagnostic category is defined specifically by low intellectual functioning. However, even in such diagnostic circumstances, errors of measurement must be taken into account, and an IQ cut=off is not sufficient for the diagnosis--the person must also be shown to have very low adaptive behavior (akin to social intelligence).

Are global-IQ or specific cognitive ability scores ever invalid for an individual? In what ways could they be invalid?

It is common for IQs or scores on separate scales to be invalid for an individual. the tests are administered individually, and rapport with the examiner is important. If the examiner doesn't succeed in establishing and maintaining rapport with a child or adult, then that person might not give full effort. Apart from low motivation, IQs are sometimes invalid estimates of a person's true functioning because behaviors such as distractibility, anxiety, or low frustration tolerance can interfere with test performance and lead to a low IQ or a low standard score on a test of working memory--even when the person truly has exceptional ability.

How much can these scores change over a person's lifetime, and how limiting are a person's scores for obtaining what they want out of life?

For groups of individuals, IQs are fairly stable between childhood and adulthood, but for specific individuals within a group, IQs can--and do--vary greatly over a lifetime. The IQs will vary as a result of specific interventions (such as preschool enrichment programs), quality education (or the lack of it), injuries that affect brain functioning, and other aspects of the environment that either enhance or diminish one's cognitive ability. In addition, errors of measurement are much larger than people tend to think, and, therefore, an individual's IQs will vary from time to time--sometimes substantially--simply due to the chance fluctuations that accompany any repeated measurement. And, there is more to life success than the ability to score high on IQ tests. People can be successful based on their creativity, street smarts, and personality variables.

How can IQ tests be misused?

IQ tests are misused when a person is given a number or a set of numbers that supposedly define their intellect, but are not given an explanation of what the numbers mean. They are also misused whenever IQs and other standard scores are presented without providing confidence intervals around the obtained scores. They are misused when they are used as the sole method for making a decision, such as special education placement or being hired for a job or gaining entrance into a prestigious program.

It seems to me that since IQ-global scores or even specific cognitive ability scores derived from individually-administered testing sessions are still used to make important real-life decisions--even today-- it is all the more reason for clinicians to adopt the intelligent testing approach. I'm sure you agree?

I do agree, naturally, because that is what I have been "preaching" for 30 years.

References

Fletcher-Janzen, E. (2009). Intelligent testing: Bridging the gap between classical and romantic science in assessment. In J. C. Kaufman (Ed.), Intelligent testing: Integrating psychological theory and clinical practice (pp. 15-29). New York: Cambridge University Press.

Kaufman, A. S. (1979). Intelligent testing with the WISC-R. New York: Wiley.

Kaufman, A.S. (2009). IQ Testing 101. New York: Springer.

*** (Personal Aside: Life is funny. When I was a child, my bad experience was with the Wechsler Intelligence Scale for Children (WISC). This experience contributed to my decision to become an intelligence and creativity researcher. As it turns out, when Alan Kaufman was at The Psychological Corporation he worked with David Wechsler on the revision of the WISC-- the exact test that was the bane of my existence. Here I am, years later, collaborating with Alan and his son James. I am thankful that they have been so supportive of my career and hold them both in very high esteem.)