The Creativity Post | Imagining a New College Entrance Examination

The new SAT is causing a lot of debate and criticism. Rightly so.

For many parents, students, educators, and college admissions committees, the revised SAT is just a minor variation on the same theme (e.g., more passage analysis, more relevant vocabulary, more focused math questions). In addition to criticizing the skills that are being measured, critics also question whether the new test will promote diversity. As the president of Bard College recently noted, "The time has come for colleges and universities to join together with the most innovative software designers to fundamentally reinvent a college entrance examination system."

But what would a truly innovative college entrance examination system actually look like?

Imagination

“The possible's slow fuse is lit by the Imagination.” ― Emily Dickinson

The SAT is a measure of convergent thinking: the ability to arrive at the single best answer deemed correct by a select committee. One big thing that is missing on these tests is the assessment of divergent thinking: the ability to generate multiple possibilities, ideas, and solutions to a problem.

Convergent and divergent thinking aren't completely separate processes: both rely on "executive functions" such as concentration, mental flexibility, and the ability to maintain strategies in working memory. Nevertheless, convergent and divergent thinking skills can be investigated independently of each other (even on the same test), and both skills independently contribute to creativity.

The most widely administered test of divergent thinking is the Torrance Tests of Creative Thinking (TTCT) developed by E. Paul Torrance in the late 1950s. The Torrance test includes both verbal and figural tests that require people to generate multiple possibilities (e.g., "How many uses are there for bubble gum?"), and imagine the consequences of unimaginable events hapenning (for instance, “What would happen if people could become invisible at will?”).

The Torrance test has proved useful in a number of educational settings. For one thing, use of the Torrance tests is a helpful addition to gifted and talented assessment. Research shows that teachers are more likely to identify students as gifted who are achievers and teacher pleasers than students who are creative, because creative students are typically more unconventional. Torrance once estimated that if we identify gifted children only on the basis of IQ and achievement test scores, we are leaving out of consideration 70 percent of the top 20 percent of creative students. Considering divergent thinking as part of a giftedness assessment really isn’t all that difficult for teachers and administrators. In a recent paper I teamed up with creativity researcher James C. Kaufman and assessment specialist Elizabeth Lichtenberger to recommend ways that IQ test administrators can find divergent thinking in the test taker by looking for creativity in the responses. If you don't look for creativity in students, you aren't going to find it.

Unfortunately, there’s evidence that we aren’t appreciating divergent thinking skills in our students. Kyung Hee Kim recently found that since 1990, Torrance test scores have significantly decreased, with the decline between kindergarten and third t ade being the most significant. This is particularly troublesome considering that ethnic differences on creativity tests are rare. Creativity researcher James C. Kaufman argues that creativity assessment can offer a more accurate reflection of the intellectual abilities of ethnic minorities, and can reduce stereotype threat. Similarly, Robert Sternberg has found that minority enrollment at college improves by adding imagination items like those on the Torrance tests to the admissions equation.

How well do the Torrance tests predict creative achievement? The strength of the prediction varies depending on the sample, testing conditions, and form of creative achievement, but on the whole the tests are somewhat predictive. In one large review based on 47,197 participants, Kim found a small but positive correlation between divergent thinking and creative achievement, with the relationship between divergent thinking and creative achievement highest when measured eleven to fifteen years apart. What's more, she found that divergent thinking test scores did a much better job predicting creative achievement in art, music, writing, science, leadership, and social skills than IQ tests.

In the late 1950s Torrance conducted his own long-term study of creativity, providing one of the longest studies ever conducted on the relationship between creative abilities and achievement. His sample included every student attending two Minnesota elementary schools from 1958 to 1965. Students took the TTCT every year and also completed IQ and academic achievement tests.

In 1999 Jonathan Plucker reanalyzed Torrance’s data from 1958 to the present day and found that divergent thinking test scores explained about half of the differences in adult creative achievement. What’s more, divergent thinking scores explained more than three times the amount of variance in creative achievement compared to IQ scores.

A more recent analysis published in 2010 by Mark Runco, Garnet Millar, Selcuk Acar, and Bonnie Cramond followed up Torrance’s sample fifty years later. By this point the Torrance test scores no longer significantly predicted public creative achievement, but the tests did show a weak relationship with personal creative achievement (such as various hobbies). IQ wasn’t correlated with any form of creative achievement (public or personal).

Which raises the question: what other characteristics are important for success in college and ultimately, in life?

The Beyonder Characteristics

In 1983 Torrance reported that the extent to which children fell in love with a future dream was consistently a better predictor of creative achievement than multiple indicators of scholastic achievement, including school grades and IQ test scores. As he noted,

“One of the most powerful wellsprings of creative energy, outstanding accomplishment and self-fulfillment seems to be falling in love with something— your dream, your image of the future.”

After conducting more interviews with the children in his longitudinal study, and really taking the time to get to know them, he compiled a list of "Beyonder Characteristics" that distinguished the high academic achievers in school ("Sociometric Stars") from the high adult creative achievers ("Beyonders"). Based on these interviews, he created The Beyonder Checklist, that included the following characteristics:

Love of work
Persistence
Purpose in life
Diversity of experience
High energy
Creative self-concept
Risk taker
Open to change
Deep thinking
Tolerance of mistakes
Well-roundedness
Feeling comfortable as a minority of one

At the 50-year follow-up, Runco and colleagues found that a composite of these Beyonder characteristics was a better predictor of lifelong personal and publicly recognized creative achievement than IQ or divergent thinking.

Toward a New College Entrance Examination

What are the implications of all of these findings for the development of a college entrance examination system that fully captures what a child is capable of acheiving in college and beyond?

Standardized achievement test scores do a good job measuring cognitive ability, and divergent thinking tests do a good job measuring creative cognition, but both tests miss out on other important learning characteristics (e.g., active learning strategies, interests, self-control, persistence) that are crucial for academic success. High school grades, which reflect learning over the course of an entire semester are a much better predictor of college GPA than SAT scores. This is because report card grades are a much better indicator of a variety of important learning characteristics, including active learning strategies and self-control, than performance on standardized tests.

Also, whatever labels test companies decide to put on their tests—IQ, SAT, GRE— if a battery of tests place a heavy burden on working memory by requiring the student to mentally manipulate complex information and give a quick answer, those with lower levels of working memory won’t get the chance to fully express what they are really capable of achieving in college. It’s very difficult—if not impossible—to design a test that is administered in a short time span and requires novel problem-solving that doesn’t place considerable demands on working memory.

Perhaps most crucially, standardized tests miss out on potential for long-term creative achievement. The reason why the Beyonder characteristics remain predictive of personal and public creativity long after the effects of test scores disappear, is because creativity requires an extended period of deep mastery, trial and error, and perseverance. No single test of reasoning or divergent thinking can replace the value of deep immersion in a personally valued domain. Indeed, many characteristics that matter the most in life are hidden from view if potential is assessed in a single slice in time.

At the end of the day, the messy fact of life is that over the long haul, there are multiple paths to the same outcome. In the real world, including the college setting, people can mix and match their own unique package of characteristics to achieve the same outcome. Some students may indeed get by with their high IQ, but others may get there more through cultivation of grit, passion, and active learning strategies. Standardizing minds ignores the many paths to intellectual achievement, and the possibility for compensation and capitalization across the lifespan.

What does this mean for a new college entrance examination system? In my view, the only truly equitable and fair system is one that fundamentally eliminates the entire notion of a decisive moment of evaluation and appreciates the journey. That recognizes that potential is a constantly moving target, and is most accurately and fairly assessed holistically over an extended period of time.

Imagine if starting from the very first day of high school, all students started building a college portfolio that allowed them, on their own terms, to display what they are intellectually and creatively capable of achieving given time and opportunity? The portfolio could be continually revisable, and include multiple indicators of college readiness, such as teacher evaluations of quality of class participation, including behaviors that indicated imagination and critical thinking, homework completion, commitment to learning, and the use of active learning strategies in the classroom.

The portfolio could also include the fruits of meaningful projects and written reports by the students about the outcomes of discovery based learning. No two portfolios would be the same, but colleges would expect and appreciate that fact, and focus on the holistic package of each applicant based on four years of engagement in school, and fit for proposed major in college. In essence, imagine what would happen if we replaced all that time spent on preparing for a single judgement day, with the cultivation of a deep knowledge base and display of individual possibility.

I think with imagination and ingenuity, we could create such a system that is feasible. For instance, we could ask students to select the top 5 items from their portfolio that highlight their capabilities. In other words, we allow each student to make his or her own case for college readiness. After all, we allow students to retest the SATs and many colleges only take the best score.

I believe that if we implemented such a system, we'd be pleasantly surprised by just how much intellectual and creative potential there already exists all around us.

I'll leave you with Torrance's Manifesto for Children, inspired by Torrance's interviews with teachers about the characteristics most important for school success: © 2014 Scott Barry Kaufman, All Rights Reserved.