When I give talks about my brain theory, audiences are usually quick to grasp the significance of prediction as it relates to a host of human activities. They ask many related questions. Where does creativity come from? What is consciousness? What is imagination? How can we separate reality from false beliefs? Although these topics have not been in the forefront of my motivations for studying brains, they are of interest to nearly everyone. I don’t pretend to be an expert in these topics, but the memory-prediction framework of intelligence can provide some answers and useful insight. (…)
What is Creativity?
I’m frequently asked about creativity, I suspect because many people see creativity as something a machine couldn’t do, and therefore it is a challenge to the entire idea of building intelligent machines. What is creativity? Creativity is not something that occurs in a particular region of the cortex. Nor is it like emotions or balance, which are rooted in particular structures and circuits outside of the cortex. Rather, creativity is an inherent property of every cortical region. It is necessary component of prediction.
How can this be true? Isn’t creativity some extraordinary quality that requires high intelligence and giftedness? Not really. Creativity can be defined simply as making predictions by analogy, something that occurs everywhere in cortex and something you do continually while awake. Creativity occurs along a continuum. It ranges from simple everyday acts of perception occurring in sensory regions of the cortex (hearing a song in a new key) to difficult, rare acts of genius occurring at the highest levels in the cortex (composing a symphony in a brand-new way). At a fundamental level, everyday acts are so common we don’t notice them. By now you have a basic understanding of how we create invariant memories, how we use invariant memories to make predictions, and how we make predictions of future events that are always somehow different from anything we have experienced in the past. Recall also that our invariant memories are of sequences of events. We make predictions by combining the invariant memory recall of what should happen next with the details pertaining to this moment in time. Prediction is the application of invariant memory sequences to new situation. Therefore all cortical predictions are predictions by analogy. We predict the future by analogy to the past.
Imagine you are about to have dinner in an unfamiliar restaurant and you want to wash your hands. Even though you have never been in this building before, your brain predicts that there will be a restroom somewhere in the restaurant with a basin suitable for hand washing. How does it know this? Other restaurants you have been in have a restroom, and by analogy this restaurant will likely have one, too. Further, you know where and what to look for. You predict it will be toward the back of the restaurant, either by the bar or down a hall, but generally not in plain view of eating areas. Again, you have never been in this particular restaurant before, but by analogy to other eating establishments you are able to find what you need. You don’t look randomly. You look for expected patterns that let you find the restroom quickly. This kind of behavior is a creative act; it is predicting the future by analogy to the past. We don’t normally think of this as being creative, but it very much is.
Recently I bought a vibraphone. We have a piano, but I had never played the vibraphone before. The day we brought it home, I took a sheet of music from the piano, placed it on the stand over the vibraphone, and started playing simple melodies. My ability to do this was not remarkable. But in a fundamental way, it was a creative act. Think about what was involved. I have an instrument that is very different from a piano. The gold bars are big and gradually change in size; the keys are small and of two different sizes. The gold bars are arranges in two different rows; the black and white keys are interleaved. On one instrument I use my fingers, and on the other I swing mallets. Fir this I’m standing up, and for that I’m sitting down. The particular muscles and motions needed to play the vibraphone are completely different from those needed to play the piano.
So how was I able to play the melody on an unfamiliar instrument? The answer is that my cortex sees an analogy between the keys on a piano and the bars on a vibraphone. Using that similarity allowed me to play a tune. It isn’t really any different from singing a song in a new key. In both cases, we know what to do by analogy to past learning. I realize that to you the similarity between these two instruments may appear obvious, but that is only because our brains automatically see analogies. Try to program a computer to find similarities between objects such as pianos and vibraphones and you will see how incredibly difficult this is. Prediction by analogy- creativity- is so pervasive we normally don’t notice it. We do, however, believe we are being creative when our memory – prediction system operates at higher level of abstraction, when it makes uncommon predictions, using uncommon analogies. For example, most people would agree that a mathematician who proves a difficult conjecture is being creative. But let’s take a close look at what’s involved with her mental efforts. Our mathematician stares hard at an equation and says, “How am I going to tackle this problem?” If the answer isn’t readily obvious she may rearrange the equation. By writing it down in a different fashion, she can look at the same problem from a different perspectives. She stares some more. Suddenly she sees a part of the equation that looks similar to the structure of another equation that looks familiar. She thinks, “Oh, I recognize this. There’s a structure to this equation that is similar to the structure of another equation I worked on several years ago.” She then makes a prediction by analogy. (…)
Shakespeare’s metaphors are the paragon of creativity. “Love is a smoke made with the fume of sighs”. “Adversity’s sweet milk, philosophy.” “There’s daggers in men’s smiles.” Such metaphors become obvious when you see them but they’re very hard to invent, which is one reason why Shakespeare is regarded as a literary genius. To create such metaphor he had to see a succession of clever analogies. When he writes “There’s daggers in men’s smiles, “ he is not talking about daggers or smiles. Daggers are analogous to ill intent, and men’s smiles are analogous to deceit. Two clever analogies in only five words! At least that is how I interpret it. Poets have the gift of correlating seemingly unrelated words or concepts in manners that illuminate the world in new ways. They create unexpected analogies as a means of teaching higher-level structure.
In fact, highly creative works of art are appreciated because they violate our predictions. When you see a film that break the familiar mold of character, story line, or cinematography (including special effects), you like it because it is not the same old same old. Paintings, music, poetry, novels- all creative artistic forms – strive to break convention and violate the expectations of an audience. There is a contradictory tension in what makes a work of art great. We want art to be familiar yet, at the same time to be unique and unexpected. Too much familiarity is retread or kitsch; too much uniqueness is jarring and difficult to appreciate. The best works break some expected patterns while simultaneously teaching us new ones.
Consider a great piece of classical music. The best music has an appeal at a simple level – good beat, simple melody and phrasing. Anyone can understand and appreciate it. However, it is also a little different and unexpected. But the more you listen to it, the more you see there is pattern in the unexpected parts, such as repeated unusual harmonies or key changes. The same is true with great literature or great movies. The more you read or see them, the more creative detail and complexity of structure you observe.
You probably had the experience of looking at something when a twinge of recognition goes off in your head: “Hmmm, I’ve seen this pattern before, someplace else…” You might not have been trying to solve a problem, it’s just that an invariant representation in your brain was activated by a novel situation. You saw an analogy between two normally unrelated events. I might recognize that promoting a scientific idea is similar to selling a business idea or that bringing about political reform is like raising children. If I’m a poet, voilà!, I have a new metaphor. If I’m a scientist or engineer, I have a new solution to a longstanding problem. Creativity is mixing and matching patterns of everything you’ve ever experienced or come to know in your lifetime. It’s saying, “this is kinda like that.” The neural mechanism for doing this is everywhere in the cortex.
Are Some People More Creative Then Others?
A related question I often hear it, “If all brains are inherently creative, why are there differences in our creativity?” The memory-prediction framework points to two possible answers. One has to do with nature and the other with nurture. On the nurture side, everyone has different life experiences. Therefore everyone develops different models and memories of the world in his or her cortex, and will make different analogies and predictions. If I have been exposed to music, I will be able to sing songs in new keys and play simple melodies on new instruments. If I have never been exposed to music, I will not be able to make these predictive leaps. If I have studied physics, I will be able to explain the behavior of everyday objects via analogy to the laws of physics. If I grew up with dogs, I am apt to see analogies about dogs and will be better at predicting their behavior. Some people are more creative in social situations or in language, math, or diplomacy, all based on the environment they grew up in. Our predictions, and thus our talents, are built upon our experiences.
In chapter 6, I described how memories are pushed down the cortical hierarchy. The more you are exposed to certain patterns, the more the memory of these patterns are re-formed at lower levels. This allows you to learn the relationships among higher- order abstract objects at the top. It’s the essence of expertise. An expert is someone who through practice and repeated exposure can recognize patterns that are more subtle then can be recognizes by a nonexpert, such as the shape of a fin on a late-fifties car or the size of a spot on a seagull’s beak. Experts can categorize patterns on top of patterns. Ultimately there is a physical limit to what we can learn constrained by the size of our cortex. But as humans, our cortex is large compared to other species and we have a tremendous flexibility in what we can learn. It all depends on what we are exposed to throughout our lives.
On the nature side, brains exhibit physical variations. Certainly some of the differences are genetically determined such as the size of regions (individuals can show as much as a three-fold difference in the gross area V1) and hemispheric laterality (women tend to have thicker cables connecting the left and right sides of the brain then man do). Among humans, some brains probably have more cells or different kinds of connections. It’s unlikely that Albert Einstein’s creative genius was purely a function of the stimulating environment in the patent office where he worked as a young man. Recent analyses of his brain – which had been thought lost, but was found preserved in a jar a few years ago – reveal that his brain was measurably unusual. It had more support cells, called glia, per neuron that average. It showed an unusual patterns and grooves, or sulci, in the parietal lobes- a region thought to be important for mathematical abilities and special reasoning. It was also 15 percent wider then most other brains. We may never know why Einstein was as creative and smart as he was, but it is a safe bet that part of his talent derived from genetic factors.
Whatever the difference between brilliant and average brains, we are all creative. And through practice and study we can enhance our skills and talents.
Can You Train Yourself to be More Creative?
Yes, most definitely. I have found there are ways to foster finding useful analogies when working on problems. First, you need to assume up front that there is an answer to what you are trying to solve. People give up too easily. You need confidence that a solution is waiting to be discovered and you must persist in thinking about the problem for an extended period of time.
Second, you need to let your mind wander. You need to give your brain the time and space to discover the solution. Finding a solution to a problem is literally finding a pattern in the world, or a stored pattern in your cortex that is analogous to the problem you are working on. If you are stuck on a problem, the memory-prediction model suggests that you should find different ways to look at it to increase the likelihood of seeing an analogy with a past experience. If you just sit there and stare at it over and over, you won’t get very far. Try taking parts of your problem and rearranging them in different ways – literally and figuratively.
When I play Scrabble, I constantly shuffle the order of the tiles. It isn’t that I hope the letters will by chance spell a new word, but that different letter combinations will remind me of words or pattern of words that might be a part of a solution. If you are looking at a drawing of something that just doesn’t make sense, try drawing it upside down, changing colors, or changing perspectives. For example, when I was thinking about how different patterns in V1 could lead to invariant representations in IT, I was stuck. So I flipped the problem around and asked how a constant pattern in IT could lead to different predictions in V1. Inverting the problem was immediately helpful, ultimately leading to my belief that V1 should not be viewed as a single cortical region.
If you get stuck on a problem, go away for a little while. Do something else. Then start again, rephrasing the problem anew. If you do this enough times something will click sooner or later. It may take days or weeks, but eventually it will happen. The goal is to find an analogous situation somewhere in your past or present experience. To succeed you must ponder the problem often but also do other things so the cortex will have the opportunity to find an analogous memory.
Here is another example of how rearranging a problem led to a novel solution. In 1994, my colleagues and I were trying to figure out how to enter text on handheld computers. Everyone was focused on handwriting recognition software. They said, “Look, you write things on pieces of paper, you should be able to write the same way on a computer screen”. Unfortunately, this turns out to be really hard. It’s another one of those things that computers aren’t good at, even though brains find it quite simple. The reason is that the brain uses memory and current context to predict what is written. Words and letters that are unrecognizable on their own are easily recognizes in context. Pattern matching with computers is not sufficient to the task. I had designed several computers that used traditional handwriting recognition but it was never good enough.
I struggled with how to make the recognition software work better for several years and was stuck. One day I stepped back and decided to look at the problem from a different perspective. I looked at analogous problems. I said to myself, “How do we enter text into desktop computers? We type on a keyboard. How do we know how to type on a keyboard? Well, actually, it’s not easy. It’s a recent invention and it takes a long time to learn. Touch-typing on a typewriter – style keyboards is hard and not intuitive, it isn’t at all like writing – yet millions of people learned how. Why? Because it works. My thinking continued by analogy, “Maybe I can come up with a text input system that is not necessarily intuitive, that you have to learn, but people will use it because it works.”
Literally, that’s the process I went through. I used the act of tying on a keyboard as an analogy to figure out how to enter text with a stylus on a display. I recognized that people were willing to learn a difficult task (typing) because id was a reliable and fast way to enter text into a machine. Therefore if we could create a new method of entering text with a stylus that was fast and reliable, people would use it even though it required learning. So I designed an alphabet that would reliably translate what you wrote into computer text; we called it Graffiti. With traditional handwriting recognition systems, when the computer misinterprets your writing you don’t know why. But the Graffiti system always produces the correct letter unless you make a mistake in writing. Our brains hate unpredictability, which is why people hate the traditional handwriting recognition systems.
Many people thought Graffiti was a sensationally stupid idea. It went against everything they believed about how computers should adapt to the user, not the other way around. But I was confident that people would accept this new way of entering text by analogy to the keyboard. Graffiti turned out to be a good solution and was widely adopted. To this day I still hear people claim that computers should adapt to users. This isn’t always true. Our brains prefer systems that are consistent and predictable, and we like learning new skills.
Can Creativity Lead Me Astray? Can I Fool Myself?
False analogy is always a danger. The history of science is rife with examples of beautiful analogies that turned out to be wrong. For example, the celebrated astronomer Johannes Kepler convinced himself that the orbits of the six known planets were defined by the Platonic solids. The Platonic solids are the only three-dimensional shapes that can be constructed entirely out of regular polygons. There are exactly five of them: tetrahedron (four equilateral triangles), sextahedron (six squares, aka cube), octahedron (eight equilateral triangles), dodecahedron (twelve regular pentagons), and icosahedron (twenty equilateral triangles). They were discovered by the ancient Greeks, who were obsessed with the relationship of mathematic and the cosmos.
Like all Renaissance scholars, Kepler was heavily influenced by Greek thought. It seemed to him that it couldn’t possibly be a coincidence that there were five Platonic solids and six planets. As he put in his book The Cosmic Mystery (1598): “The dynamic world is represented by the flat-faced solids. Of these there are five: when viewed as boundaries, however, these five determine six distinct things: hence the six planets that revolve about the sun. This is also the reason why there are but six planets.” He saw a beautiful but entirely false analogy.
Kepler went on to account for the orbits of the planets in terms of nested Platonic solids that were all centered on the sun. He took the sphere defined by Mercury’s orbit as his baseline and circumscribed it with octahedron. The tips of the octahedron defined a large sphere, which gave the orbit of Venus. Around Venus’s orbit he circumscribed an icosahedron whose outer tips yielded the orbit of the Earth. The progression continued: a dodecahedron drawn around the Earth’s orbit gave the orbit of Mars, a tetrahedron drawn around Mar’s orbit gave the orbit of Jupiter, and a cube around Jupiter’s orbit gave the orbit of Saturn. It was elegant and beautiful. Given the limited precision of astronomical data in his day, he was able to convince himself that this scheme worked! (Years later, Kepler realized ha had been mistaken after he got hold of the high-precision astronomical data of his deceased colleague Tycho Brahe, which proven that planetary orbits are ellipses, not circles).
Kepler’s excitement serves as a cautionary tale for scientists, and indeed for all thinkers. The brain is an organ that builds models and makes creative predictions, but its models and predictions can as easily be specious as valid. Our brains are always looking at patterns and making analogies. If correct correlations cannot be found, the brain is more happy to accept false ones. Pseudoscience, bigotry, faith, and intolerance are often rooted in false analogy.
The article is a part of Jeff Hawkins’ book “On Intelligence” reposted here with author's permission.
Jeff Hawkins on how brain science will change computing: Watch the video HERE.
