Hipsters go to TED for inspiration about what the future will bring, but the world’s greatest physical scientists go to Solvay. It was there in 1927, at the fifth Solvay Conference that Albert Einstein famously said, “God does not play dice with the universe,” to which Niels Bohr retorted, “Einstein, stop telling God what to do.”

Bohr’s quip was much more than a clever line, but a tipping point in the world of physics toward a quantum world of probabilities rather than the deterministic universe that Einstein preferred.  Even Einstein, when faced with a flaw in his core beliefs, was unable to adapt and it doomed the latter part of his career.

It makes you wonder what chance the rest of us have.  We all like to think of ourselves as innovative and agile, but when our core beliefs are called into question, the cards are stacked against us.  Our brain chemistry, social networks and even our basic instinct for survival will resist the change.  To master the art of the shift, we first need to master ourselves.

The Stubbornness Of Old Ideas

We tend to think that we experience the world as it is.  We see and hear things, store them away as knowledge and then take new facts into account.  Yet that’s not what we really do.  In fact, we filter out most of what we experience, so that we can focus on particular points of interest.  In effect, we forget most things so we can focus on what we feel is most important.

What’s more, the effect is cumulative.  What we think of as knowledge is really connections in our brains called synapses which develop over time.  These pathways strengthen as we use them and and degrade when we do not.  Or, as scientists who study these things like to put it, the neurons that fire together, wire together.

So as we go through life and learn the ways of the world, we become less able to imagine other possibilities.  Our mental models become instinctive and standard practices become “the right way to do things.”  This effect becomes even stronger and more pervasive if we see our mental models as being responsible for our success, as Einstein did.

Einstein lived almost a full three decades after his great debate with Bohr and, in that time, quantum mechanics became the dominant physical theory.  Yet although it was one of his early papers that launched this revolution, he was never able to accept it and became in later life, as Oppenheimer put it, “a landmark, but not a beacon".

The Network Effect

While our previous experiences tend to blind us to new developments, those around us will help reinforce common beliefs.  In fact, a series of famous experiments done at Swarthmore College in the 1950’s showed that we will conform to the opinions of those around us even in if they are obviously wrong.

Yet few of the issues we face are that clear cut.  We tend to cling to our beliefs because they have worked for us.  Einstein’s belief in a deterministic universe led to his great discoveries and worldwide acclaim.  How could he abandon the very thing that led him to such enormous success?

As Thomas Kuhn pointed out in The Structure of Scientific Revolutions new paradigms don’t emerge whole, but first arrive as a series of quirky anomalies that are easy to dismiss as “special cases” that we can work around.  This usually works pretty well for a while and things go on much as before.

So even if we notice that something is awry, that things aren’t quite what we thought they were, we will most likely forget about it and get back to business.  After all, not only do we believe in our present working model, so do those around us.  The world is a messy place and every rule has exceptions.  Life goes on.

The Efficiency Paradox

Another barrier to adaptation is that change incurs real costs.  How much time, effort and resources do we want to expend on a hunch? Especially when our own instincts and those around us tell us that our present course is the right one?

As General Stanley McChrystal explained in Team of Teams, building the kind of agility and interoperability it takes to respond to disruptive change almost always results in diminished efficiency, at least in the short term.  That’s the efficiency paradox.  To effectively deal with disruption, we need to sacrifice our effectiveness in normal, stable environments.

Most of this understand this principle intuitively, but in practice our instinct for loss aversion takes over.  Evolution engineered us to survive, not to seek out the great yonder.  So we are wired to avoid incurring losses, even if we have an equal or better chance of making gains.

In McChrystal’s case, he had to take people out of their units so that they could contribute to better coordination.  For example, he embedded intelligence officers in commando units and vice versa.  He also took some of his best operators out of the field altogether and made them liaison officers, usually a role reserved for those past their prime.

These were not easy changes to make and they most likely diminished the effectiveness of individual units.  However, as a collective, his forces increased their efficiency by a factor of seventeen, measured by the amount of raids they were able to execute.

Managing For Disruption

The story of McChrystal is an inspiring one, all the more so when you realize that he had no guarantee of success.  That he was able to manage it successfully is a tribute to his skills as a leader.  Most change efforts do not go nearly as well.

The problem is that disruption always seems strange to us, because it always starts with them. It is those on who operate outside of our social networks and everyday experience that have no loyalty to the existing model or the status quo.  They are unencumbered by synapses that fire in unison with the present paradigm and unconnected to those who benefit from it.

That’s why we need to manage not for stability, but for disruption.  We need to stretch ourselves into the realm of the uncomfortable and unlikely, to break bread with those whose neurons fire along different pathways and who travel in circles detached from our everyday experience. We cannot live our lives surrounded by the familiar and expect to embrace the new and different.

So in order to adapt we must, as Whitney Johnson puts it, disrupt ourselves.  To seek out people, places and experiences that challenge our core beliefs.  The status quo has many champions—our brain chemistry, our social networks and our sense of security.  We can only embrace the future if we are willing to make the effort to break free of the past.

This article originally appeared at DigitalTonto.

Follow Greg on Twitter  @Digitaltonto