Limits to Knowing

October 2011

By Patrick Crehan, Director, Club of Amsterdam, CEO and Founder, Crehan, Kusano & Associates

The way we think about the future has immense influence and impact on both our professional and personal lives. This is especially true for those who work in positions of responsibility for organizations and the people in them. They are the ones who decide on a regular if not continuous basis what time and money, human and material resources should be allocated to which activities so as to ensure optimal outcomes for themselves and their families, for their organizations and society as a whole.

The tools we employ to think about the future are constantly evolving. Our ability to gather store and process information about the past and the present state of the world is expanding at an extraordinary rate. Several pas Club of Amsterdam events have looked at the extraordinary pace of progress in sensing, connecting and computing. These have helped our members explore the consequences of how our ability to sense the present, combine it with knowledge of the past and simulate the future, has expanded at an extraordinary rate.

But it is worth while taking a look at the limits of our knowing about the future to see if we really understand how to use these powerful tools and ask if we really are on the right track, if we really are mastering the tools required to help us design and build the better worlds we want to create.

Swept up in the euphoria of technological progress, there is a risk of “irrational exuberance”, that we might overlook small issues of great consequence. For this reason it is useful occasionally to go back to basics and take stock of where we are and try to filter what is real from what is mere illusion.

The first reality check concerns the nature of our ability to model the world, simulate it and make predictions. Despite the extraordinary progress we have already made, and the very reasonable expectation that by the end of this decade we will have succeeded in feats as complex as simulating the workings of an entire human brain, there are real limits to what we can simulate and what we can predict. So far I am aware of at least 3 hard barriers to success in modeling and simulation, and there may be many more.

The first is a demonstration by the philosopher Carl Popper, about the impossibility of predicting the future. His argument is very elegant and relies on special relativity. Effectively he provides a proof that if the world is governed by the principles of relativity, then even if we have perfect theories, and infinitely fast computing capabilities, we will still never have enough information available to always make accurate predictions even arbitrarily small times into the future. Of course we will get away with ‘good enough’ most of the time, but he explains that there is a hard barrier between that and being able to guarantee getting it right every-time.

The second barrier has to do with the discovery of quantum mechanics and has to do with the ‘knowability’ of nature. The initial insights came from the work of Heisenberg and have been debated ever since. The general consensus is that it is impossible to simultaneously possess knowledge of arbitrary accuracy about the state of the physical world. In physical terms it means that we may know the position of a particle with arbitrary accuracy at a given time, but only by sacrificing accuracy in our knowledge of its state of motion. This is a hard limit on what we can know about the world and seems to be no way around it. Once again we can get by pretty well for most intents and purposes but bear in mind that many modern engineered products rely on relativity and quantum physics for their operation. Both relativity and quantum physics have left the realm of science and entered the realm of engineering many years ago. So these limits we refer to are real, impact our work and are faced by engineers every day.

The third major barrier is one which only really emerged or became clear in the 1970s with the discovery of what is called ‘deterministic chaos’. This has to do with a form of ‘unknowability’ that afflicts even old fashioned Newtonian systems. It does not rely on artifacts of relativity of quantum mechanics. It would exist even if quantum mechanics or relativity were not true. This insight into the limits of ‘knowability’ go back to the discovery of dynamical systems that can be modeled perfectly, for which solutions can be shown to always exist, but which can never be calculated by any algorithm with any degree of accuracy. These systems often appear random or chaotic, when they are absolutely deterministic. Even though we know everything about what drives these systems, we also know that we cannot simulate them in any reliable way. In practice what happens is that arbitrarily small errors in the measurement of the parameters of the system lead to arbitrarily large errors in the results of a simulation.

These are three hard barriers to what we can know from measuring modeling and simulating the world. There are others. Despite these limits we do pretty well and simulation can be a very useful tool when used in the right way. The spectacular collapse of Long Term Capital Management (LTCM) run by Nobel Prize wining economists is the text-book case of what can happen when the models are applied outside of their domain of applicability. We could move on from discussion about errors in simulations based on models to the impact of errors in the model itself, but that is a subject for another day.

Instead it is interesting to look at issues relating not so much to our ability to predict the future, but to our ability to control it. The future is highly subjective. No one creates it alone. Chance requires the cooperation or complicity of a great number of actors. It is an iron rule of change and it is true whether we are talking about change on the level of the global economy, a business unit or our personal circle of family and friends.

To affect change it is not enough for one person to know about the future, they need to bring along everyone else by forming change-coalitions for want of a better word. The starting point is creating and sharing relevant knowledge. This touches upon the philosophy and mission of the Club of Amsterdam, and there are many techniques for doing this. But even this is not enough. Given all the knowledge and understanding in the world, people may then need to act. This is the real barrier to making change happen, especially when we are looking at long-terms issues that do require an immediate solution. Such issues tend to get put off until it is too late. Making change happen requires not so much progress in simulation but progress in understanding factors such as motivation, confidence, courage, the will to act in ones own interest.

This was why it took about 50 years before clear and overwhelming evidence linking smoking to lung cancer became generally accepted. It is why even today people do things like smoking that they know will shorten their lives and limit the time they have to enjoy the good things of this life. Arguable it is also one of the reasons why progress is so difficult on issues such as climate change.

Despite the incredible progress we have made in out ability to collect and analyze data, model and simulate the world, make predictions about the future, we are still very poor I moving from knowledge to action.

The future of the future needs to include initiatives that more explicitly not only the barriers to knowing but the barriers to acting when the knowing battle has been won.