Co-authored by John Norbury (Oxford University)
"Isn't it strange how we like to regard weather forecasting as a uniquely incompetent science - as though this subject of vital economic and social importance can attract only the most inept researchers armed with bungling, bogus theories?" So wrote Philip Ball in his review of our new bookInvisible in the Storm: the role of mathematics in understanding weather. Well, the last few days have seen the critics out in force: the UK Met Office held a meeting on Tuesday (June 18) to reappraise the success of its seasonal forecasts, and the bloggers have been busy - very busy - reminding us of the promised barbeque summers that never were. And things are little better for the forecasters in the United States, with several articles in recent months highlighting the "second rate" performance of their computer predictions.
In order to understand the challenges forecasters face, we need to look at the history behind modern forecasting. The story begins in 1904 when Vilhelm Bjerknes, a Norwegian physicist, published a paper describing how weather prediction could be formulated as a problem of maths and physics.
Turning Bjerknes's ideas into a practical scheme requires mathematics and modern technology. But it is maths - the maths of a totally different problem - that not only reveals the immense scale of the challenge Bjerknes was taking on, but also how to deal with it.
In the 1890s mathematicians and astronomers tried to decide if the Solar System was stable. That is, could a collision between an asteroid and a planet change the future orbits of all the planets, perhaps resulting in Earth being flung into outer space?
The stability of the Solar System became a hot topic, but how could anyone answer such a question? The possible configurations of the planets and asteroids are endless, and the equations allow for ferociously complicated behaviour, so surely any mathematical calculations are intractable?
The breakthrough was made in the 1890s by a star mathematician of Paris, Henri Poincaré, and his work led to the discovery of chaos. However, Poincaré confined his analysis to a study of the motion of just three planets under their mutual gravitational force, which has become known as the three-body problem. And even this problem is phenomenally difficult.
The essence of Poincaré's idea is to calculate several forecasts, called an ensemble. Each ensemble member is a forecast started from a state that is slightly different from all the other members. In weather prediction, these different states reflect our ignorance of exactly how any given weather system forms. If the ensemble predicts similar outcomes, we can be reasonably confident in any one of them, but if they produce very different scenarios, then the disparity indicates the physics controlling the weather system are more complex than usual and more analysis is needed.
Bjerknes's original scheme involves the calculation of seven numbers, which specify seven basic variables - wind speed and direction (3 variables), pressure, temperature, air density and humidity - at many thousands of data 'pixels' in the atmosphere. Modern computer power means that the resolution of our 'digital image' of the atmosphere is such that Bjerknes' scheme becomes more than 10 million equations in 10 million unknowns, and each one is potentially as difficult to solve as the three body problem. This is why ensembles are so useful when predicting the weather.
But the use of ensembles involves a culture shift: instead of trying to predict exactly what tomorrow's weather will be, we instead predict what it will most probably be. The nature of climate may also be understood in probabilistic terms. It is not the exact sequence of weather that has predictability in the years ahead, but rather aspects of the statistics of the weather - for example, in 20 years' time will our summers be warmer and wetter? Though the weather on any one day may be entirely uncertain so far into the future, the persistent influence of slowly evolving sea surface temperatures may change the odds for a particular type of weather occurring on that day. In a rough analogy to the process of throwing dice, the subtle effects of the coupling between the atmosphere and its surroundings can be likened to 'loading' the dice. On any given throw we cannot foretell the outcome, yet after many throws the biased dice will favour one particular outcome over others. In this way we may still predict changes in seasonal behaviour years ahead, even though we cannot predict the actual weather at a given place on any given day.
Ball concludes his review by pointing out that the weather forecasters may still have to brave the occasional rebuke when the storm that wasn't forecast wreaks havoc in a community, but these episodes are becoming increasingly rare. And, if we use such episodes to discredit the forecasters, it will be at our detriment.