Forecasters have an awful record in predicting energy markets

Forecasting is discredited. The army of futurologists marches on producing ever more scenarios of feast or famine but the forecasts they come up with receive and deserve less and less attention.

For the energy sector, forecasting has become a mildly amusing hobby of little relevance to real business decisions. How then can companies plan long-term investments that are designed to produce returns over decades?

There are two reasons why the world has lost confidence in forecasts. First: the record is awful. Remember the predictions of oil at $200 a barrel or the view that nuclear energy would be so cheap that no one would bother to meter its use?

The second reason is that events, especially around technology, are moving so rapidly that it is difficult to keep up with what is happening already, never mind what could come next. Artificial intelligence, energy storage and, at a very different level, the spread of religious fundamentalism are all potential game changers in the energy market. Yet predictions of how and when their influence will be felt are no more than guesses.

My lesson in the limits of forecasting came early. It was 40 years ago, in 1978, as a young graduate that I wrote a detailed paper about Iran’s emergence as a major industrial and economic power by 1990.

The energy sector has moved from an age of scarcity, in which prices of finite resources could reasonably be expected to rise over time, to the new reality of an age of plenty

The paper was the result of a lot of work. I read the plans of the Iranian government and of numerous private companies. I gathered the data on the country’s resources and skills. My newly minted Cambridge economics degree gave me the confidence that I had all the necessary skills to predict the future.

I even presented the paper to a study group including the head of the Middle East Department of the British Foreign Office. A kindly man, he said my paper was very interesting and asked when I had last been in Tehran.

The answer, of course, was never. The paper was written from the safety of my desk in London, EC4. Six months later, Ayatollah Khomeini flew into Iran and the rest is history.

When I read forecasts now I am always tempted to ask the author when they were last in Venezuela or Saudi or whichever country they are talking about.

If forecasting is discredited, what can companies do? The answer is to focus on competitiveness. The energy sector has moved from an age of scarcity, in which prices of finite resources could reasonably be expected to rise over time, to the new reality of an age of plenty.

Despite price volatility, there is no shortage of oil, or gas, or coal. Public policy is supporting low-carbon sources of energy supply — from nuclear to renewables — in many countries and wind and solar are enjoying dramatic reductions in costs that should soon make them competitive without subsidies.

Technology is adding to supply on all fronts — including from hydrocarbons such as shale — and is also limiting demand. For example, while the UK economy has grown by about 1.5 to 2 per cent a year over the past decade, total energy consumption is down by 17 per cent.

The losers are likely to be projects with high upfront costs and long payback periods such as new nuclear or tidal power projects

The only way for companies to compete, therefore, is to be at the lower end of the cost curve. Even that is not a perfect guarantee of success because technology might produce new competitors. It is, however, the only viable starting point.

The other element of the response to the death of forecasting is a shortening of the time period in which projects are expected to deliver returns. The longer that period runs, the more likely it is that disruptive change will undermine your plans.

This approach gives an advantage to large-scale, low-cost oil and gasfields and easily constructed developments such as solar farms.

The losers are likely to be projects with high upfront costs and long payback periods such as new nuclear or tidal power projects. In the absence of willing private investors, such businesses are becoming reliant on state funding which itself is in scarce supply.

While futurologists apply their skills in different ways — in particular to identifying risks and their potential impact — the old art of forecasting will be remembered as the product of a time when the world seemed set and certain. We thought we knew how the world worked and what should come next.

Now, all that we know is that we do not know.

Nick Butler is visiting professor and chair of the Kings Policy Institute at Kings College London and a regular contributor to the Financial Times

Explore the rest of our stories on Energy Producers

• Solar and wind power surge marks production shift
• Big Oil harnesses power of data analysis to ensure survival
• High costs and renewables challenge the case for nuclear power
• Natural gas vies for role in low-carbon economy
• Coal fades in developed world but is far from dead in Asia