Bridging the forecasting gap

Improvements in data and climate science mean forecasters are able to predict patterns much further ahead and in far greater detail. Melissa Bray looks at what this may mean for farmers.

Waikato dairy farmer Andrew Reymer likens a farmer’s response to drought to building a bridge.

It’s all about knowing how long the bridge has to be, he says.

It rained just twice on his property between December and May during the summer drought of 2020.

Reymer, like his neighbours, managed his way through the five-month dry spell.

But he had to juggle a range of decisions around factors such as how many cows to dry off, how many to sell, what type of feed he needed to buy in and when to do it.

He says having a better idea of how long he’d have to wait for the next rain – how long he would need to build that “bridge” – would make decision-making much easier.

“You have to know how long it’s going on for. If you’re confident about the length of the bridge, you can plan for it.”

Locals expect dry summers at Whanawhana in Hawke’s Bay, but sheep and beef farmer Louis Beamish says the summer of 2020 was drier than most with no significant rain from January through to May.

“We had a good team, and we knew what we had to do,” he says.

Flock of sheep on Hawkes Bay farm are fed maize and baleage during 2020 drought.

The level of work increased significantly. It took Beamish and his team half a day, every day to feed all of the animals.

“I only have confidence in maybe a week’s forecast, and even that may change, but every day I was looking at the forecast.”

Weather is every farmer’s constant companion, and planning for what the weather may do is becoming increasingly important.

Climate change is exacerbating extreme weather events and NIWA’s long-term climate projections point towards an increasing risk of prolonged dry spells across much of Aotearoa New Zealand in the decades ahead, particularly in northern and eastern regions.

The implications of such projections are significant – both for individual farmers and the country at large.

Treasury figures show that drought has cost New Zealand more than $5 billion in lost agricultural production over the past 15 years.

The Waikato region went 61 days straight without rain during the 2020 drought, and the impact on the region was more than $150 million.

It’s no easy task managing any property through a prolonged dry spell. But what if there was a tool that could help farmers like Reymer and Beamish accurately project how long the “drought bridge” is likely to be?

Working dogs, Rum and Blue (white tipped tail), take a welcome drink on Whanawhana Station in Hawke's Bay during the 2020 summer drought.
Neelesh Rampal shows how NIWA35 uses artificial intelligence to create high resolution forecasting at 5km (right), from a low resolution input forecast at 50km.

This is where NIWA’s new drought forecasting project, NIWA35, comes in.

Auckland-based data scientist Neelesh Rampal has been working with his NIWA colleagues, meteorologists Ben Noll and Tristan Meyers. The trio have designed a model that will deliver accurate projections of dryness and drought risk out to 35 days ahead and at much higher resolutions than currently available.

“The model will provide farmers with a reasonable estimation of a best and worst case scenario over the next five weeks so that they can scenario-plan effectively. It’s one-of-a-kind in New Zealand and is novel on an international scale,” Noll said.

“To continue with the bridge analogy, this new guidance will provide a range of dimensions for farmers, as well as a most likely length.”

NIWA has joined forces with the Ministry of Primary Industries (MPI) to implement a drought forecasting tool and is working with farmers and growers to fine tune the model this summer, with plans to launch rolling 35-day drought forecasts, at 5km high resolution radius, to the public next year.

NIWA35 comes with a history. In 1999, NIWA started producing the Seasonal Climate Outlook (SCO). This offers regional projections for temperature, rainfall, soil moisture and river flows which look out three months into the future. The basis of the SCO is created using historical climate patterns, particularly sea temperatures.

“Predictability on the three-month scale comes from ocean memory, or the persistence of sea temperatures over long periods,” says Noll. “An example of this is the El Niño Southern Oscillation.”

These three-month forecasts and soil moisture reports are necessarily broad scale and heavily qualified. However, they still provide useful detail for landowners. The soil moisture report, for example, is published every week on Federated Farmers' rural news site, FedsNews.

But the meteorological search for more accurate, detailed projections continued and in 2009, NIWA climate scientists began running a model that took global data and provided a 15-day local forecast for New Zealand. It was a significant advancement, and its accuracy has steadily improved.

Despite this, accurately projecting weather patterns, at an effective local scale, in the space between the next two weeks or the broad three-month outlook like the SCO has to-date eluded forecasters.

As Noll says: “It is the challenging middle ground – the sub-seasonal space.”

The development of NIWA35 however means that current sub-seasonal void is about to be filled. Recent advances in both atmospheric modelling and artificial intelligence, coupled with the processing power of the latest generation supercomputers, mean that the science of forecasting can be pushed out much further and with a higher degree of both certainty and resolution.

“If the 2009 model is a Ford Model T, the NIWA35 model is a Tesla,” says Noll.

Data collection to create NIWA35 is extensive, and researchers are leveraging off a United States model established by the National Oceanic and Atmospheric Administration (NOAA).

NIWA meteorologist Ben Noll.
Looking out across the normally green fields of the Waikato during February 2020. The drought cost the region more than $150 million in lost production.

The raw data is pulled down onto NIWA’s high-performance computer from NOAA every day. It is at a 50km resolution and a 31-member ensemble, which means that data is drawn from 31 different instances of the model.

This supplies 31 different scenarios for the forecast. When these scenarios agree, forecast confidence is generally higher, and when they diverge you’ve got lower confidence.

“Our initial review of model skill during summer 2021–22 showed that it picked up the extreme dryness that developed over New Zealand during January,” says Rampal.

As well as extending out to 35-day forecasts, NIWA35 projects conditions at a much higher resolution. It scales the ‘coarse’, global 50km input grid down to a 5km radius over New Zealand. This resolves the country’s complex topography far more effectively.

“It’s challenging to make decisions from something coarse and 50km is not representative of the wide diversity of our terrain in New Zealand,” says Rampal.

This is where data science and the power of deep learning really comes into play. Drawing on convolutional neural networks (CNN) to extract information, the model can extract far more complex relationships than principal component analysis.

“It’s different from dynamical downscaling because we’re not using physics. We’re trying to learn physics from what’s happened in the past,” says Rampal.

“It enables us to look at what’s happened in the past and fill in the gaps.”

This allows for judgements to be made over a smaller area and forecast down to a 5km radius, with the prospect of getting resolutions as low as a 1km radius.

Downscaling has been done before in the forecasting space, but deep learning adds speed of process. “We’re talking at least 1,000 times faster,” says Rampal.

In his media release earlier this year announcing MPI was partnering with NIWA to develop better drought forecasting systems, Agriculture Minister Damien O’Connor said the joint project was important to help farmers and growers get their businesses ready for future climate conditions.

“Knowing well in advance when dry conditions are heading your way means you can cut your cloth accordingly at critical times on-farm."

“Improved forecasting will alleviate some of the financial and mental burden that drought puts on farmers and growers."

“It will also make our primary industries more resilient, productive and sustainable.”

Over in Hawke’s Bay, Louis Beamish will welcome anything that helps landowners to be more proactive and manage their risk during those dry summer months.

“Any information on an extended forecast is useful,” he says.

For farmers like Reymer, NIWA35 would bring confidence about the length of the bridge.

“I could completely change my plan – forward contract palm kernel and silage, wean the calves in December, de-stock early, a number of things.”

He says during the summer of 2020, rain seemed to be predicted every week – but it didn’t turn up.

“We could see rain coming out of the tropics, but it kept getting pushed off. We need to know if the weather system is going to hit.”

And five kilometres changes so much. “We’re close to Mount Pirongia. They can get great rain and we still get none.

“Weather is the number one thing on your mind all of the time,” says Reymer.

This story forms part of Water and Atmosphere - December 2022, read more stories from this series.