From sky to server

Ominously large and bloated with moisture, angry clouds sweep across the Fiordland coast.

Winds drive the storm system higher and torrential rain begins thudding into the earth below. It doesn’t take long before heavy drops start falling plumb into the mouth of the Takahe Valley rain gauge, in the Murchison Mountains just west of Lake Te Anau.

The raindrops are about to make their mark at a national scale.

In a matter of seconds, they will trigger a response from the automated gauge and be registered in an ever-growing climate database that plays a key role in decision making across the country.

The raindrops falling at NIWA’s Takahe Valley weather station collect in a small, seesaw-like receptacle called a tipping bucket. Bit by bit, the water level inches upward. The bucket over balances, dumps its load and resets for another take.

A datalogger records the tip and pings the information to a server at NIWA’s Christchurch office. From there it travels along a high-speed network to NIWA’s Wellington office, where it is automatically entered into long-term storage in New Zealand’s national Climate Database – more commonly known as CliDB.

For a meteorologist, CliDB is the motherlode. It stores weather data from all over New Zealand, Antarctica and the Pacific Islands. The earliest recordings are from the mid-1800s, with the latest information flowing in from a network of hundreds of stations like that in Takahe Valley.

The database is growing fast. If it is raining hard in the Murchison Mountains, the automated rain gauge generates a measurement every six seconds. Ten minute and hourly readings are also taken – rain, hail or shine. Weather stations, of course, measure far more than just rainfall. Sensors put together by NIWA’s Instrument Systems division simultaneously record temperature, wind speed, air pressure and humidity. Some specialist stations measure solar radiation, and every instrument is busy feeding its own stream of data into the network.

With stations dotting the country and weather fronts hitting all points of the compass, the numbers quickly add up. In January alone, more than 1.5 million new rainfall data points flooded into CliDB.

In total, CliDB is connected to more than 600 weather stations – a network that stretches out across New Zealand, north into the Pacific and as far south as Antarctica. Many of the facilities are maintained by other agencies, such as MetService, the Department of Conservation and regional councils.

The database also incorporates a host of additional weather information provided by sources ranging from drifting ocean buoys and shipping vessels to historic observations gathered from lighthouses or aircraft.

Readings from seven key sites – Auckland, Masterton, Wellington, Nelson, Hokitika, Lincoln and Dunedin – are particularly important. Stations at these locations have been benchmarking New Zealand’s average annual temperature since 1909. NIWA’s seven station temperature series has confirmed NZ’s average annual temperature has increased by about 1°C over the past 100 years.

NIWA is the custodian and the curator of all the information flooding into CliDB. The data is rigorously quality controlled and audited to international standards, with machine learning playing an increasingly significant role as computers are programmed to hunt for gaps and flag anomalies in the dataset.

Once checked and curated, the data is made available to end users through a range of software. Much of the information is freely available through a web app called CliFlo – developed by NIWA’s principal climate technician Errol Lewthwaite.

With more than 50,000 registered users, CliFlo is accessed hundreds of times an hour by individuals and organisations. MetService is a major user, along with commercial entities in the primary, engineering, energy, insurance and consulting sectors. CliFlo data is also used by criminal investigators and even competitive pigeon racers.

The volume, and range, of real-time data feeding into CliDB is pivotal in letting meteorologists such as NIWA principal forecaster Chris Brandolino more accurately analyse and forecast weather conditions and trends.

Brandolino and his team use innovative visual software to transform the raw details into user-friendly forecasts for both public and commercial clients.

The team is currently working with Emirates Team New Zealand using observations pulled from CliDB to forecast race day wind shifts on the Hauraki Gulf at unprecedented scale.

Data from CliDB is used to produce a host of other targeted climate products, including national soil moisture maps, drought indicator maps and NIWA’s three-month seasonal climate outlooks.

The historic records stored in CliDB also let climate researchers understand trends over much broader time frames. NIWA climate scientist Dr Trevor Carey-Smith says weather observations are vital for understanding New Zealand’s long-term climate.

“The amount of rainfall falling in a typical year can vary quite a bit. The more data we have, the better our understanding of that normal variability. And, unless we know about variability, it’s hard to determine trends.”

Every six hours, observations from CliDB are pulled into a weather model called NZLAM that predicts the future state of the atmosphere. This gives researchers information to run forecast models down to scales of 300 metres.

Using the computational power of NIWA’s supercomputers, weather observations are assimilated into sophisticated numerical models that capture the current state of the atmosphere with as much detail as possible.

NIWA researchers are constantly comparing observations and forecasts from the past – using the difference between the two to improve their models.

Capable of processing more than 2000 trillion calculations per second, the Cray supercomputers can run atmospheric models to forecast the weather and generate climate scenarios decades into the future.

NIWA uses the national climate database to help communities and industries inform and improve decision making across New Zealand. CliDB is a fundamental part of that decision-making infrastructure.

The database helps fire crews on the ground read conditions to battle forest flames. It means farmers and power companies can plan months ahead for drought or low water flows, and it is critical to modelling the projections communities and businesses need to prepare for climate scenarios decades into the future.

Given all this, it is about time this vital national database – and its humble rain gauges – got their moment in the sun.

This story forms part of Water and Atmosphere - February 2021, read more stories from the series.