Ag Weather Forum

Most of you are familiar with the bad image that the U.S. NOAA weather forecast model has, much of it stemming from the European forecast model's more-accurate depiction of last fall's Superstorm Sandy track into the New York-New Jersey coast. With that in mind, I think you'll be interested in this recent article from the Washington Post, written by University of Michigan Atmospheric Science professor and former NASA Senior Executive Service member Richard Rood.--Bryce

Twitter @BAndersonDTN

To be the best in weather forecasting: Why Europe is beating the U.S.

By Richard B. Rood

The superior performance of the European forecast model relative to the U.S. GFS model, in high impact storms affecting the East Coast - namely Sandy and Snowquester - has raised the question: why is the U.S. model - generally - not as good? Guest contributor Richard Rood offers an in-depth perspective...

As early as 1995, the weather forecasts from the European Centre for Medium-Range Weather Forecasts (ECMWF) were emerging as higher quality than U.S. products. U.S. scientists and science managers found this development a matter of great concern.

Their first reactions were to attribute the gap to budget, time-criticality of mission, computers and visitors programs. For example, ECMWF often brought new capabilities into their forecast system using visiting scientists. It was noted that many of these visiting scientists were from the U.S. and had to “fly over Washington” (i.e. NASA and NOAA) to go to Reading, England. A recommended response, therefore, was to start visiting scientist programs to stop “the fly over Washington problem.”

The high quality of ECMWF’s forecast for Hurricane Sandy in fall of 2012 has brought the forecast gap back into discussion (for example, Science Magazine, USA Today, the Weather Channel, Cliff Mass Weather Blog). Much of the current discussion focuses on familiar perceived differences, with much of the reporting on U.S. deficiencies in computational resources. However, there are far more meaningful underlying practices and approaches to the provision of weather and climate products that distinguish ECMWF.

From 1992-2005, I performed research, worked with and directed organizations at NASA and NOAA whose missions include weather forecasting and supercomputing. Over those years I spent many days at ECMWF. David Burridge, then Director, and Tony Hollingsworth, who held many leadership positions, graciously provided me insights into the management of scientific organizations.

In 2000, I was the lead author on a report for the Office of Science and Technology Policy on U.S. management of climate modeling and supercomputing. Those conversations at ECMWF, as well as formal interviews, largely framed the management recommendations in that 2000 report. I have contributed to several more reports on this topic in the past decade, and many of the conclusions of that year 2000 report remain true today.

ECMWF is, by U.S. standards, a small institution with a tightly focused mission. It can manage operations, research and infrastructure in a unified way to meet their mission. It has ownership of its budget and can direct that budget on whatever is required to make the best forecast. There is an internalized incentive structure that focuses people’s efforts on their products. They have benefitted from stability, strong management and focused leadership. They know how to satisfy their customers.

Those organizational attributes are all part of ECMWF’s success. A more important attribute, however, is the scientific culture of ECMWF. ECMWF has integrated research and operations together with institution-wide attention to science-based, validated products. This stands in contrast to the United States, where we draw sharp contrasts between research and operations. In the United States scientists and science-program managers place high value on research, especially basic research. There is lower value on use-driven research; synthesis of research to provide products; the complex entanglement of observational, computational and scientific capabilities that must be brought together to produce a product; and the operations, monitoring and assessment of those products.

ECMWF knows to invest in software and to spend on computers. For example, when faced with a paradigm shift in computational technology, as in the late 1990s, ECMWF invested, far in advance, in both software and sustained vendor-based benchmarking in order to be ready when the paradigm shift occurred. This practice has continued. In the United States, we remain largely reactionary to the evolution of high-performance computing systems. Therefore, each shift in computing technology is a moment in time that the forecast gap is increased.

ECMWF has much of its focus on a single software system, the Integrated Forecast System. This system supports research and product generation. This focus is in contrast to U.S. centers, where often several software systems are in simultaneous use. If a promising result were found in one of the research models, then there is no easy path to operations. The attention at ECMWF to these fundamental issues of software and systems allows not only better operations, but also, supports the scientific method of investigation. It accelerates the incorporation of promising research. Hence, this management discipline leads to a scientific organization, rather than an organization full of scientists.

A weather-forecasting system is made of several major components. One component accumulates the observations and checks them for accuracy and consistency. This observation manager then presents quality-controlled observations to other components that blend the observations and the weather model together to initiate the forecast.

Examination of the performance of these components shows that effective use of the information in the observations is an essential ingredient in forecast accuracy. In fact, ECMWF pioneered the examination of failed forecasts and discovered that the forecast busts often hinged on the inclusion or exclusion of a small number of specific observations. This field of data-use is not one that is often highlighted in U.S. research programs. It is a sometimes-tedious practice that requires tight control of information across complex interfaces.

At one time I had the job of identifying at NASA which new observations I could bring into the forecast system to improve a set of known deficiencies. My group identified priority observations; however, those observations were not “owned” by an agency program manager vested in the success of that instrument. From my management’s point of view, I had to demonstrate improvement with an instrument “owned” by the program.

In contrast, ECMWF, which owns no instruments, can and does identify the observations that would most improve the forecast. ECMWF invests in observation quality control and the observation-use interface. In the past decade, ECMWF has been able to implement advanced methods that blend or assimilate observed information into the weather model. More generally, ECMWF identifies important basic research performed by the worldwide community of scientists, targets that research and consumes it into their Integrated Forecast System.

U.S. organizations with missions and aspirations similar to ECMWF’s have far less stability and are stunningly fragmented. The fragmentation is inherent in our system. U.S. agencies prefer to fund basic research performed by individual investigators; this is true for science, data systems, and computational infrastructure. How this leading-edge basic research should migrate into integrated, science-validated, products is left to fate or imagination.

The “science” budget is placed in tension with budgets for integration and infrastructure, and scientific research is usually perceived to be of highest value. The operational mission is deemed less worthy, or something that should just happen because of excellent “science.” Essential strategic capabilities are relegated to competitive programs and a proposal environment where success is uncertain. There is opportunistic reliance on sporadic pools of money, such as the stimulus funds. It’s a funding environment that can breed hostility as organizations with too broad mission statements try to prove themselves to their sponsors. Even if proposals are successful, there is no meaningful integration across proposals, and the time spent going through this process leaves the operational capability a few years behind the state of the art.

The attention to the entire weather forecasting system and the infrastructure to support its operations, quality evaluation, and customer satisfaction allows ECMWF to focus on the priorities that will improve the forecast. By absorbing research pre-determined to be of value to the Integrated Forecast System, ECMWF can focus their in-house research on the foundation that is needed to produce excellent, science-based forecasts – as well as maintaining competitive advantage. The attention to homogeneous, stable, documented infrastructure provides an inviting environment for a visiting researcher, who can reasonably expect in a short visit to contribute to the world’s best forecast system and improve career advancement at home.

It is remarkable that in the fragmented science culture of the United States, that the U.S. forecasts have consistently and systematically improved. A serious question is whether the continued piecemeal approach is sustainable. It is certainly not efficient, and there is no evidence with this approach that the U.S. can close the gap with ECMWF.

As long as our approach to funding and scientific culture supports, values, and rewards fragmentation, the benefit of additional money can be important, but the impact will be patchy. To be the best in forecasting, the U.S. must face the underlying issues of fragmentation and provide the U.S. organizations responsible for weather forecasting a stable environment in which to function.

Managers who are expected to produce the best forecasts must have the ability to align and focus on all of the elements that comprise a weather forecasting system. These elements must be synthesized into that forecasting system with exquisite attention to detail. It is hard work, and as I heard on many visits to ECMWF: “There is no magic.”

Richard B. Rood is a Professor at the University of Michigan in the Department of Atmospheric, Oceanic and Space Sciences. He blogs about climate change for Wunderground.com and is a former member of NASA’s Senior Executive Service.

(ES)

© Copyright 2013 DTN/The Progressive Farmer. All rights reserved.