Study: NASS Needs More Data Points

WASHINGTON (DTN) -- The National Academy of Sciences wants USDA's lead statistical agency to develop new systems for gathering information for crop reports and other surveys.

In a report released Wednesday, the National Academy highlighted that the National Agricultural Statistics Service could make several changes over time to improve the accuracy of its reports, as well as develop guidelines that make the data collection more transparent and reproducible.

NASS entered into an agreement with the National Academy of Sciences three years ago, specifically to have the academy study how NASS does county-level crop estimates as well as cash-rent estimates. While NASS data is used by private companies as well as the Farm Service Agency and Risk Management Agency, NASS recognized it was facing a problem that is also hitting a lot of federal agencies that rely on surveys.

"Virtually all statistical agencies are seeing declines in survey response rates, and NASS is no exception," the academy study stated. "As a result, the number of counties for which NASS has been able to publish estimates for crop acreage, yield and cash rents has been declining over time."

The academy panel made its recommendations "in terms of a vision for NASS in 2025." That vision would focus on changing the Agricultural Statistics Board to models that incorporate multiple data sources. The Agricultural Statistics Board should evolve into reviewing model-based predictions, macro-editing and ensuring the models are continually reviewed, assessed and studied, the academy recommended.

Currently, the National Academy of Sciences committee noted, NASS uses multiple data sources through the Agricultural Statistics Board. However, "While the current process follows specific steps and guidelines, it is inherently subjective and neither transparent nor reproducible." The work of the Agricultural Statistics Board could be improved if the board started using model-based estimates that "synthesize" or marry up with multiple other data sources.

Linda Young, chief mathematical statistician and director of research and development at NASS, said the study was a confirmation of the direction NASS has been moving in its own research and development. The academy study didn't necessarily indicate NASS needs to change the direction it is already moving, Young said.

"We feel like there is a real opportunity for us to improve our estimates and continue to evolve as an agency and stay relevant to the ag industry and to Congress," she said. "That's our role."

The Academy report highlights how USDA data is used for a broad array of uses in the marketplace and how data collection has evolved over time based on congressional mandates in the farm bill.

Over the last decade, NASS has developed more county-level data for farm programs by introducing more large-scale surveys to provide the information. The only problem, as the National Academy noted, is "farmers have raised questions with program agencies about the use of estimates derived from these surveys because they feel that the county average does not represent their own farm experience."

The declining response to NASS surveys is happening in several areas. For the cash-rents survey, the response rate nationally fell from 75.9% in 2014 to 69.1% in 2016. Because the survey standard requires at least 30 surveys per county, or responses from at least 25% of the acreage by crop, USDA reported cash rents on 2,597 counties in 2016, or 282 fewer counties than were reported on the 2014 survey.

For the county acreage survey, the response rate for larger row crops has fallen from 62.8% in 2011 to 56.4% in 2016. The decline of responses in a county increases the likelihood that no estimate can be published for that county.

Regarding surveys, Young noted all surveys, governmental and non-governmental, are suffering from declining response rates. NASS has a response-rate review team to look for ways to make it easier to respond. Ideally, NASS eventually will have a "dashboard" for farmers to more easily answer surveys in the future, she said.

"We're going to be heading in that direction," she said. "It's just a question of how fast we can get there."

Market analysts have grown increasingly critical of NASS reports in recent years. DTN Market Analyst Todd Hultman said of the academy report that there seems to be an assumption that NASS estimates were better when survey response rates were higher, but there is no empirical evidence offered with that assumption. Further, NASS could save time and effort by stopping the practice of making field-based yield estimates in early August for fall-harvest crops when it is far too early to obtain reliable results.

"In the ideal world, if all NASS estimates were accurate, we could say that these estimates help market participants anticipate cost and assess the availability of food, but the reality is that NASS estimates are not always accurate and there is no mention in this report of the harm that has come from bad estimates," Hultman said. "It would be interesting to see a poll of market participants rate their view of the reliability of NASS estimates. I suspect NASS estimates are not as important in decision-making as the academy study describes."

Hultman added, "It would be far wiser for USDA to keep its focus on providing actual data to the market and take a more humble approach in its efforts to predict the variables that are not yet known, and in some cases, cannot be known. More is not always better."

The Farm Service Agency demonstrated to the academy panel how relatively small variations in county yield estimates can result in relatively large changes in the ARC-County program payments and possibly no payments being triggered at all.

The academy made several recommendations for ways NASS could obtain better county-level data, such as using geospatial field data and precision-ag measurements in databases to help complete the geospatially referenced farm-level database. The academy also said NASS should collaborate with the RMA to obtain acreage and production data and conduct comparisons with NASS data for the same entity.

The recommendations are notable because farm groups are demanding Congress switch from using NASS county yield data for the Agricultural Risk Coverage (ARC-County) program to instead using crop insurance information reported to RMA. The groups argue crop-insurance reported information will provide a more complete set of information for calculating county yields.

The study also stated NASS should work with farmer cooperatives to ensure that NASS is one of the government agencies farmers can use to share relevant precision-ag data. NASS should also develop a system for reporting precision-ag information for the County Agricultural Production Survey. Farmers who reported such information would then either not receive an additional survey form or receive one that is more simplified to fill out.

The report acknowledges that, given its limited resources, it may take NASS many years to develop a geo-referenced list frame, appropriate models, and other components needed for this vision of improved county-level estimates. The report suggests a two-part plan of action that could be completed by 2025, as well as breaking down each stage into individual projects to be executed by different groups within NASS, each over the course of three years.

For more information, visit http://national-academies.org/…

Chris Clayton can be reached at Chris.Clayton@dtn.com

Follow him on Twitter @ChrisClaytonDTN

(AG/BAS)

Copyright 2017 DTN/The Progressive Farmer. All rights reserved.