Just How Trustworthy Are Agricultural Statistics?

So you turn to the USDA Census of Agriculture ”“ as you should any time you want to know anything about farming anywhere in the United States ”“ and you pull up a great little overview on Phillips County’s profile. You see that that total sales from its 282 farms came in at just under $248 million in 2012. You see that Phillips County is the soybean capital of Arkansas, and ranks in the top 1 percent of all 2,162 soybean-producing counties in the U.S.

And you also notice that between the 2007 and 2012 censuses, Phillips County is reported to have lost nearly 85,000 acres of farmland. That’s nearly one-fifth of its 2007 total, or an area four times the size of Manhattan. Bluntly, that’s an awful lot.

Perhaps you’d conclude that urban development there has gobbled up lots of acreage. Or maybe you’d wonder whether economic pressures had so squeezed the soybean farmers of Phillips County that they’d simply abandoned a collective 85,000 acres. And you’d be wrong on both counts.

In 2007, the county’s population was estimated at just over 22,000, haven fallen steadily for more than 50 years, and by 2012 it had dipped below 21,000. That’s hardly a scenario for runaway development. And the average price of a bushel of soybeans went from $7.75 in 2007 to $13.95 in 2012 ”“ not the sort of price trend that prompts farmers to call it quits.

Robert Goodson, an extension agent in Phillips County, adds a final blow to the notion: “I would be very surprised if we had any kind of decrease in acres at all,” he told Modern Farmer.

Not all the information returned by 80 percent of American farmers is complete and accurate.

So what gives?

Statistics.

The National Agricultural Statistics Service (NASS), an agency of the USDA, uses farmer surveys and several other methods to calculate Census estimates of farmland acreage (including cropland, pasture and woodland) across the country. Not every farmer returns a census form, though, and the response rate has been trending downward. In 2012, the national response rate was at just over 80 percent, said Jeff Bailey, a NASS statistician and branch chief.

Furthermore, not all the information returned by 80 percent of American farmers is complete and accurate.

For these and other reasons, Bailey and the several hundred other statisticians employed by NASS put the numbers through a mathematical wringer in an attempt to account for blips, glitches, holes and other aberrances in its massive dataset on the country’s 2.1 million farms.

In general, the messier or more incomplete the data, the more statistical intervention required, and the more uncertain the published estimate. NASS’s statisticians also quantify this uncertainty by calculating the “standard error” associated with these farm acreage estimates. This allows the discerning Census of Agriculture user to sort the wheat from the chaff, statistically speaking.

Without getting into more intense math, (or go ahead if you must) suffice it to say that when a given farm acreage estimate changes by an amount greater than twice the standard error, it’s considered “statistically significant.” If the change is less than two times the standard error, it’s not statistically significant ”“ or, in layman’s terms, probably not as big a deal as it might appear.

Let’s go back to Phillips County, alleged loser of a whopping 85,000 acres of farmland. The standard error associated with that estimate is more than 61,000 acres (the figure is published deep down in this table). Because the change in farm acreage ”“ 85,000 acres ”“ is well below twice that standard error ”“122,000 acres ”“ it’s not statistically significant.

‘Seeing that acreage number double surprises me.’

Among other things, the large standard error figure could suggest that the soybean farmers of Phillips County slacked off on returning their census forms. Regardless, it also tells us that the Phillips County farmland crisis suggested by a cursory glance at the Census of Agriculture probably isn’t what it seems ”“ particularly when paired with the assurances of the county’s extension agent. The soybean fields of Phillips County are pretty much intact. The 85,000 acres that once were lost have now been found.

There are plenty of other examples out there. Holt County, Neb., looks to have lost 118,000 acres of farmland between 2007 and 2012. Augusta County, Va., is reported to have lost 26,000 acres over the same period. Both figures, however, are statistically insignificant, and in both counties, extension agents and other local sources say those figures defy on-the-ground logic.

Statistical vagaries can also work the other way: farm acreage in Waldo County, Maine, purportedly nearly doubled between 2007 and 2012, from a bit over 68,000 acres to about 131,000 acres. Again, this figure doesn’t meet the criteria for statistical significance, nor does it pass local muster.

“I don’t think [it] has changed all that much,” said county extension agent Rick Kersbergen. “Seeing that acreage number double surprises me.”

All told, the 2012 Census of Agriculture published farm acreage estimates for 3,130 counties (or county equivalents) across the country. Of that total, 361 county farm acreage estimates vary by at least 10 percent from the 2007 total but, on closer inspection, prove to be statistically insignificant. And in 943 counties, farm acreage estimates went up or down by at least 5 percent from 2007 but aren’t statistically significant.

The Census of Agriculture and other NASS publications offer the most comprehensive, detailed information about American farming available, but don’t forget that these figures are just estimates gleaned from an enormous and tangled set of raw data ”“ not ironclad, indisputable Truths. Before you bet the farm on your local acreage statistics, or any other agricultural tidbit from the Census, you should probably get a knowledgeable second opinion.