Skip to Main Content

Shao's Work on Map Accuracy Confirmed in New Publication

It is not uncommon for a researcher to publish the findings of his/her work after testing particular hypotheses. Having industry colleagues dedicate an entire paper to confirming one’s work is a bit more rare, but it is exactly what happened to Dr. Guofan Shao recently.

“Overselling overall map accuracy misinforms about research reliability,” a paper published by Shao in August 2019, challenged the use of the overall accuracy metric in Earth resource remote sensing. Shao, and his co-authors L. Tang and J. Liao, both based in Xiamen, China, declare that the use of overall accuracy as the single metric for map assessment is flawed and that it must be used in conjunction with other metrics.

Shao and team discussed four noteworthy problems with the metric and how overall accuracy is often misinterpreted. After analyzing literature, hypothetical examples and mathematical equations, the group concluded that overall accuracy may be unreliable and that map providers should publish the error matrix and its development procedures transparently in order to offer map users the ability to use whatever metrics they desire to determine accuracy.

In May, S.V. Stehman of the SUNY College of Environmental Sciences and Forestry, and J. Wickham of the Office of Research and Development with the U.S. Environmental Protection Agency, affirmed Shao’s findings in their own paper “A guide for evaluating and reporting map data quality.”

In beginning their own publication, Stehman and Wickham express their appreciation for Shao’s challenge of the current model.

“We appreciate and thank Shao et al. (2019) for their thought-provoking editorial reminding the readers of Landscape Ecology to consider the quality of the remotely sensed land cover and other thematic data they use in their work. Shao et al.’s (2019) caution against overselling overall map accuracy provides an opportune setting to revisit other important issues of reporting and interpreting of map accuracy results as these results pertain to use of spatial products in landscape ecology.”
Map Accuracy Graph
They further state:
“We appreciate the important and timely reminder offered by Shao et al. (2019) that landscape ecologists should take stock of map quality and that assessment of map quality should extend well beyond OA. In addition, we offer two easy-to-implement, nonquantitative questions that landscape ecologists can use to evaluate the veracity of the accuracy assessment protocols and the resulting map data quality: (1) Do the producers of the mapped data follow the established rigorous protocols (sampling design, response design, and analysis) for the collection and analysis of map accuracy data? and; (2) do the map producers report class-specific UA and PA estimates with accompanying standard errors?

"To supplement Shao et al.’s (2019) advice, a worthwhile standard of practice (sensu Olofsson et al. 2014; Stehman and Foody 2019) for landscape ecologists to adopt would be to require affirmative responses to both questions before proceeding to incorporate the map accuracy data into their analyses. The ability to provide this affirmation in publications and reports is the joint responsibility of map users and producers through proper documentation of thematic accuracy methods and results."
Code to copy:

Featured Stories

Mature open oak woodland with a diverse understory after implementing a shelterwood harvest and prescribed fire as stewardship practices.
Publication Teaches Landowners How to Support Oak-Hickory Ecosystems

Oak-hickory forests, which are comprised of a variety of different tree species, shrubs, grasses,...

Read More
A picture of the dairy judging team with their awards. Pictured from left to right are Emma Townsend, Evan Coblentz, Jackie Mudd, and Alaina Weaver.
Purdue Dairy Judging Team garners success at recent contest

The Purdue Dairy Judging Team competed in the Western National Collegiate Dairy Judging Contest....

Read More
Professor works in lab at Purdue
Purdue-led fishing expedition nets new pupfish family member in New Mexico

Scientists have identified a new member on the genetic family tree of an endangered pupfish...

Read More
pots of spruce and other native trees sit in the bed of a wooden trailer behind the Grounds Department Truck
Thousands of trees, hundreds of volunteers, five years and one giant leap for the Purdue Arboretum

The clayey Indiana soil, still saturated from the last spring shower, squishes under shovels. The...

Read More
Alex Dudley holds a black vulture; Alex is pictured through a hole in a rock formation; Alex holds her camera in front of a forested mountain landscape.
Meet FNR Outstanding Senior Alex Dudley

From her research on black vulture ecology in the Zollner lab and on digital forestry under Dr....

Read More
Yellow flowers against a leafy green background
April showers bring May flowers to Jules Janick Horticulture Garden

The sweet smell of hundreds of blossoms draws pollinators and people alike to the Jules Janick...

Read More
To Top