Assessing Accuracy Methods of Species Distribution Models: AUC, Specificity, Sensitivity and the True Skill Statistic

Assessing Accuracy Methods of Species Distribution Models: AUC, Specificity, Sensitivity and the True Skill Statistic

Farzin Shabani

Contact

Lalit Kumar

Contact

Mohsen Ahmadi

Contact

Assessing Accuracy Methods of Species Distribution Models: AUC, Specificity, Sensitivity and the True Skill Statistic

Article Fingerprint

ReserarchID

JHN1L

Assessing Accuracy Methods of Species Distribution Models: AUC, Specificity, Sensitivity and the True Skill Statistic Banner

AI TAKEAWAY

Connecting with the Eternal Ground

Abstract

We aimed to assess different methods for evaluating performance accuracy in species distribution models based on the application of five types of bioclimatic models under three threshold selections to predict the distributions of eight different species in Australia, treated as an independent area. Five discriminatory correlative species distribution models (SDMs), were used to predict the species distributions of eight different plants. A global training data set, excluding the Australian locations, was used for model fitting. Four accuracy measurement methods were compared under three threshold selections of i) maximum sensitivity + specificity, ii) sensitivity = specificity and iii) predicted probability of 0.5 (default). Results showed that the choice of modeling methods had an impact on potential distribution predictions for an independent area. Examination of the four accuracy methods underexamined threshold selections demonstrated that TSS is a more realistic and practical method, in comparison with AUC, Sensitivity and Specificity. Accurate projection of the distribution of a species is extremely complex.

References

81 Cites in Article

Reference Format

Mohsen Ahmadi,Mohammad Kaboli,Elham Nourani,Afshin Alizadeh Shabani,Sohrab Ashrafi (2013). A predictive spatial model for gray wolf ( <i>Canis lupus</i> ) denning sites in a human‐dominated landscape in western Iran.
Omri Allouche,Asaf Tsoar,Ronen Kadmon (2006). Assessing the accuracy of species distribution models: prevalence, kappa and the true skill statistic (TSS).
R Anderson,M Dudík,S Ferrier,A Guisan,R Hijmans,F Huettmann,J Leathwick,A Lehmann,J Li,L Lohmann (2006). Novel methods improve prediction of species' distributions from occurrence data.
Miguel Araújo,Antoine Guisan (2006). Five (or so) challenges for species distribution modelling.
Miguel Araújo,Robert Whittaker,Richard Ladle,Markus Erhard (2005). Reducing uncertainty in projections of extinction risk from climate change.
Arlene Ash,Michael Shwartz (1999). R2: a useful measure of model performance when predicting a dichotomous outcome.
(2017). Atlas of Living Australia.
M Austin (2002). Spatial prediction of species distribution: an interface between ecological theory and statistical modeling.
Mike Austin (2007). Species distribution models and ecological theory: A critical assessment and some possible new approaches.
S Barry,J Elith (2006). Error and uncertainty in habitat models.
William Bean,Robert Stafford,Justin Brashares (2012). The effects of small sample size and sample bias on threshold selection and accuracy assessment of species distribution models.
Åke Berg,Ulf Gärdenfors,Ted Von Proschwitz (2004). Logistic regression models for predicting occurrence of terrestrial molluscs in southern Sweden – importance of environmental data quality and model complexity.
H Beyer (2004). Getting Started with ArcGIS: Data Management and Basic Spatial Analysis Tools.
Leo Breiman (2001). Random Forests.
L Breiman (2001). Statistical modeling: The two cultures (with comments and a rejoinder by the author).
Lluís Brotons,Wilfried Thuiller,Miguel Araújo,Alexandre Hirzel (2004). Presence‐absence versus presence‐only modelling methods for predicting bird habitat suitability.
Laëtitia Buisson,Wilfried Thuiller,Nicolas Casajus,Sovan Lek,Gaël Grenouillet (2010). Uncertainty in ensemble forecasting of species distribution.
J Busby (1991). BIOCLIM-a bioclimate analysis and prediction system.
S Cantor,C Sun,G Tortolero-Luna,R Richards-Kortum,M Follen (1999). A comparison of C/B ratios from studies using receiver operating characteristic curve analysis.
G Carpenter,A Gillison,J Winter (1993). DOMAIN: a flexible modelling procedure for mapping potential distributions of plants and animals.
Rich Caruana,Alexandru Niculescu-Mizil (2006). An empirical comparison of supervised learning algorithms.
Manuela D'amen,Anne Dubuis,Rui Fernandes,Julien Pottier,Loïc Pellissier,Antoine Guisan (2015). Using species richness and functional traits predictions to constrain assemblage predictions from stacked species distribution models.
C Dormann,O Purschke,J Márquez,S Lautenbach,B Schröder (2008). Components of uncertainty in species distribution analysis: a case study of the great grey shrike.
Jane Elith,Mark Burgman,Helen Regan (2002). Mapping epistemic uncertainties and vague concepts in predictions of species distribution.
Jane Elith,Michael Kearney,Steven Phillips (2010). The art of modelling range-shifting species.
J Elith,J Leathwick,T Hastie (2008). A working guide to boosted regression trees.
Jane Elith,Steven Phillips,Trevor Hastie,Miroslav Dudík,Yung Chee,Colin Yates (2011). A statistical explanation of MaxEnt for ecologists.
Oren Farber,Ronen Kadmon (2003). Assessment of alternative approaches for bioclimatic modeling with special emphasis on the Mahalanobis distance.
Alan Fielding,John Bell (1997). A review of methods for the assessment of prediction errors in conservation presence/absence models.
A Fielding,J Bell (1997). A review of methods for the assessment of prediction errors in conservation presence/absence models.
G Foody (2008). GIS: biodiversity applications.
J Franklin (2010). Mapping species distributions: spatial inference and prediction.
(2015). Supplemental Information 1: Geographic occurrence records of thecostracan barnacles available in the Global Biodiversity Information Facility (GBIF) database (https://www.gbif.org/)..
C Graham,S Ferrier,F Huettman,C Moritz,A Peterson (2004). New developments in museum-based informatics and applications in biodiversity analysis.
Gurutzeta Guillera‐arroita,José Lahoz‐monfort,Jane Elith (2014). Maxent is not a presence–absence method: a comment on Thibaud <i>et al</i>..
A Guisan,C Rahbek (2011). SESAM-a new framework integrating macroecological and species distribution models for predicting spatio-temporal patterns of species assemblages.
A Guisan,J Theurillat,F Kienast (1998). Predicting the potential distribution of plant species in an alpine environment.
Antoine Guisan,Wilfried Thuiller (2005). Predicting species distribution: offering more than simple habitat models.
J Hanley,B Mcneil (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve..
Jan Hanspach,Ingolf Kühn,Oliver Schweiger,Sven Pompe,Stefan Klotz (2011). Geographical patterns in prediction errors of species distribution models.
F Harrell (2001). Regression modeling strategies: with applications to linear models, logistic regression and survival analysis.
T Hastie,R Tibshirani (1990). Generalized additive models.
Robert Hijmans,Steven Phillips,John Leathwick,Jane Elith (2015). dismo: Species Distribution Modeling.
J Hortal,J Lobo,Jiménez-Valverde (2007). Limitations of Biodiversity Databases: Case Study on Seed-Plant Diversity in Tenerife, Canary Islands.
Jonathan Jeschke,David Strayer (2008). Usefulness of Bioclimatic Models for Studying Climate Change and Invasive Species.
A Jiménez-Valverde,J Lobo,J Hortal (2008). Not as good as they seem: the importance of concepts in species distribution modeling.
Ronen Kadmon,Oren Farber,Avinoam Danin (2003). A SYSTEMATIC ANALYSIS OF FACTORS AFFECTING THE PERFORMANCE OF CLIMATIC ENVELOPE MODELS.
A Latimer,S Wu,A Gelfand,J Silander (2006). Building statistical models to analyze species distributions.
A Liaw,M Wiener (2002). Classification and regression by randomForest.
Canran Liu,Pam Berry,Terence Dawson,Richard Pearson (2005). Selecting thresholds of occurrence in the prediction of species distributions.
Jorge Lobo,Alberto Jiménez‐valverde,Raimundo Real (2008). AUC: a misleading measure of the performance of predictive distribution models.
Bette Loiselle,Christine Howell,Catherine Graham,Jaqueline Goerck,Thomas Brooks,Kimberly Smith,Paul Williams (2003). Avoiding Pitfalls of Using Species Distribution Models in Conservation Planning.
Stéphanie Manel,H Williams,S Ormerod (2001). Evaluating presence–absence models in ecology: the need to account for prevalence.
J Mcpherson,W Jetz,D Rogers (2004). The effects of species' range sizes on the accuracy of distribution models: ecological phenomenon or statistical artefact.
Cory Merow,Mathew Smith,Thomas Edwards,Antoine Guisan,Sean Mcmahon,Signe Normand,Wilfried Thuiller,Rafael Wüest,Niklaus Zimmermann,Jane Elith (2014). What do we gain from simplicity versus complexity in species distribution models?.
H Nix (1986). A biogeographic analysis of Australian elapid snakes.
Jennie Pearce,Simon Ferrier (2000). Evaluating the predictive performance of habitat models developed using logistic regression.
Richard Pearson,Terence Dawson (2003). Predicting the impacts of climate change on the distribution of species: are bioclimate envelope models useful?.
R Pearson,T Dawson,P Berry,P Harrison (2002). SPECIES: A Spatial Evaluation of Climate Impact on the Envelope of Species.
A Peterson,Victor Sánchez-Cordero,C Ben Beard,Janine Ramsey (2002). Ecologic Niche Modeling and Potential Reservoirs for Chagas Disease, Mexico.
Steven Phillips,Robert Anderson,Robert Schapire (2006). Maximum entropy modeling of species geographic distributions.
S Phillips,M Dudík (2008). Modeling of species distributions with Maxent: new extensions and a comprehensive evaluation.
Steven Phillips,Miroslav Dudík,Jane Elith,Catherine Graham,Anthony Lehmann,John Leathwick,Simon Ferrier (2009). Sample selection bias and presence‐only distribution models: implications for background and pseudo‐absence data.
Anantha Prasad,Louis Iverson,Andy Liaw (2006). Newer Classification and Regression Tree Techniques: Bagging and Random Forests for Ecological Prediction.
H Pulliam (2000). On the relationship between niche and distribution.
(2016). R Development Core Team. R: A Language and Environment for Statistical Computing R Foundation for Statistical Computing.
Greg Ridgeway,Gbm Developers (2006). gbm: Generalized Boosted Regression Models.
Carlo Rondinini,Kerrie Wilson,Luigi Boitani,Hedley Grantham,Hugh Possingham (2006). Tradeoffs of different types of species occurrence data for use in systematic conservation planning.
Víctor Sánchez-Cordero,Verónica Cirelli,Mariana Munguial,Sahotra Sarkar (2005). Place prioritization for biodiversity content using species ecological niche modeling.
Marco Schmidt,Holger Kreft,Adjima Thiombiano,Georg Zizka (2005). Herbarium collections and field data‐based plant diversity maps for Burkina Faso.
Pedro Segurado,Miguel Araújo (2004). An evaluation of methods for modelling species distributions.
Flemming Skov,Jens‐christian Svenning (2004). Potential impact of climatic change on the distribution of forest herbs in Europe.
Imelda Somodi,Nikolett Lepesi,Zoltán Botta‐dukát (2017). Prevalence dependence in model goodness measures with special emphasis on true skill statistics.
D Stockwell (1999). The GARP modeling system: problems and solutions to automated spatial prediction.
Vladimir Svetnik,Andy Liaw,Christopher Tong,J Culberson,Robert Sheridan,Bradley Feuston (2003). Random Forest: A Classification and Regression Tool for Compound Classification and QSAR Modeling.
Wilfried Thuiller,Sandra Lavorel,Miguel Araújo (2005). Niche properties and geographical extent as predictors of species sensitivity to climate change.
Nobuyuki Tsuji,Yoshitaka Tsubaki (2004). Three new algorithms to calculate the irreplaceability index for presence/absence data.
I Vaughan,S Ormerod (2005). The continuing challenges of testing species distribution models.
D Warren,R Glor,M Turelli (2010). ENMTools: a toolbox for comparative studies of environmental niche models.
J Watling,S Romanach,D Bucklin,C Speroterra,L Brandt,L Pearlstine,F Mazzotti (2012). Do bioclimate variables improve performance of climate envelope models?.
F Woodward (1987). Climate and plant distribution.

Download References

Funding

No external funding was declared for this work.

Conflict of Interest

The authors declare no conflict of interest.

Ethical Approval

No ethics committee approval was required for this article type.

Data Availability

Not applicable for this article.

How to Cite This Article

Farzin Shabani. 2018. \u201cAssessing Accuracy Methods of Species Distribution Models: AUC, Specificity, Sensitivity and the True Skill Statistic\u201d. Global Journal of Human-Social Science - B: Geography, Environmental Science & Disaster Management GJHSS-B Volume 18 (GJHSS Volume 18 Issue B1): .

More Citation Formats

Select Citation Style:

Download Citation

Download Article

GJHSS Volume 18 Issue B1
Pg. 7- 18

Explore Journals Explore Volume Read This Issue

Journal Specifications

Crossref Journal DOI 10.17406/GJHSS

Print ISSN 0975-587X

e-ISSN 2249-460X

Keywords

Not Found

Classification

GJHSS-B Classification: FOR Code: 040699

Submission ReceivedDecember 9, 2017
Peer Review Double Blind
Handling Editor
Accepted December 31, 2017
Published January 15, 2018

Version of record

v1.2

Issue date

March 27, 2018

Language

Experiance in AR

Explore published articles in an immersive Augmented Reality environment. Our platform converts research papers into interactive 3D books, allowing readers to view and interact with content using AR and VR compatible devices.

View in VR

Read in 3D

Your published article is automatically converted into a realistic 3D book. Flip through pages and read research papers in a more engaging and interactive format.

View in 3D

Article Matrices

Total Score: 133

Country: Australia

Subject: Global Journal of Human-Social Science - B: Geography, Environmental Science & Disaster Management

Authors: Farzin Shabani, Lalit Kumar, Mohsen Ahmadi (PhD/Dr. count: 0)

View Count (all-time): 164

Total Views (Real + Logic): 3391

Total Downloads (simulated): 1586

Publish Date: 2018 03, Tue

Monthly Totals (Real + Logic):

Month 1: 36 views
Month 2: 47 views
Month 3: 59 views
Month 4: 28 views
Month 5: 37 views
Month 6: 31 views
Month 7: 29 views
Month 8: 34 views
Month 9: 36 views
Month 10: 50 views
Month 11: 42 views
Month 12: 22 views
Month 13: 31 views
Month 14: 35 views
Month 15: 33 views
Month 16: 20 views
Month 17: 20 views
Month 18: 29 views
Month 19: 29 views
Month 20: 48 views
Month 21: 49 views
Month 22: 25 views
Month 23: 28 views
Month 24: 34 views
Month 25: 26 views
Month 26: 31 views
Month 27: 34 views
Month 28: 34 views
Month 29: 16 views
Month 30: 42 views
Month 31: 44 views
Month 32: 32 views
Month 33: 28 views
Month 34: 28 views
Month 35: 38 views
Month 36: 41 views
Month 37: 16 views
Month 38: 37 views
Month 39: 28 views
Month 40: 35 views
Month 41: 29 views
Month 42: 21 views
Month 43: 43 views
Month 44: 37 views
Month 45: 34 views
Month 46: 35 views
Month 47: 40 views
Month 48: 38 views
Month 49: 38 views
Month 50: 38 views
Month 51: 42 views
Month 52: 20 views
Month 53: 30 views
Month 54: 21 views
Month 55: 23 views
Month 56: 36 views
Month 57: 33 views
Month 58: 28 views
Month 59: 36 views
Month 60: 22 views
Month 61: 45 views
Month 62: 44 views
Month 63: 44 views
Month 64: 13 views
Month 65: 20 views
Month 66: 28 views
Month 67: 20 views
Month 68: 15 views
Month 69: 19 views
Month 70: 12 views
Month 71: 24 views
Month 72: 36 views
Month 73: 23 views
Month 74: 33 views
Month 75: 29 views
Month 76: 22 views
Month 77: 48 views
Month 78: 42 views
Month 79: 25 views
Month 80: 44 views
Month 81: 31 views
Month 82: 28 views
Month 83: 38 views
Month 84: 33 views
Month 85: 33 views
Month 86: 18 views
Month 87: 43 views
Month 88: 29 views
Month 89: 36 views
Month 90: 55 views
Month 91: 29 views
Month 92: 65 views
Month 93: 59 views
Month 94: 53 views
Month 95: 73 views
Month 96: 29 views
Month 97: 50 views

Total Views: 3391

Total Downloads: 1586

2026 Trends

Published Article

Our website is actively being updated, and changes may occur frequently. Please clear your browser cache if needed. For feedback or error reporting, please email [email protected]