Evaluating the accuracy of manual classification in satellite images using supervised algorithms

Zulema Yamileth  Cervantes Hernández; Emmanuel Morales García; Cecilia  Cruz López; Itzel Alessandra Reyes Flores

Authors

Dra. Zulema Yamileth Cervantes Hernández Universidad Veracruzana https://orcid.org/0009-0000-9453-2582
Msc. Emmanuel Morales García Universidad Veracruzana https://orcid.org/0000-0002-6837-9227
Dra. Cecilia Cruz López Universidad Veracruzana https://orcid.org/0000-0002-9156-5669
Dra. Itzel Alessandra Reyes Flores Universidad Veracruzana https://orcid.org/0000-0003-0733-8453

Keywords:

Remote sensing, Machine learning, Infrared imaging, Data science

Abstract

This research evaluated manual land-cover classification using images extracted from Sentinel-2. Supervised algorithms were applied to validate and enhance this process. Three algorithms were selected based on their computational efficiency: K-Nearest Neighbors (KNN), Random Forest (RF), and Support Vector Machine (SVM).The results show that KNN achieved optimal performance, demonstrating a strong balance between accuracy, F1 score, and runtime compared to RF. RF, in turn, obtained higher precision and F1 scores, indicating its superior ability to correctly identify classes; however, it required greater computational resources. SVM exhibited lower performance in the evaluated metrics but achieved a shorter runtime. Nevertheless, it was identified as the algorithm with the greatest limitations in separating classes within this dataset derived from the different study areas.Overall, the comparison confirmed that the manual classifications developed in QGIS are supported and validated by the application of these supervised methods. The use of such algorithms contributes to improving accuracy, consistency, and efficiency in geospatial classification tasks.

Downloads

Download data is not yet available.

Author Biographies

Dra. Zulema Yamileth Cervantes Hernández, Universidad Veracruzana

Zulema Yamileth Cervantes Hernández is a student pursuing a bachelor's degree in Statistics at the Faculty of Statistics and Computer Science, Universidad Veracruzana. She is currently in her final semester of her degree. Her research interests include data science, machine learning, and spatial statistics.
Msc. Emmanuel Morales García, Universidad Veracruzana

He holds a Bachelor's degree in Statistical Sciences and Techniques and a Specialist in Statistical Methods from the University of Veracruz, with a Master's degree in Geospatial Information Sciences from the Geo Center, Mexico City (CONAHCYT Center). He is currently pursuing a PhD in Computer Science at the University of Veracruz. He is a professor in the Bachelor's program in Statistics and in the Specialization in Statistical Methods at the same university, where he has supervised 12 undergraduate theses and 4 specialized theses. He also has experience as a Statistical Analyst at the Government Program Office of the State of Veracruz. My research interests include computational methodologies, statistical programming, multivariate statistics, spatial analysis, data science, and statistical models, with applications in biology, medicine, and administrative and social sciences. He has participated in various national and international conferences.
Dra. Cecilia Cruz López , Universidad Veracruzana

She holds a PhD in Educational Research (Universidad Veracruzana), a Master of Science degree specializing in Applied Statistics (ITESM Monterrey Campus), and a Bachelor of Statistics degree (Universidad Veracruzana). She has published four books, nine book chapters, ten research articles, and one article in Extended Proceedings. She has supervised three master's theses, 24 specialized theses, and 26 undergraduate theses. Her research interests include Statistics Education and Applications of Statistical Methodology.

She collaborated with CENEVAL from 2010 to 2017, advising, evaluating, and developing items for the Extra-Es Statistics exam.
She is currently the coordinator of the Specialization in Statistical Methods.
Dra. Itzel Alessandra Reyes Flores , Universidad Veracruzana

With a solid academic background and a passion for Computer Science, I have forged a career dedicated to advancing Computer Science and improving the digital User Experience (UX). I hold a Ph.D. in Computer Science and a Master's degree in User-Centered Interactive Systems.

My work experience spans teaching in programming, web and mobile development, UX, and software engineering, as well as consulting in UX/UI design and software development. I possess knowledge in programming languages such as C#, Java, Python, PHP, and JavaScript; development frameworks including .NET, React, and Node.js; web design technologies such as HTML, CSS, and Wordpress; web service development and usage with REST and SOAP APIs; mobile development with Android Studio; design tools like Photoshop, Figma, Adobe XD, and Corel Draw; database management with MySQL, SQLite, MongoDB, and SQL Server; development methodologies like SCRUM and Design Thinking; and the use of user-centered interface design guidelines.

References

[1] Ouchra, H., Belangour, A., & Erraissi, A. (2023). Machine learning algorithms for satellite image classification using Google Earth Engine and Landsat satellite data: Morocco case study. IEEE Access, 11, 71127–71142. https://doi.org/10.1109/ACCESS.2023.3293828.

[2] Wang, Y., Sun, Y., Cao, X., Wang, Y., Zhang, W., and Cheng, X. (2023). A review of regional and Global scale Land Use/Land Cover (LULC) mapping products generated from satellite remote sensing. ISPRS Journal of Photogrammetry and Remote Sensing, 206, 311-334.

[3] Joshi, P., Saxena, P., & Sharma, A. (2024). Trends and Advancements in Satellite Image Classification: From Traditional Methods to Machine Learning Approaches. ResearchGate. 10.13140/RG.2.2.13239.23201.

[4] European Space Agency. (n.d.). Introducing Sentinel-2. ESA. https://www.esa.int/Applications/Observing the Earth/Copernicus/Sentinel- 2/Introducing Sentinel-2.

[5] Phiri, D., Simwanda, M., Salekin, S., Nyirenda, V. R., Murayama, Y., and Ranagalage, M. (2020). Sentinel-2 data for land cover/use mapping: A review. Remote sensing, 12(14), 2291.

[6] Segarra, J., Buchaillot, M. L., Araus, J. L., & Kefauver, S. C. (2020). Remote sensing for precision agriculture: Sentinel-2 improved features and applications. Agronomy, 10(5), 641.

[7] Drusch, M., Del Bello, U., Carlier, S., Colin, O., Fernandez, V., Gascon, F., ... & Bargellini, P. (2012). Sentinel-2: ESA’s optical high- resolution mission for GMES operational services. Remote sensing of Environment, 120, 25-36.

[8] Fragoso-Campo´n, L., QUIRO´ S, R. E., & GUTIE´RREZ, G. J. (2020).

Clasificacio´n supervisada de ima´genes PNOA-NIR y fusio´n con datos LiDAR-PNOA como apoyo en el inventario forestal. Caso de estu- dio: Dehesas. CUADERNOS DE LA SOCIEDAD ESPAN˜ OLA DE

CIENCIAS FORESTALES: Sociedad Espanola de Ciencias Forestales, 45(3), 77-96.

[9] Zhang, C., Liu, Y., & Tie, N. (2023). Forest land resource information acquisition with sentinel-2 image utilizing support vector machine, k- nearest neighbor, random forest, decision trees and multi-layer percep- tron. Forests, 14(2), 254.

[10] Yuh, Y. G., Tracz, W., Matthews, H. D., & Turner, S. E. (2023). Application of machine learning approaches for land cover monitoring in northern Cameroon. Ecological informatics, 74, 101955.

[11] Thanh Noi, P., & Kappas, M. (2017). Comparison of random forest, k- nearest neighbor, and support vector machine classifiers for land cover classification using Sentinel-2 imagery. Sensors, 18(1), 18.

[12] Knudby, A. (2021). Accuracy assessment. Pressbooks. https://ecampusontario.pressbooks.pub/remotesensing/chapter/chapter- 7-accuracy-assessment/

[13] Tehsin, S., Kausar, S., Jameel, A., Humayun, M., & Almofarreh, D.

K. (2023). Satellite image categorization using scalable deep learning. Applied Sciences, 13(8), 5108. https://doi.org/10.3390/app13085108.

[14] Belgiu, M., & Dragut, L. (2016). Random forest in remote sensing: A review of applications and future directions. ISPRS Journal of Photogrammetry and Remote Sensing, 114, 24–31.

[15] Maxwell, A. E., Warner, T. A. & Fang, F. (2018). ”Implementation of machine-learning classification in remote sensing: An applied review.” International Journal of Remote Sensing, 39(9), 2784-2817.

[16] Balcik, F. B., Senel, G., & Goksel, C. (2020). Object-Based Classifi- cation of Greenhouses Using Sentinel-2 MSI and SPOT-7 Images: A Case Study from Anamur (Mersin), Turkey. IEEE Journal Of Selected Topics In Applied Earth Observations And Remote Sensing, 13, 2769- 2777. https://doi.org/10.1109/jstars.2020.2996315

[17] Rahman, A., Abdullah, H. M., Tanzir, M. T., Hossain, M. J., Khan, B. M., Miah, M. G., & Islam, I. (2020). Performance of different machine learning algorithms on satellite image classification in rural and urban setup. Remote Sensing Applications Society And Environment, 20, 100410. https://doi.org/10.1016/j.rsase.2020.100410

[18] Feizizadeh, B., Omarzadeh, D., Garajeh, M. K., Lakes, T., & Blaschke, T. (2021). Machine learning data-driven approaches for land use/cover mapping and trend analysis using Google Earth Engine. Journal Of Environmental Planning And Management, 66(3), 665-697. https://doi.org/10.1080/09640568.2021.2001317

[19] Santiago, R. M., Gustilo, R., Arada, G., Magsino, E., & Sybingco,

E. (2021). Performance Analysis of Machine Learning Algorithms in Generating Urban Land Cover Map of Quezon City, Philippines Using Sentinel-2 Satellite Imagery. 2021 IEEE 13th International Conference on Humanoid, Nanotechnology, Information Technology, Communi- cation and Control, Environment, and Management (HNICEM), 1–6. https://doi.org/10.1109/hnicem54116.2021.9731856

[20] Torres, C., Jéssica Gerente, Rodrigo, Caruso, F., Providelo, L. A., Marchiori, G., & Chen, X. (2022). Canopy Height Map- ping by Sentinel 1 and 2 Satellite Images, Airborne LiDAR data and machine learning. Remote Sensing, 14(16), 4112-4112. https://doi.org/10.3390/rs14164112.

[21] Kumar, A., Kumar, G., Patil, D. S., &Gupta, R. (2024). Evaluating Machine Learning Classifiers for IRS High Resolution Satellite Images Using Object-Based and Pixel-Based Classification Techniques. Journal Of The Indian Society Of Remote Sensing. https://doi.org/10.1007/s12524-024-02084-w.

[22] Purwanto, A. D., Wikantika, K., Deliar, A., & Darmawan, S. (2022). Decision Tree and Random Forest Classification Algorithms for Mangrove Forest Mapping in Sembilang National Park, Indonesia. Remote Sensing, 15(1), 16. https://doi.org/10.3390/rs15010016.

[23] Breiman, L. (2001). Random Forests. Machine Learning, 45(1), 5–32. https://doi.org/10.1023/a:1010933404324.

[24] Farhadi, Z., Bevrani, H., Feizi-Derakhshi, M., Kim, W., & Ijaz, M. F. (2022). An Ensemble Framework to Improve the Accuracy of Prediction Using Clustered Random-Forest and Shrinkage Methods. Applied Sciences, 12(20), 10608. https://doi.org/10.3390/app122010608.

[25] Tamamadin, M., Lee, C., Kee, S., & Yee, J. (2022). Regional Typhoon Track Prediction Using Ensemble k-Nearest Neighbor Machine Learning in the GIS Environment. Remote Sensing, 14(21), 5292. https://doi.org/10.3390/rs14215292.

[26] Adugna, T., Xu, W., & Fan, J. (2022b). Comparison of Random Forest and Support Vector Machine Classifiers for Regional Land Cover Mapping Using Coarse Resolution FY-3C Images. Remote Sensing, 14(3), 574. https://doi.org/10.3390/rs14030574.

[27] Du, K., Jiang, B., Lu, J., Hua, J., & Swamy, M. N. S. (2024b). Exploring Kernel Machines and Support Vector Machines: Principles, Techniques, and Future Directions. Mathematics, 12(24), 3935. https://doi.org/10.3390/math12243935.

[28] S. Visa, B. Ramsay, A. Ralescu, and E. Van Der Knaap, “Confusion matrix-based feature selection,” CEUR Workshop Proc., vol. 710, pp. 120-127, 2011.

[29] Pikabea, I. (2022). Review of learning models in the field of question answering [Undergraduate thesis, University of the Basque Country, Faculty of Informatics]. University of the Basque Country.

[30] Nahm, F. (2022). Receiver Operating Characteristic Curve: Overview and Practical Use for Clinicians [Epub 2022 Jan 18]. Korean Journal of Anesthesiology, 75 (1), 25-36. https://doi.org/10.4097/kja.21209