The objective from this article evaluates the thematic quality of automatic mappings with supervised classification for land use and land cover, using high spatial resolution satellite images as "ground truth". This consider the advancement of remote sensing technologies has enabled the acquisition of satellite images with various spatial resolutions, which are essential for thematic mapping and automatic classifiers in the context of land use and land cover mapping. In fact, the wide availability of high, medium, and low spatial resolution satellite images has significantly optimized the time and resources required by using more accurate classifiers during data processing. The image used for verification of this paper was GeoEye, with a spatial resolution of 0.5m, dated October 2023. The images submitted to the automatic classifier were Sentinel-2A with a spatial resolution of 10m and Planet with a spatial resolution of 5m, both from the same satellite pass period (October 2023) over the study area, aiming to avoid seasonal and phenological variations in vegetation, as well as changes in the environment due to anthropogenic intervention. The classification method adopted was Maximum Likelihood (MAXVER). The classification accuracy was rigorously evaluated to ensure the reliability of the results using the Kappa index, assessing the agreement between the observed and expected classifications. Based on the methods presented, the set of mapped classes in this study showed good accuracy for the Planet image and very good accuracy for the Sentinel image.

Keywords: Kappa index, thematic quality, supervised classification, land use, land cover

INPE, Instituto Nacional de Pesquisas Espaciais; IR, infra read; MAXVER, maximum likelihood classification; RGB, Read/Green/Blue; RPA, remotely piloted aircraft; RTK, real-time kinematic

Nowadays, there is an ongoing effort to reduce costs and produce more efficient land use and land cover maps, which can be used as geoprocessing products for administrative management and decision-making. The accuracy of a classification impacts the quality of the results resulting from the classification.¹ In fact, it is important to highlight recent works, such as² (who simulated deforestation based on the influence of increased and decreased protected areas, with an estimation of CO₂ emissions, using open-source software and geoprocessing techniques with Geographic Information Systems), Inguaggiato and³ (using geoprocessing for spatial study and analysis applied to transportation and health, specifically regarding the spread of COVID-19, considering public transportation in Araraquara-SP and the disease's dissemination),⁴ (joint use of geoprocessing and artificial intelligence for planning the installation of biogas plants in the sugarcane agroindustry),⁵ (applying geoprocessing techniques to the temporal analysis of the microclimate in the Pequiá Stream Watershed, State of Maranhão),⁶ (applying geoprocessing with geostatistics considering water contamination in rural areas in Northeast Brazil), among others.

In this context, considering the objectives of this research, freely available satellite images, such as Sentinel-2A images with a spatial resolution of 10 m and Planet images with a spatial resolution of 5 m, become attractive and economically viable products. Thus, in terms of the state of the art, it is worth highlighting some recent research that supports and motivates this work, such as:

1. ⁷Quantified peat exploitation and carbon emissions in high-extraction areas in Ireland using Sentinel-2 satellite images and Google Earth Engine;
2. ⁸Conducted research employing land use mapping and snow detection in the Himalayan region using machine learning techniques and Sentinel-2 multispectral satellite images;
3. ⁹Mapped the expansion of aquaculture in the coastal region of China using Sentinel-2 images from 2017 to 2021; 4) ¹⁰Mapped integrated crop-livestock systems using fused time-series images from Sentinel-2 and PlanetScope and applying deep learning;
4. ¹¹Have studied the influence of the DTM resolution applied in the result of the hydrologic modeling by using the Open Source Software as HEC-RAS and the authors have done the correlation with the more recent Brazilian normative;
5. ¹²Have published an article about the evaluation of environmental damage in clandestine mining when the authors have worked with the LIDAR (Light Detection and Ranging) altimetry and it was possible to calculate the volume of excavation carried out in 2009, as well as the comparison with the data obtained from RPAs (Remotely piloted aircraft) allowed an accurate estimate of irregularly mined mining.

Considering the state of the art and the demands reported here, this research evaluates the thematic quality of mappings with supervised classification through the Maximum Likelihood Classification (MAXVER) method for land use, using higher resolution satellite images as the "ground truth." In this regard, this article presents the materials and methods, including details of the procedures, the results obtained, and, finally, the conclusions.

Characterization of the study area

For this study, an area of 5,716.88 hectares was selected, located in Itabirito, 70 km from the Metropolitan Region of Belo Horizonte, State of Minas Gerais, Southeast Brazil, as shown in Figure 1. The municipality of Itabirito covers an area of approximately 540 km² and is positioned in the Iron Quadrangle, a region rich in mineral resources, especially iron ore. This region has been recently addressed and studied by several authors, such as,^13–18 for example.

Figure 1 Location of the study area.
Source: by the authors

This study area also has a geological and geomorphological structure known as the Bação Metamorphic Complex or Bação Complex. These physical-geographic characteristics present a diversity of land use and cover typologies, including mining activity, agriculture, erosive processes, and different natural formations.

To better understand the methodology, the process flowchart is presented in Figure 2.

Figure 2 Flowchart of the methodological process.
Source: by the authors

Collection and processing of satellite images

The satellite images used in the supervised classification were Sentinel-2A with a spatial resolution of 10 meters and Planet with a spatial resolution of 5 meters, both with R, G, B, and IR bands and from the same period: October 2023.

Definition of training samples for land use classes

After selecting and preprocessing the image, the next step in a supervised classification is training. This stage consists of selecting and recognizing the samples (spectral signatures) of the land use classes to be mapped. The mapped classes were: Clean Field, Forest Fragment, Mining, Pasture, and Exposed Soil.

In the sample selection phase, scatter plots were created for each typology in its respective image (Figure 3) (Figure 4). The scatter plot made it possible to evaluate the quality of the samples. Its representation allows comparison of multiple samples of a class and the distribution of the set of samples by class. The greater the differentiation of training samples by class, the less overlap of points in the plots and the greater their dispersion.

Figure 3 Scatter plots of samples by class in the planet image.
Source: by the authors

Figure 4 Scatter plots of samples by class in the Sentinel-2A image.
Source: by the authors

In the graphical representations, it is possible to observe that the samples from the Exposed Soil class showed considerable diversity in the scatter plot, driven by the Red band, whose spectral difference can be observed in its intersection with the Green and Blue bands, respectively in their first two graphs, from left to right. This diversification also occurs in the "Mining" class, though in smaller proportions.

It is possible that, beyond visual interpretation, the same land use class, "Exposed Soil," presents multiple samples of different spectral ranges due to the different compositions of these soils, whether resulting from leveling, erosion by gullies, or just vegetation removal. Added to this are the aspects of a mining area that contains all these characteristics together, resembling this class.

Supervised classification of images

The classification method adopted was Maximum Likelihood Classification (MAXVER), a pixel-by-pixel multispectral classifier that considers the weighting of distances between the means of the digital levels of the classes using statistical parameters.¹⁹ The land use maps and their respective area quantities, products generated by the supervised classifications, can be seen in Figure 5 and Figure 6 below:

Figure 5 Planet image classified by the MAXVER method.
Source: by the authors

Figure 6 Sentinel-2A image classified by the MAXVER method.
Source: by the authors

When comparing the classifications, the sample results indicate significant similarity in the area quantities between the Forest Fragment and Clean Field classes. However, the Mining, Pasture, and Exposed Soil classes showed greater quantitative variation in the results. This behavior can be attributed to the intrinsic differences in the resolution of the images and the heterogeneity of the samples in these classes, which impacts the results. Spatial analysis revealed that the main discrepancies occurred in anthropized areas such as Mining, Pasture, and Exposed Soil classes, while natural surface classes like Clean Field and Forest Fragment showed less sample dispersion.

Verification of thematic quality

The Kappa index recently has employed by several author as such as,^20–23 for example, concept has originated in,²⁴ is a statistical measure used to evaluate agreement between two sets of categorical data, adjusting for agreement that would occur by chance. In the context of geoprocessing, the Kappa index is widely applied to validate the accuracy of land use and cover classifications from remote sensing images. According to,²⁵ the Kappa index is one of the main methods used to assess agreement between ground truth and thematic maps. A significant advantage of using the Kappa statistic is that the coefficient calculation includes all elements of the error matrix, providing a more robust accuracy assessment. A Kappa value of zero suggests that the classification is no better than a random classification of pixels, while values close to 1 indicate high agreement between the classification and the ground truth.

Validation samples

For the start of the thematic quality assessment, it was necessary to collect stratified random samples in sufficient quantity for analysis. Stratification is justified by the different area quantities of the classes. The sample size for the assessment was obtained using equation (1), as shown below:

$n=\frac{Z_{\alpha /2^2}.p.q}{E^2}$   (1)

Where:

Z_α/2: critical value corresponding to the desired confidence level (1.96); p = probability of success (0.95); q = probability of error (1-p = 0.05); E: margin of error (0.0345).

Applying these parameters in equation 1, we get:

n = (1.96² * 0.95 * 0.05) / (0.0345)² = 806 samples

These 806 samples were distributed proportionally to the class by unit area, as shown in Figure 7.

Figure 7 Distribution of the sampling units.
Source: by the authors

Data tabulation: Mapping classes vs. "Field" verification

In this sense, the analysis of this study involves the agreement between the mapped classes and the typologies interpreted in the verification image. The image used has a spatial resolution of 0.5m, which allowed for a clear visualization and interpretation of the typologies correlated with the mapping classes (Figure 8).

Figure 8 GeoEye image comparison (Field Truth).
Source: by the authors

Data tabulation consists of verifying if the typology of a particular mapped class corresponds to the same typology in the high-resolution image for each sample point.

Confusion matrix – Kappa index and overall accuracy

The use of the Kappa index as a measure of thematic accuracy provided a robust evaluation of the Maxver classification method. This methodology is widely used in various recent academic works.^26–30

Below is a Figure 9 showing the classification for interpreting the Kappa Index from several authors who have studied the topic:

Figure 9 Classification for interpretation of the Kappa Index based in Foody (2020).
Source: by the authors

The confusion matrix is a classic and binary model that, in this study, results from the cross-tabulation between the image classification data and the correctly classified reference information.³¹ The confusion matrices for the Planet (Table 1) and Sentinel-2A (Table 2) tables are presented below:

According to the results from the Planet Classification Confusion Matrix, the Forest Fragment class showed the best accuracy for both the producer (0.87%) and the user (0.92%). The Mining class had the second-best accuracy for the producer at 0.81%; however, according to the matrix, approximately 100 samples presented commission errors with other classes, mostly Pasture, reducing the producer accuracy to 0.12%.

Another class that achieved good producer and user accuracy was Clean Field: 75% and 64% respectively, although about 80 samples showed incorrect classification of this typology where it should have been Pasture.

For the Sentinel image Classification, its Confusion Matrix shows that the Forest Fragment class continues to have good producer (0.82%) and user (0.92%) accuracies. High accuracies were also seen for the Clean Field class (0.73% – producer and 0.86% – user) and Pasture (0.75% – producer and 0.62% – user). Although the Mining class had significant differences between producer (0.88%) and user (0.50%) accuracies, its omission occurred in only 2 samples, and commission had the same number of correct samples (14).³²

The results showed that the supervised classification using the Maxver method yielded good Kappa Index and Overall Accuracy results for both images. For the supervised classification of the Planet image, the values were 56% and 68%, respectively. For the Sentinel image classification, the Kappa index was considered very good (66%) according to the interpretation percentages of ³², and the Overall Accuracy was 76%.

The Forest Fragment and Clean Field classes showed the highest accuracy and precision in the Planet image classification, though the latter presented 80 samples misclassified as Pasture. In the classification of the Sentinel image, using the same samples, the best-performing classes were Forest Fragment, Clean Field, and Pasture.

For both images, other land-use classes such as mining and land use had lower user accuracy, requiring more detailed photointerpretation analyses to achieve better precision. The scatter plots for these classes already showed greater variability in their training samples at the beginning of the study.

Analyzing the scatter plots of the samples at the start of the mapping process is crucial in selecting the training sample set, a step prior to classification. This analysis directly influences the data generated in the Confusion Matrix, which compares the proposed mapping with the ground truth. The better the choice of training samples, the higher the accuracy values.

In addition to evaluating the Kappa index between the supervised classifications in this study, the use of high-resolution images as a verification tool, considered as "ground truth," was positively evaluated. This increased productivity by enabling the use of a larger number of samples in less time and at a lower cost. Finally, it is worth noting that new technologies, such as aerial surveys using RPAs (Remotely Piloted Aircraft), can further enhance speed and reduce costs in field validation sample collection. Through automated flight plans, a drone equipped with RTK can be programmed to follow a route to only the validation sample points, previously generated in the office, providing high-resolution images with centimeter-level positional accuracy.

Acknowledgments for the Analysis and Modeling from Environmental System Post Graduation Program of Federal University of Minas Gerais State – Brazil.

None.

There is no conflict of interest.

1. Foody GM. Challenges in the real world use of classification accuracy metrics: From recall and precision to the Matthews correlation coefficient. PLoS One. 2023;18(10):e0291908.
2. Oliveira WL, Nero MA. Simulação de desmatamento com base na influência do aumento e redução de áreas protegidas com estimativa de emissão de CO². Boletim Goiano de Geografia. 2023;43(1):e69343.
3. Inguaggiato FF, Olivatto TF. Correlação entre contágio de covid-19 e fluxo pelo modal transporte público análise espacial no município de Araraquara (SP). Boletim Goiano de Geografia. 2024;44(1):e76433.
4. Pina MDA, Cremon EH. Dados geográficos e inteligência artificial na predição de locais favoráveis para a instalação de usinas de biogás da agroindústria canavieira. Boletim Goiano de Geografia. 2024;44(1):e77808.
5. Rizzo FA, Santos A, Silva DCC, et al. Técnicas de geoprocessamento aplicadas para análise temporal do microclima na Bacia Hidrográfica do Córrego do Pequiá, Maranhão. Boletim Goiano de Geografia. 2024;44(1):e78032.
6. Santos RSS, Filho MSC, Marques IM, et al. Groundwater contamination in a rural municipality of Northeastern Brazil: application of geostatistics, geoprocessing, and geochemistry techniques. Water, Air, & Soil Pollution. 2024;235(3):179.
7. Habib W, Ingle R, Saunders M, et al. Quantifying peatland land use and CO2 emissions in Irish raised bogs: mapping insights using Sentinel-2 data and Google Earth Engine. Sci Rep. 2024;14(1):1171.
8. Saini R, Singh S. Land use land cover mapping and snow cover detection in himalayan region using machine learning and multispectral sentinel-2 satellite imagery. International Journal of Information Technology. 2024;16(2):675–686.
9. Sun W, Hou T, Chen C, Set al. Mapping China’s coastal aquaculture ponds expansion with sentinel-2 images during 2017–2021. International Journal of Digital Earth. 2024;17(1):2297943.
10. Werner JPS, Belgiu M, Bueno IB, et al. Mapping integrated crop–livestock systems using fused Sentinel-2 and PlanetScope time series and deep learning. Remote Sensing. 2024;16(8):1421.
11. Nero MA, de Morais VTP, Elmiro MAT, et al. Assessment of the influence of DTM quality on dam rupture simulation processes. MOJ Eco Environ Sci. 2024;9(2):61–70.
12. Alípio, JEJ, Alves ISL, Tavares JR, et al. Evaluation of environmental damage in clandestine mining. MOJ Eco Environ Sci. 2023;8(6):240–244.
13. Daher APC, Uhlein GJ, Mattioli J. Caracterização da Bacia Sedimentar da Bandeira na Serra do Tamanduá, Região Nordeste do Quadrilátero Ferrífero, Minas Gerais, Brasil. Revista Brasileira de Geologia. 2024;54(1):e20230009.
14. Ferreira RF, Lima RMF. Avaliação da abordagem heurística para liberação mineral em itabiritos. Tecnologia em Metalurgia, Materiais e Mineração. 2024;21:e3075.
15. Leite FAZ, Castro PTA. Inventário de geossítios localizados na RPPN Sítio Arqueológico Mina de Cata Branca (Itabirito-MG). Revista Brasileira de Ecoturismo (RBEcotur). 2024;17(1).
16. Lobo JL, Cioni IF. Unidades de Conservação e movimentos sociais na conservação dos campos rupestres ferruginosos: Um estudo no Quadrilátero Aquífero-Ferrífero (Minas Gerais): Conservation Units and social movements in the conservation of ferruginous rupestrian grassland: A study in the Aquifer-Iron Quadrangle (Minas Gerais, Brazil). AMBIENTES: Revista de Geografia e Ecologia Política. 2024;6(1):28–68.
17. Mozart MS, Neto AVC, Soares MB, et al. Geochemistry of BIF in the Quadrilátero ferrífero, Brazil, as a proxy to neoarchean paleoenvironmental and depositional conditions. Journal of South American Earth Sciences. 2024;143:105009.
18. Reis ALM, Clemente N, Faria ALL, et al. Chemical and physical characterization of swift guano in quartzitic karst landscape in Brazil. Sociedade & Natureza. 2024;36:e70150.
19. Instituto Nacional De Pesquisas Espaciais - Inpe. Spring: Tutorial De Geoprocessamento: Classificação De Imagens. 2009.
20. Della-Silva JL, Pelissari TD, Santos DH, et al. Land use predicition accuracy of different supervised classifiers over agriculture and livestock economy-based municipality in Brazil. Remote Sensing Applications: Society and Environment. 2024;35:101257.
21. Din SUl, Yamamoto K. Urban spatial dynamics and geo-informatics prediction of Karachi from 1990–2050 using remote sensing and CA-ANN simulation. Earth Syst Environ. 2024;8:849–868.
22. Van Duynhoven A, Dragićević S. The geographic automata tool: a new general-purpose geosimulation extension for ArcGIS Pro. Applied Sciences. 2024;14(15):6530.
23. Yadav PK, Mishra VN, Kumari M, et al. Spatially explicit simulation and forecasting of urban growth using weights of evidence based cellular automata model in a millennium city of India. Physics and Chemistry of the Earth, Parts A/B/C. 2024;136:103739.
24. Cohen J. A coefficient of agreement for nominal scales. Educational and Psychological Measurement. 1960;20(1):37–46.
25. Foody GM. Explaining the unsuitability of the kappa coefficient in the assessment and comparison of the accuracy of thematic maps obtained by image classification. Remote Sensing of Environment. 2020;239:111630.
26. Souza DDR, Filho JCA, Araújo MSB, et al. Análise espaço-temporal do uso da terra em municípios do núcleo de desertificação de Cabrobó, Pernambuco. Revista Contexto Geográfico. 2024;9(18):133–142.
27. Santos MHO, Johann JA, Moura V, et al. Google earth engine in the mapement of authose of strong and annual cultures in Rondan. Anais do XX Simpósio Brasileiro de Sensoriamento Remoto. 2023;20:155984.
28. Bernardo ES, et al. Classificação dos níveis de degradação de pastagens em municípios paulistas por meio da combinação de bandas espectrais OLI/Landsat-8 e 9 e MSI/Sentinel-2. In: Congresso Interinstitucional de Iniciação Científica, Campinas, 2023. Campinas. Anais [...]. Campinas: Embrapa Territorial, 2023.
29. Fleiss JL, Cohen J, Everitt BS. Large sample standard errors of kappa and weighted kappa. Psychological Bulletin. 1969;72(5):323–327.
30. Monserud RA, Leemans R. Comparing global vegetation maps with the Kappa statistic. Ecological Modelling. 1992;62(4):275–293.
31. Congalton RG. A review of assessing the accuracy of classifications of remotely sensed data. Remote Sensing of Environment. 1991;37(1):35–46.
32. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–174.