1. Introduction

Modern agriculture is undergoing a transformative phase with the integration of cutting-edge technologies, data analytics, and machine learning. The ability to accurately predict crop yields is crucial for sustainable farming practices, resource optimization, and ensuring food security [1–4]. In this context, the application of outlier detection algorithms has emerged as a promising avenue for refining crop yield prediction models. This paper delves into the intricacies of outlier detection within the realm of precision agriculture, focusing on a case study conducted in the Davangere region—an agriculturally significant area in Karnataka, India [5–8].

The Davangere region, known for its diverse agricultural landscape, presents unique challenges that demand tailored outlier detection techniques. Traditional methods often struggle to address the complexities inherent in agricultural datasets, characterized by varying soil conditions, climate dynamics, and crop types [9–12]. Hence, this study explores the efficacy of six advanced outlier detection algorithms: isolation forest, elliptic envelope, one-class support vector machine (one-class SVM), iterative R, spatial singular value decomposition (SSVD), and spatial multiview outlier detection (SMVOD) [5]. By applying these algorithms to the locally curated dataset, we aim to scrutinize their performance in identifying anomalies that could significantly impact crop yield predictions.

In addition to evaluating the algorithms individually, we applied a variant of the elliptic envelope approach, which combines principal component analysis (PCA) with density-based spatial clustering of applications with noise [3, 13–16]. This method is specifically designed to address the intricacies of the Davangere dataset, aiming to enhance outlier detection and improve the overall accuracy of crop yield predictions. The outcomes of this study not only contribute to the growing body of knowledge in precision agriculture but also hold practical implications for farmers, policymakers, and stakeholders involved in agricultural decision-making in the Davangere region and similar agroecological contexts [17–21]. This research strives to provide valuable insights for sustainable farming practices in the face of evolving environmental and climatic conditions.

2. Literature Review

Arun Kumar et al. proposed the spatial iterative R algorithm, emphasizing its efficiency in identifying outliers and mitigating the impact of masking and swamping effects on the dataset [1]. This iterative approach holds promise in addressing the challenges associated with outlier detection, especially in scenarios where anomalies might be overshadowed by prevalent data patterns [1].

Zhao et al. introduced SMVOD, a method that leverages spatial information from multiple views to comprehensively capture outliers. This approach acknowledges the multidimensional nature of data and strives to enhance outlier detection by considering diverse aspects simultaneously [2].

Ma and Yin focused on PCA as a novel method for large-scale and time-dependent data analysis. The use of robust techniques suggests a dedicated effort toward addressing challenges posed by outliers and anomalies within the data, particularly in the context of dynamic and voluminous datasets [3].

Xu et al. explored the isolation forest algorithm, introducing a new representation scheme involving randomly initialized neural networks. This innovative approach is aimed at enhancing data partitioning through random axis-parallel cuts, showcasing a creative adaptation of neural network–based techniques for outlier detection [4].

Arun Kumar et al. provided a comprehensive analysis of spatial multivariate outlier detection methods. Their study not only explores existing methods but also sheds light on their strengths and limitations, contributing to a nuanced understanding of spatial multivariate outlier detection techniques [5].

Nasir Usman et al. conducted an analysis using elliptic envelope, isolation forest, and one-class SVM. Notably, they underscored the impact of hyperparameter tuning on the models’ performance, showcasing improvements in metrics such as precision, recall, specificity, accuracy, AUC, and F1 score, particularly for isolation forest [6].

Liu et al. proposed PCA-DBSCAN, addressing practical challenges such as deviations from spectrometers and anomalies in spectrum characteristic peak intensities. This approach highlights the often-overlooked presence of outliers in categorical data, offering a novel perspective on outlier detection in specific domains [7].

Collectively, these studies contribute to the evolving landscape of outlier detection methods, offering valuable insights and innovative approaches tailored to diverse datasets and application domains. The diverse methodologies and considerations presented in these works provide a rich foundation for further advancements in the field of outlier detection.

3. Dataset Collection and Description

For this research, we collected an extensive agricultural dataset from Taralabalu Krushi Vignyana Kendra, Davangere, capturing crucial parameters that significantly influence crop yield prediction. The dataset encompasses diverse features, providing a holistic view of the agricultural landscape in the Davangere region [8, 22–25]. The dataset was locally sourced, ensuring its relevance to the specific agroecological conditions prevalent in the area.

4. Outlier Detection Algorithm

This part delves into the pivotal role of outliers in influencing the precision and dependability of agricultural predictions. We embark on an exploration of diverse outlier detection and handling techniques aimed at fortifying the resilience of our crop yield prediction models. The arsenal of methods enlisted comprises isolation forest, DBSCAN, one-class SVM, elliptic envelope, iterative R, SSVD, and SMVOD. Each of these methods offers a distinctive approach to discerning and mitigating outliers, thereby enriching our comprehension of the dataset and elevating the efficacy of predictive models [36–39].

4.2. Elliptic Envelope

The standard elliptic envelope algorithm is used for robustly estimating the covariance of a dataset and identifying outliers. It assumes that the majority of the data follows a Gaussian distribution, and it models the inlying data points as an elliptical-shaped envelope which is shown in Figure 1. Outliers, which deviate significantly from the assumed distribution, are identified based on their Mahalanobis distances [6].

In the diagram, the green area is an ellipse. Therefore, an imaginary elliptical area is created around a given dataset by the elliptical envelope algorithm. Any values outside of the envelope are returned as outliers, while values inside the envelope are regarded as typical data. Naturally, this algorithm should recognize the red data points in the above diagram as outliers. Figure 1 makes it clear that a Gaussian distribution of the data is ideal for the algorithm to function.

Mahalanobis distance is a measure of the distance between a point and a distribution, taking into account the correlation between variables. It is defined as

$$ (2)

where x is the data point, μ is the mean of the distribution, and Σ is the covariance matrix of the distribution.

The elliptic envelope fits around the inlying data points, assuming a Gaussian distribution. The envelope is defined as

$$ (3)

where $$ is the chi-square threshold for a given confidence level α.

The proposed elliptic envelope uses PCA and Euclidean distance instead of Mahalanobis distance. PCA transforms the data from a high-dimensional space into a lower-dimensional principal component (PC) space:

$$ (4)

where X is the original data matrix (n × d) and W is the matrix of PCs (eigenvectors of the covariance matrix). X^′ is the transformed data in PC space (reduced dimensions), Euclidean distance for outlier detection. Instead of using Mahalanobis distance, the proposed method computes the Euclidean distance from the center:

$$ (5)

Instead of assuming an elliptical distribution, the proposed method determines a threshold dynamically:

$$ (6)

where P95 is the 95th percentile of all distances. If a point is farther than this threshold, it is classified as an outlier.

The key difference between the proposed elliptic envelope and standard elliptic envelope is that the standard elliptic envelope fails on skewed or non-Gaussian data, which real-world agricultural datasets often have, and it also assumes an elliptical boundary, which is not flexible for complex data distributions, whereas the proposed elliptic envelope used PCA and Euclidean distance, which are more flexible because PCA captures the major variance, reducing noise from irrelevant features. Euclidean distance is simpler and works better for arbitrary-shaped distributions: more robust for high-dimensional data, where Mahalanobis distance struggles.

5. Performance Analysis

Evaluating performance through precision, recall, and F-score offers valuable insights into the efficacy of outlier detection methods. These metrics gain significance, especially in scenarios involving imbalanced datasets, where the presence of outliers is notably smaller compared to the abundance of inliers. The confusion matrix helps in assessing the performance of the algorithm by providing a clear breakdown of correct and incorrect classifications [21–25]. Figures 2 and 3 show the confusion matrix of the respective six algorithms which we used for outlier detection.

According to Figure 4, all the outlier detection methods exhibit reasonable to high performance in terms of precision, recall, and F-score. Elliptic envelope and one-class SVM stand out with particularly high recall values, suggesting that they are effective in capturing a large proportion of actual outliers. Isolation forest, iterative R, SSVD, and SMVOD demonstrate well-balanced performance, providing reliable outlier detection with a reasonable trade-off between precision and recall.

6. Results

Before the implementation of an outlier detection algorithm, the dataset typically reflects the unprocessed distribution of raw data. At this stage, anomalies, noise, or outliers may be present, leading to irregular patterns or unexpected values in the dataset. Visualizations or figures generated from this raw data aid in illustrating the initial characteristics of the dataset. Figures 5a, 6a, 7a, 8a, 9a, and 10a depict the plot before the removal of outliers, showcasing scattered points with irregular patterns, and some observations deviate from the overall pattern [26–30]. Following the application of an outlier detection algorithm, the figures portray the dataset after processing, wherein outliers or anomalies have been identified and potentially eliminated. This step is pivotal for enhancing the data quality, particularly in scenarios where anomalies could adversely affect subsequent analyses or modeling. Figures 5b, 6b, 7b, 8b, 9b, and 10b illustrate the removal of outliers, resulting in a clearer and more representative pattern.

The accuracy of each outlier detection model is depicted in Figure 11. Among these models, the elliptic envelope exhibits the highest accuracy at 91%, while the remaining models show accuracies ranging from 81% to 86%, with isolation forest registering the lowest accuracy [31–35]. Table 2 gives the measure of performance based on performance metrics like precision, recall, and F1 score of all algorithms.

7. Conclusion

This paper systematically explored and evaluated various outlier detection methods in the context of crop yield prediction for the Davangere region. Leveraging a diverse dataset from Taralabalu Krushi Vignyana Kendra, the study investigated isolation forest, DBSCAN, one-class SVM, elliptic envelope, iterative R, SSVD, and SMVOD. Each algorithm exhibited distinctive strengths and weaknesses, with elliptic envelope demonstrating the highest accuracy. Spatial methods showcased unique perspectives on outlier detection, while iterative R contributed to robust regression against outliers. Performance analysis using precision, recall, and F-score provided a comprehensive assessment of algorithm effectiveness, particularly valuable for imbalanced datasets. The visualizations illustrated the transformative impact of outlier removal on the dataset. Overall, this research contributes insights that enhance the understanding and application of outlier detection in the precision agriculture domain, supporting further research and development.

Conflicts of Interest

The authors declare no conflicts of interest.

Funding

No funding was received for this manuscript.