1.1. Motivation of the Study

The widespread mortality from LC remains among the top cancer-related causes because medical professionals find it challenging to detect early symptoms without sophisticated diagnostic systems. Traditional CT scan diagnosis methods, which are time-consuming, labor-intensive, and prone to human error, often produce numerous false positives (FPs) and false negatives (FNs). The application of ML and DL for automated diagnosis continues to encounter three key limitations which involve class unbalance within datasets and data noise problems together with the difficulty of extracting crucial clinical features from complex image data. The hierarchical feature extraction capabilities of CNNs for our dataset are advanced, while these tools demonstrate inadequate performance for working with clinical data formats. Ensemble models including eXtreme gradient boosting (XGBoost) and CatBoost excel at processing structured data information while operating poorly on imaging data in its raw format. The complementary strengths of these techniques inspired the development of a hybrid framework that integrates their advantages. ConvXGB and ConvCatBoost, which integrate CNN-based deep feature extraction with gradient boosting classifiers to create an accurate, reliable, and interpretable early LC detection system, are proposed. The combination of these methods produces an improved diagnostic procedure which overcomes individual framework limitations. The combination method lowers misdiagnosis errors due to unbalanced datasets and enhances performance across various healthcare scenarios through its scalable deployment capabilities. The aim of this study is to establish a data system which connects imaging processes to structured information analysis to deliver prompt doctor-driven medical knowledge. This research work serves to advance intelligent diagnostic systems which detect LC at an earlier stage with higher precision along with improved operational efficiency for potential life-saving applications that transform clinical oncology decisions.

1.2. Contribution of the Study

3.1. Experimental Setup

The experimental setup was conducted on a high-performance computing workstation equipped with an Intel Core i7-11800h processor running at 2.30 GHz, 32 GB DDR4 RAM, and an NVIDIA GeForce RTX 3060 GPU with 6 GB of dedicated memory, operating on Windows 11 Professional 64-bit. All experiments were implemented using Python 3.10 as the primary programming environment. DL models, including the lightweight 1D CNN used for feature extraction, were developed using Tensor Flow 2.11 and Keras libraries. Traditional ML models such as logistic regression (LR), KNN, SVM, RF, XGBoost, and CatBoost were implemented using the Scikit-learn, XGBoost, and CatBoost libraries. Data preprocessing, augmentation, and visualization were performed using Pandas, NumPy, Matplotlib, and Seaborn libraries [62]. The setup leveraged the computing power of the workstation to handle the complex tasks of training DL models and evaluating performance metrics.

3.2. Dataset

A publicly available dataset (https://www.kaggle.com/datasets/mysarahmadbhat/lung-cancer) was utilized for this study [63]. The dataset involved a survey conducted among individuals with and without LC, aiming to investigate various characteristics that could be correlated with the illness. The dataset comprises 309 instances with 15 clinical attributes (gender, age, smoking, yellow fingers, anxiety, peer pressure, chronic disease, fatigue, allergy, wheezing, alcohol consumption, coughing, shortness of breath, swallowing difficulty, and chest pain) and one target variable (LC) [4]. To address the imbalanced class distribution between LC and non-LC instances in the dataset, SMOTE is applied [64]. Non-null counts indicate the number of available values per column, with most features being integers. However, gender and LC are categorical or textual type objects. This detailed description encapsulates preprocessed patient information or data for further processing within an ML pipeline.

3.3. ML Classifiers

In the diagnosis of LC, ML models play a pivotal role. This study employs a range of classifiers, including LR, KNN, SVM, RF, XGBoost, and CatBoost. Additionally, this work involves developing ensemble models called ConvXGB and ConvCatBoost which integrates CNN with XGBoost and CatBoost algorithms. All algorithms trained meticulously to leverage its unique strengths, aiming to deliver precise LC prognosis. To enhance model robustness, k-fold cross-validation with k = 5 was utilized, ensuring reliable performance assessment and minimizing overfitting [65]. The next section addressed the overall methods addressed in the following section.

3.4. CNN Architecture

CNN [78] is a computational model that integrates convolution, pooling, and fully connected layers. The convolution and max-pooling layers’ extract features crucial for analysis, while the fully connected layers map these features to the final output. Our investigation involved implementing a 1D CNN architecture design for experimental analysis consisting of a sequence of three Conv1D layers followed by two max-pooling layers for downsampling. This CNN architecture is customized to effectively manage one-dimensional data, thus rendering it suitable for a variety of classification tasks. The architecture of this CNN [78] used in our work shows a summary of the provided CNN model (Table 1).

Our work introduced a CNN model with an input shape of (15, 1), representing 15 time steps and a single feature (channel). It employs three convolutional layers, with 30, 60, and 90 filters, respectively. Each filter operates with a kernel size of (3, 1), extracting features from the input data. In order to introduce nonlinearity in the network, ReLU activation functions throughout all layers were employed. Following the convolutional layers, the study applied two max-pooling layers of size (2,1) and formed new feature maps with reduced spatial dimensions when only the highest values from each region were retained. Then, a flatten layer is used to map the multidimensional feature maps to a one-dimensional vector to fit to subsequent fully connected layers. To avoid overfitting while training, a dropout layer with a 0.5 dropout rate was added. This technique randomly deactivates half of the neurons in each training step, which encourages better generalization. The network then moves to two dense layers with 128 and 50 neurons, both layers being followed by ReLU activation for enhancement of learning capacity. Finally, the output layer is a neuron with a sigmoid activation function whose output represents a probability score for the binary classification predicting LC or not in the patient. The CNN model, trained with binary cross-entropy loss, well suited for two-class problems, used Adam for optimization. The model performance with accuracy as the vital parameter was assessed. Creative design of our CNN architecture was meant to take care of 1D sequential data like the time series and structured clinical record data, which made it suitable for healthcare prediction tasks.

3.5. ConvCatBoost

ConvCatBoost is a novel approach that combines CNN with the CatBoost algorithm to enhance classification performance, particularly in LC detection. This fusion leverages CNN’s ability to extract intricate features from input data, while CatBoost refines predictions through gradient-boosted decision trees [82]. By integrating these two methodologies, ConvCatBoost is well-equipped to manage numerical and categorical data effectively. The resulting model exhibits improved accuracy and robustness, making it a valuable tool for early LC detection.

3.6. ConvXGB

An ensemble learning based ConvXGB algorithm is proposed to address classification challenges for LC detection effectively. ConvXGB combines the functionalities of CNN [83] and XGBoost [73], thereby leveraging the strengths inherent in both methodologies. ConvXGB encompasses two fundamental components, aiming to synergistically merge the capabilities of CNN [83] and XGBoost [84]. Employing CNN for feature extraction effectively captures important spatial patterns essential for identifying LC. Pretrained CNN models are employed to extract features, which are fed into an ensemble part for classification. After extracting features, ConvXGB leverages the XGBoost component to analyze the data, iteratively refining predictions. By combining these refined predictions, ConvXGB provides accurate diagnoses of LC, reflecting the probability of its presence based on both input data and extracted features. The detailed steps of this process are outlined (Algorithm 1), which defines the pseudocode for the ConvXGB model algorithm.

3.6.2. Integration of CNN Features With CatBoost and XGBoost

The decision to develop ensemble models ConvXGB and ConvCatBoost was driven by the need to integrate the strengths of both DL and gradient boosting approaches for robust LC prediction. CNNs are well-suited for capturing hierarchical and nonlinear patterns within structured data, but they are prone to overfitting when trained on small or imbalanced clinical datasets and often lack interpretability. In contrast, XGBoost and CatBoost are gradient boosting algorithms that excel in processing tabular data, offer superior regularization capabilities, and are highly effective in handling class imbalance—a common issue in medical datasets. By first extracting deep features using a lightweight 1D CNN and then passing those high-level representations into XGBoost and CatBoost classifiers, the ensemble models harness both local feature learning and strong predictive generalization. This architecture was specifically selected after comparative evaluations demonstrated that these combinations consistently outperformed standalone CNN, XGBoost, CatBoost, and traditional ML models across all key performance metrics. Thus, the ensemble strategy not only enhances classification performance but also ensures robustness, scalability, and clinical interpretability—all critical factors for real-world LC diagnostic applications.

The method uses CNN features through a structured two-stage ensemble model which incorporates CatBoost along with XGBoost. A preprocessing step normalizes clinical tabular features through min–max normalization and one-hot encoding of categorical data. This process applies to the dataset which contains 309 samples and 15 features. The class unbalance is solved through the SMOTE that produces an equalized dataset for training purposes. A 1D CNN operates as a lightweight extractor for high-level features during the second processing stage. The designed CNN uses three Conv1D layers with ReLU activation functions followed by two MaxPooling1D layers alongside two dense layers with 128 neurons and 50 neurons, respectively. Training of the CNN model occurs through 150 epochs with 160 batch size using the binary cross-entropy loss function and the Adam optimizer [91]. The trained deep feature vectors consist of 50 dimensions extracted from the second-last dense layer, which produces a feature matrix with a shape of (309, 50) [92]. The extracted features provide input to independently train two boosting algorithms known as ConvCatBoost operated by CatBoost and the ConvXGB system built using XGBoost. CatBoost adopts a logistic loss for its optimization process. The CNN trains separately from both boosting models, which execute the last stage of binary cancer classification [93]. The analysis uses a fivefold cross-validation technique throughout CNN feature extraction and training of two boosting classifiers (CatBoost and XGBoost) to achieve a consistent, robust, and unbiased assessment. These boosting models from ConvCatBoost and ConvXGB generate two-class predictions for LC-Yes or LC-No, which receive their performance evaluation metrics. The two-stage ensemble methodology lets CNN extract complicated patterns from data before gradient boosting algorithms use them to improve clinical decisions, thus forming a practical framework for detecting LC.

To clarify the integration process, we describe the complete model pipeline for ConvXGB and ConvCatBoost as follows:

• •

  Data preprocessing: The raw clinical dataset (309 samples × 15 features) is normalized using min–max scaling, and categorical variables (e.g., gender and LC status) are encoded using one-hot encoding.

• •

  Balancing classes: SMOTE is applied to handle class imbalance, resulting in a balanced dataset (550 samples).

• •

  CNN-based feature extraction:

  • •

    A lightweight 1D CNN is trained on the structured tabular data. The architecture includes three convolutional layers followed by max-pooling layers, dropout, and dense layers.

  • •

    Deep features are extracted from the penultimate dense layer (128 or 50 dimensions depending on configuration), resulting in a feature matrix of shape (N, D), where N is the number of samples and D is the feature vector size (e.g., 50).

• •

  Ensemble learning stage:

  • ➢

    The extracted features are fed into two separate classifiers:

    • ▪

      ConvXGB: uses XGBoost for final classification.

    • ▪

      ConvCatBoost: uses CatBoost for final classification.

  • ➢

    Both classifiers are trained on the CNN-generated feature vectors using binary logistic loss.

• •

  Model evaluation:

  • •

    Fivefold cross-validation is applied.

  • •

    Metrics such as accuracy, precision, recall, F1-score, and AUC are computed.

This modular pipeline ensures that CNN handles deep pattern extraction, while boosting classifiers focus on robust, interpretable decision boundaries. A visual overview is provided in Figures 2 and 3. The process includes data preprocessing, class balancing using SMOTE, CNN-based feature extraction, and classification using gradient boosting models (XGBoost and CatBoost) in Figure 3.

The ensemble models ConvXGB and ConvCatBoost were chosen to exploit the complementary strengths of CNNs and gradient boosting techniques (XGBoost and CatBoost). CNNs excel at hierarchical feature extraction, especially in handling nonlinear patterns from structured or spatial data. However, they may struggle with small or imbalanced datasets and lack interpretability. On the other hand, XGBoost and CatBoost are highly efficient with structured clinical/tabular data, offer strong regularization, and are particularly effective in managing class imbalance, which is common in medical datasets. By combining CNN for deep feature extraction with CatBoost and XGBoost for robust classification, we aim to create an ensemble architecture that mitigates individual weaknesses (like CNN overfitting or limited tabular performance) and boosts classification accuracy, precision, and recall, especially in early-stage LC detection. These specific ensemble models were selected after a comparative evaluation of multiple classifiers, showing consistent improvement across performance metrics over both standalone CNN and traditional ML models.

3.7. Performance Evaluations Method

These performances were assessed using the confusion matrix, which comprises TP, true negative (TN), FP, and FN elements.

In our study, model performance is assessed using standard classification metrics including accuracy, precision, recall, F1-score, and area under the receiver operating characteristic curve (AUC-ROC). The confusion matrices presented in figures are based on results from individual validation folds or single 90:10 train-validation splits and reflect model performance at a fixed decision threshold of 0.5 [96]. In contrast, the AUC values reported in summary tables are calculated from ROC curves that consider all possible classification thresholds, providing a threshold-independent evaluation of model performance. Furthermore, AUC results are averaged over fivefold cross-validation to ensure robustness and generalizability [97]. This distinction explains the differences observed between confusion matrix–based estimations and AUC scores in the results.

4.1. Data Preprocessing and Dataset Preparation

The initial steps in detecting LC involve preprocessing the dataset to address outliers and remove unnecessary information, ensuring data quality and reliability. Dealing with missing values is crucial, and various ML techniques are employed to address this issue by enhancing dataset quality. Training aims to optimize model parameters for accurate LC detection, while testing addresses the validation of the model in generalizing performance over new data instances.

Prior to the analysis, the dataset was preprocessed to ensure proper scaling and encoding of the variables. Continuous features, such as age, are scaled using min–max normalization to transform the values to a common range between 0 and 1. Then categorical variables, such as gender and LC status, were encoded using one-hot encoding. This technique transforms each categorical variable into a set of binary values, where each column represents a unique category.

A detailed overview of the dataset partitioning, both before and after applying SMOTE, is summarized in Table 2. The dataset was split into training and validation sets using a 90:10 ratio.

4.2. Feature Extraction

Feature analysis in the study utilized an extensive array of variables to encompass the complex aspects of LC. These variables encompassed factors like smoking history, cough presence, alcohol consumption, and yellow finger appearance. The correlation matrix and heat map visualizations were utilized to identify trends and associations among the distinct characteristics in the dataset. This thorough feature engineering process aimed to capture the most relevant and informative predictors of LC.

The representation of features in each class is shown in Table 3. Regarding gender, both males and females demonstrate comparable probabilities of receiving an LC diagnosis. Our examination indicates that each scrutinized feature is detectable in LC patients to varying degrees, spanning from 25% to 36%. Moreover, a noteworthy portion of individuals displays these symptoms even in the absence of an LC diagnosis. Regular clinical assessments and monitoring of risk factors may assist in averting or alleviating the detrimental consequences of LC, notwithstanding the absence of the disease.

Figure 4 illustrates the distribution of participants across different age groups, revealing a higher prevalence of LC among individuals aged 50–79. Among these age groups, the 60–64 age bracket exhibits the highest frequency of LC cases. The relationships between features based on the heat map mentioned in Figure 5 were designed.

4.3. Performance Evaluation of LC Prediction Based on Unbalanced Data Using ML

To predict LC, ML algorithms such as LR, KNN, SVM, XGBoost, RF, and CatBoost with cross-validation folds such as 1, 2, 3, 4, and 5 techniques (Table 4) were used. Additionally, we employed SMOTE to balance the data. It was found that in the case of an unbalanced dataset, the RF model has the highest performance in terms of accuracy, recall, AUC, F1-score, and precision, which were found to be 0.7201, 0.9748, 0.9356, 0.9484, and 0.9198, respectively. Additionally, the mean KNN model of this study has the lowest performance in terms of accuracy, recall, AUC, F1-score, and precision, which were found to be 0.6648, 0.9680, 0.8532, 0.9347, and 0.9074, respectively.

4.4. Performance Evaluation of LC Prediction Based on Balanced Data Using ML

The LR, KNN, SVM, XGBoost, RF, and CatBoost models with fivefold cross-validation techniques are used to predict LC based on balanced data (Table 5). The low precision (0.100) for the CNN model may be due to a high number of FPs in the balanced dataset, potentially caused by insufficient feature discrimination in the standalone CNN architecture. To mitigate overfitting observed in the CNN model, a dropout rate of 0.5 was applied, and early stopping techniques were explored, which showed potential for improving generalization on the balanced dataset. It was found that the CatBoost model of this study has the highest performance in terms of accuracy, recall, AUC, F1-score, and precision, which were found to be 0.9550, 0.9398, 0.9895, 0.9541, and 0.9708, respectively, under the balance dataset. Additionally, the mean KNN model of this study has the lowest performance in terms of accuracy, recall, AUC, and F1-score, which are found to be 0.9143, 0.8286, 0.9635, and 0.9056, respectively. While the lowest precision is 0.9346 for the LR model.

4.5. Performance Evaluation of LC Prediction Based on Unbalanced Data Using CNN and Ensemble DL Models

In this section, the performance of the proposed CNN and ensemble approach is evaluated for the unbalanced dataset. To train CNN with the unbalanced dataset, it is divided into training and validation sets as 90:10, respectively. In this phase, the proposed CNN architecture facilitates simultaneous training and testing of models. With 150 epochs configured and a batch size of 160, the model iterates through the dataset, honing its understanding of the features within. The validation loss and accuracy curve for CNN across different epochs for the unbalanced dataset is shown in Figure 6.

Overall proposed CNN model learned with low bias, and the model seems to be overfitting, followed by less learning till 25 epochs; then, the model showed improved learning. In this work, an ensemble-based approach in composition of the CNN model with XGBoost and CatBoost classifiers based on their performance demonstrated (Tables 4 and 6) is proposed. Deep features are fed to ensemble learning models, namely, ConvCatBoost and ConvXGB, for the prediction of LC.

The performance of CNN models of this study is evaluated under unbalanced dataset using confusion matrices and ROC curves as shown in Figures 7 and 8, respectively. The confusion matrix provides overall performance metrics for the CNN and deep ensemble models ConvCatBoost and ConvXGB by comparing predicted and true labels for LC prediction. The CNN model’s metrics such as accuracy (0.9565), precision (0.9740), recall (0.9615), and F1-score (0.9677) highlight its effectiveness in binary classification tasks.

Overall, Figure 7 indicates that ConvXGB demonstrated superior performance compared to CNN and ConvCatBoost, particularly regarding FP and TN for predicting LC. The ROC curve in Figure 8 for CNN and ensemble models ConvCatBoost and ConvXGB using unbalanced dataset illustrates binary classification performance across various threshold settings. It plots the TPR against the FPR, offering insight into the trade-offs between precision and recall. The ensemble models outperformed the baseline CNN model in terms of the ROC curve.

The performance of the CNN and ensemble-based models, namely, ConvXGB and ConvCatBoost, was evaluated for LC prediction on the unbalanced dataset. Performance metrics such as accuracy, precision, recall, and F1-score were derived from the confusion matrix for all models. The results for CNN, ConvCatBoost, and ConvXGB (Table 6) showed that the overall LC prediction performances in terms of accuracy with the unbalanced datasets were 0.9565 for CNN, 0.9736 for ConvCatBoost, and 0.9739 for ConvXGB, respectively. Similarly, LC prediction performances in terms of precision with the unbalanced dataset were 0.9740, 0.9870, and 0.9747 for CNN, ConvCatBoost, and ConvXGB, respectively. LC prediction performances in terms of recall with the unbalanced dataset were 0.9615, 0.9744, and 0.9872 for CNN, ConvCatBoost, and ConvXGB, respectively. In terms of F1-score with the unbalanced dataset, the scores were 0.9677, 0.9806, and 0.9803 for CNN, ConvCatBoost, and ConvXGB, respectively. In the proposed work of this study, the CNN and ensemble DL models using the unbalanced data ConvCatBoost and ConvXGB exhibit superior accuracy, implying better classification despite data imbalance. While CNN and ConvCatBoost demonstrated higher precision, ConvXGB showed a slight lag in precision. Recall scores show consistency among CNN and ConvCatBoost, with ConvXGB slightly lower. ConvCatBoost tops the F1-score, followed closely by CNN and ConvXGB, indicating robust performance in LC detection despite unbalanced dataset challenges.

4.6. Performance Evaluation of LC Prediction Based on Balanced Data Using CNN and Ensemble DL Models

In this section, a performance evaluation of LC prediction using CNN and an ensemble approach for the balanced dataset was conducted. The CNN training phase comprised 150 epochs, with a batch size set to 160 for each epoch, averaging 39 ms/step. Following training, the CNNs exhibited enhanced performance in comparison to conventional ML techniques by achieving a remarkable accuracy of 95.13%. Additionally, the findings of this study are supplemented with visual representations in Figure 9, which illustrate the learning process of the CNN model on the balanced dataset.

ConvCatBoost and ConvXGB, both deep ensemble models integrating CNN and boosting algorithms, were demonstrated remarkable performance in binary classification for LC prediction. ConvCatBoost shows precision (0.9870) and recall (0.9744), with accuracy (0.9736) and notable F1-scores (0.9806), indicating accurate categorization. Meanwhile, ConvXGB achieved impeccable recall (0.9872) and high precision (0.9872), resulting in an accuracy (0.9826) and impressive F1-scores (0.9872). Overall, ConvXGB outperformed with exceptional precision, recall, and F1-scores, highlighting its efficacy in precise classification tasks. Confusion matrix for balance data with deep ensemble models CNN, ConvCatBoost, and ConvXGB shown in Figure 10 described the classification task for prediction LC with predicted and true levels. Figure 11 explains the ROC curve results for the CNN with Sigmoid model, ConvCatBoost, and ConvXGB, respectively, indicating AUC is 1.00, indicating perfect discrimination between positive and negative instances.

In the CNN-based ensemble model of this study, the performance for balanced data (Table 7) presents the performance measures such as accuracy, precision, recall, and F1-score for DL architectures: CNN utilizing sigmoid activation, ConvCatBoost, and ConvXGB. ConvXGB attains the highest values in accuracy (0.9826) and precision (0.9872), surpassing recall (0.9872) and F1-score (0.9872), demonstrating a robust overall performance. In comparison, ConvCatBoost demonstrates inferior performance when contrasted with ConvXGB, with values of accuracy, precision, recall, and F1-score of 0.9736, 0.9870, 0.9744, and 0.9806, respectively. The CNN model employing sigmoid activation achieves recall (0.9024), implying accurate positive forecasts, albeit with precision (0.100) and F1-score (0.487) in relation to ConvXGB. In summary, ConvXGB emerges as the most effective model across all criteria, displaying superior levels of accuracy, precision, recall, and F1-score.

4.7. Comparison Results Among Various LC Prediction Models

This research shows a breakdown of the models’ performance metrics, including accuracy, precision, recall, and F1-score (Table 8). An extensive assessment conducted for range of classifiers, encompassing conventional approaches like LR, KNN, SVM, XGBoost, RF, and CatBoost and DL-based CNN model, as well as ensemble methods like ConvXGB and ConvCatBoost. Overall, ConvXGB emerged as the most effective performer across evaluation metrics, displaying outstanding levels of accuracy (98.26%), recall (98.72%), and F1-score (98.72%) in contrast of precision KNN outperformed all models by exhibiting high precision (99.15%). This highlights the potency of amalgamating the representational capacity of CNN with the boosting capabilities of XGBoost for robust binary classification. CNN and ConvCatBoost also exhibited noteworthy outcomes, particularly excelling in recall and F1-score, demonstrating their adeptness in precisely recognizing positive instances. Ensemble methods such as ConvXGB and ConvCatBoost demonstrated competitive performance, attaining elevated accuracy levels and maintaining balanced precision–recall trade-offs. Traditional methods like LR, KNN, SVM, XGBoost, RF, and CatBoost also delivered respectable results, albeit with slight discrepancies in certain metrics. These discoveries give important advice to researchers and practitioners in the identification of fitting classifiers customized to demands and circumstances, hence supporting creativity and development in ML and AI research.

4.8. Comparison of the Proposed and Existing Methods

In the comparison of the proposed ensemble DL models of this study with the fundamental methodology analyzed in this study to augment accuracy and efficiency (Table 9), it was demonstrated that there is a contrast with previously documented research. This study’s proposed ensemble DL model shows a significant improvement in accuracy and efficiency compared to earlier methodologies explored in this study. Past research utilizing models such as ANN, WDELM, CNN, and RotF has reported commendable accuracy levels. However, the proposed approach of this study stands out due to its ensemble design, which integrates the strengths of CNN and XGBoost. This fusion enables the model of this study to achieve impressive accuracy, outperforming previously documented models.

This high level of performance emphasizes the effectiveness of the frame worked this study in accurately classifying cases into LC-Yes and LC-No categories. By combining the feature-learning capability of CNNs with the powerful classification strength of XGBoost, the ensemble model of this study captures both spatial and structured data patterns with high precision. Moreover, the inclusion of the SMOTE addresses class imbalance, enhancing the model’s ability to generalize across diverse clinical scenarios.

LC detection remains a critical area in medical AI due to its high mortality rate and the need for early, accurate diagnosis. Over the years, researchers have explored various ML and DL techniques to improve prediction outcomes. ANN models, for example, have been widely used due to their ability to handle complex nonlinear data. One such model achieved 90.91% accuracy on a multimodal clinical dataset [99], while another study reported 93% using public medical data [19]. Despite these results, ANN performance is often highly dependent on data quality and can struggle with large, high-dimensional datasets.

Ensemble models such as WDELM have shown moderate success by combining multiple learners, achieving 92% accuracy on LC datasets [100]. While ensemble methods offer robustness, they sometimes rely on hand-crafted features or shallow models, which may not fully capture intricate patterns in medical imaging. DL architectures like CNNs have addressed this limitation by automatically learning hierarchical representations from image data, with one CNN-based model reaching 95% accuracy [101]. Additionally, RotF, an ensemble technique, achieved 97.1% accuracy on demographic data [4], highlighting the strength of model diversification.

Building on these advances, the proposed ConvXGB model of this study introduces an ensemble architecture: first, deep features are extracted using a pretrained CNN, which captures spatial dependencies in the input; then, XGBoost is applied for high-precision classification. This combination leverages the advantages of both DL and gradient-boosted decision trees, allowing the model to refine predictions and improve classification performance iteratively.

The experimental results of this study confirm the superiority of ConvXGB, which not only achieves the highest accuracy 98.26% among the compared models but also excels across other key evaluation metrics such as precision, recall, F1-score, and AUC. These findings underscore the model’s robustness, reliability, and potential for real world clinical implementation. By harnessing the complementary strengths of CNN and XGBoost, ConvXGB offers a scalable and effective solution for LC prediction one that holds promise for enhancing diagnostic accuracy and supporting early intervention in clinical settings.

4.9. Applications of the Study

The manuscript endeavors to significantly improve the precision and efficacy of LC diagnosis by applying advanced ML methods. It assesses the effectiveness of ensemble learning compared to ConvXGB for detection, specifically focusing on the complexities related to tumor compositions. By connecting the power of ensemble learning and ConvXGB, the manuscript aims to exceed the constraints of traditional diagnostic approaches. The precision of diagnostic procedures is enhanced through the classification accuracy of ensemble learning and the feature extraction capabilities of ConvXGB. The research is aimed at identifying the most effective model for classifying LC, providing valuable insights into the efficiency of ensemble learning and ConvXGB in interpreting medical images. The expected outcomes suggest the enhancement of ML algorithms for precise cancer identification. This development has the potential to revolutionize the field of diagnosis and treatment, leading to improved patient outcomes and more informed clinical decision-making. It sets the stage for improved management of LC and a decrease in mortality rates.

4.10. Limitations of the Study

The investigation presents encouraging results, yet it recognizes various constraints. Initially, the magnitude and variety of the dataset might limit the model’s applicability, necessitating more extensive and diverse datasets for thorough validation. Moreover, the uneven distribution of classes within the data may skew model performance, underscoring the importance of addressing class imbalance. Furthermore, depending solely on predefined features derived from CT scans could potentially overlook significant biomarkers. Integrating sophisticated feature selection techniques or utilizing DL for automated feature extraction could boost the models’ capacity to detect subtle patterns and enhance diagnostic precision [102]. Additionally, the selection of validation methods can significantly influence the credibility of the study. Accordingly, employing a range of validation approaches and performing external validation on distinct datasets are essential measures to ensure the reliability of the results. The translation of the models into clinical practice presents challenges, including the necessity for validation in real-world scenarios and regulatory authorization. Ensuring the interpretability of models, especially for DL models recognized for their opaque nature, is crucial for clinical acceptance. Furthermore, ethical considerations such as patient confidentiality, data protection, and algorithmic biases should be meticulously managed to safeguard patient rights and foster trust in AI-driven healthcare solutions [103]. Transparent disclosure of model performance and potential biases is critical for ethical and accountable deployment. Tackling these constraints in future research can propel the domain of LC diagnosis forward, leading to the establishment of more dependable, clinically relevant, and ethically sound ML-based methodologies [104].

4.11. Future Scope of the Study

This research successfully proves the value of combined DL with ensemble methods for detecting LC, but multiple improvement areas exist for future development. The framework requires increased availability of diverse clinical reports from various institutions to ensure effective model generalization across different demographics and medical conditions. The characteristic features of early-stage LC become more detectable by the ensemble models when using multimodal data combination approaches that merge CT scan images with genomic profiles and laboratory test results [105]. XAI techniques, specifically SHAP and LIME, will serve as top priorities for the project because they help make model predictions more understandable to healthcare professionals, thus building trust between healthcare providers and the system [106]. AutoML solutions will undergo evaluation because they help automatically design model specifications and discover optimal hyperparameters, creating efficient development processes and exceptional model results [107]. The future work endeavors to convert proposed models into functional CDSSs, which will create user-friendly real-time assistance tools for physicians in early LC detection to enhance patient health results [108].