1 Introduction

As global urbanization accelerates, the challenges associated with traffic congestion and road safety have become more pressing. Despite advances in road safety measures, traffic accidents remain a critical issue worldwide, with recent data from the World Health Organization (WHO) indicating that road traffic crashes result in approximately 1.19 million deaths annually [1]. These crashes are the leading cause of death for individuals aged 5–29 years, with vulnerable road users—such as pedestrians, cyclists, and motorcyclists—accounting for over 50% of the fatalities. The burden is disproportionately high in low- and middle-income countries, which experience 90% of global road deaths despite having fewer vehicles overall [1]. This underscores the urgent need for smarter traffic management systems and advanced vehicle detection technologies [2]. In this research, we present a novel approach to vehicle classification designed to address these road safety challenges. Our model classifies vehicles into seven distinct categories—Bus, Car, Truck, Three-wheeler, Two-wheeler, Special Vehicles, and Van or Mini-Truck—using a self-annotated custom dataset [3] with images in different traffic conditions and timings. This classification system is a critical step towards developing intelligent traffic management solutions capable of optimizing road usage and improving safety. To enhance the accuracy of the model, we leveraged the pre-trained weights of YOLOv8, a state-of-the-art deep learning model renowned for its speed and precision in object detection. By applying transfer-learning techniques, we significantly improved the performance of the model, enabling more accurate vehicle classification even with limited dataset sizes. The detection and classification of the vehicles can be seen in Figure 1.

2 Literature Review

A literature survey was conducted to review the existing works in the field. The following summary presents the key findings from chapters related to the proposed research.

Lv et al. proposed an ensemble approach combining two state-of-the-art deep learning models, EfficientDet and YOLOv8, to address challenges associated with diverse vehicle characteristics and varying lighting conditions. The study utilized both RGB and thermal images from the FLIR dataset to enhance detection accuracy, particularly in low-light scenarios where traditional methods struggle. By incorporating extensive data augmentation techniques to address class imbalances, the ensemble model was trained to generalize effectively across different vehicle types and environments. The experimental results demonstrated that the ensemble method significantly outperformed the individual models, achieving a mean average precision (mAP) of 95.5% on thermal images and 93.1% on RGB images, with an average recall (AR) of 0.93 for thermal images. The integration of EfficientDet's BiFPN for multi-scale detection and YOLOv8's speed and real-time accuracy resulted in a robust detection system. The use of majority voting to combine model predictions further enhanced the classification performance. This study h ighlights the potential of ensemble learning techniques to overcome the limitations of individual deep learning models, making the system suitable for real-world applications, particularly in intelligent traffic management systems [7].

Marathe et al. introduced WEDGE, a multi-weather autonomous driving dataset generated using vision-language models such as DALL-E. The dataset contains 3360 images across 16 extreme weather conditions, manually annotated with over 16,500 bounding boxes for tasks like weather classification and 2D object detection. The motivation behind WEDGE is to enhance the robustness of autonomous vehicle perception systems, particularly in adverse weather conditions, which are often underrepresented in traditional datasets. The dataset showed its utility in improving state-of-the-art (SOTA) performance in real-world weather benchmarks such as DAWN, with significant improvements in detecting classes like trucks. By fine-tuning object detectors on WEDGE, the researchers achieved an increase in average precision by 4.48 points for trucks. The study emphasizes the potential of synthetic datasets to bridge gaps in sim2real transfer, improving detection in underrepresented weather conditions for autonomous driving tasks [8].

Dong et al. present an enhanced approach to vehicle classification, leveraging advancements in deep learning and vision transformers. Traditional methods, particularly convolutional neural networks (CNNs), were effective at capturing local features but struggled with global feature representation. The paper proposes a novel model, IND-ViT, which integrates CNNs to address local feature loss in vision transformers. This hybrid architecture, combining a CNN-In D module and a sparse attention module, improves the model's ability to detect subtle distinctions between similar vehicles and enhances overall feature extraction. Through the use of a contrast loss function, the model increases intra-class consistency and inter-class distinction. Experimental results show that IND-ViT outperforms traditional ViT and CNN-based models on datasets like BIT-Vehicles, CIFAR-10, and Oxford Flower-102, with improvements in classification accuracy of up to 7.54% for certain datasets. The method strikes a balance between accuracy and real-time performance, showcasing its potential for applications in autonomous driving systems [9].

Trivedi et al. present an advanced approach to vehicle classification using computer vision techniques. The study highlights the significance of vehicle classification in smart transportation systems, emphasizing its role in traffic management and automated toll collection. The authors utilize Speeded-Up Robust Features (SURF) for feature extraction and employ multiple classifiers, including Support Vector Machines (SVM), Decision Trees, and K-Nearest Neighbors (KNN), to categorize vehicles into three classes: bikes, cars, and trucks. Their methodology is validated on a dataset of approximately 11,000 images, achieving a high classification accuracy of 91% using a Medium Gaussian SVM classifier. The research underscores the challenges posed by varying lighting conditions, image resolutions, and real-time constraints, offering insights into feature extraction techniques and classifier performance. Additionally, the paper discusses the limitations of traditional classification approaches and suggests potential improvements using deep learning-based architectures for enhanced accuracy and real-time application in Intelligent Transportation Systems (ITS) [10].

Wang et al. present a deep learning-based approach to vehicle classification for intelligent transportation systems. The study leverages Faster R-CNN for vehicle detection and classification, outperforming traditional machine learning techniques. The proposed system effectively classifies vehicles into cars, trucks, minivans, and buses with a mean average precision (mAP) of 81.05% and achieves over 90% accuracy for cars and trucks. The model is trained on a dataset of over 60,000 labeled images and optimized for real-time performance, operating at 0.354 s per image on an NVIDIA Jetson TK1 embedded system. The authors highlight challenges such as varying illumination, image scales, and angles, demonstrating that convolutional neural networks (CNNs) provide robust feature extraction and classification capabilities. The research concludes that deep learning significantly enhances vehicle classification accuracy and efficiency, paving the way for real-time intelligent traffic monitoring and autonomous driving applications [11].

Osman et al. present a deep learning-based approach to real-time vehicle detection and classification in traffic surveillance systems. The study utilizes the YOLOv8 (You Only Look Once) object detection algorithm, a convolutional neural network (CNN)-based model, to classify vehicles into seven categories: car, bike, SUV, van, bus, truck, and person. The authors trained the model on real traffic video data collected from 10 intersections, using bounding box annotations for precise vehicle detection. The proposed method achieves a classification accuracy of 91%, outperforming traditional fixed-duration traffic signalization systems by dynamically adjusting green light durations based on real-time vehicle density and class distribution. The study highlights improvements in reducing traffic congestion, fuel consumption, air pollution, and driver stress through adaptive traffic signal control. The results demonstrate that deep learning techniques, particularly YOLOv8, significantly enhance vehicle recognition and smart traffic management systems, making them more efficient and responsive to real-world conditions [12].

3 Vehicle Classification System

Our vehicle detection and classification model is based on YOLOv8, a state-of-the-art object detection framework known for its real-time processing capabilities and accuracy. We have designed the model to identify vehicles across seven key classes that contribute differently to traffic congestion:

4 Self-Created Dataset

The dataset [3] for real-time vehicle classification was carefully curated and structured into train (800 images), test (215 images), and validation (311 images) sets, ensuring a balanced evaluation of the model's performance as shown in Table 1. To enhance data quality and consistency, auto-orientation of pixel data was applied by stripping EXIF metadata, and all images were resized to 640 × 640 pixels using a stretching transformation. To improve model robustness and generalization, extensive data augmentation was performed as mentioned in Table 2, generating three augmented versions of each image (800 × 3 = 2400 images for train set). Augmentations included a 50% probability of horizontal flipping, random rotation between −15° and +15°, random shear transformations between −10° and +10° both horizontally and vertically, random exposure adjustments within a −10% to +10% range, and the introduction of salt and pepper noise to 0.1% of pixels to simulate real-world variations and environmental noise. These preprocessing and augmentation strategies contribute to a diverse and resilient dataset, enhancing the model's ability to handle different lighting conditions, orientations, occlusions, and image distortions, ultimately improving its robustness in real-time traffic applications.

5 Used Methodology

In many low- and middle-income countries, despite advancements in hardware such as better cameras and sensors, traffic management continues to face challenges due to limited resources. This research addresses these challenges by leveraging advanced convolutional neural network (CNN) models, specifically YOLOv8, to classify vehicles into seven distinct categories. To optimize the model for real-time performance in low-resource environments, transfer learning [13] was employed, utilizing pre-trained weights from YOLOv8. This paper details the entire workflow, from dataset creation to model training and evaluation, highlighting the potential of the model for integration into smart traffic systems aimed at reducing accidents and improving traffic flow, as demonstrated in Figure 2.

5.3 Data Annotation

Each frame was manually annotated in the YOLOv8 format, as shown in Figure 3. The classification system was based on seven distinct vehicle categories: bus, two-wheeler, car, truck, van or mini-truck, three-wheeler, and special vehicles. These categories were designed to encompass nearly all common road vehicle types. The meticulous annotation process ensured high-quality, labeled data, providing the model with the necessary accuracy for real-time vehicle detection and classification.

The annotation count of the different classes of vehicles is given in Table 3.

TABLE 3. Annotations count of vehicle classes.

Class	Annotation count
Car	2002
Two-wheeler	996
Bus	484
Van or mini-truck	409
Three-wheeler	290
Truck	219
Special vehicle	161
Total	4561

6 Model Implementation and Fine-Tuning

This section discusses the specifics of the YOLOv8 model, its architecture, the fine-tuning process, and detailed augmentation techniques applied during training.

6.1 YOLOv8

YOLOv8 builds upon the advancements of its predecessors in the YOLO family [17], incorporating state-of-the-art developments in neural network architecture and training strategies. Like earlier versions, YOLOv8 integrates object localization and classification within a unified, end-to-end differentiable framework, effectively balancing speed and accuracy. The model's architecture is structured around three critical components, as shown in Figure 4.

6.1.1 Backbone

YOLOv8 utilizes an advanced convolutional neural network (CNN) backbone designed to capture multi-scale features from input images [18, 19]. This backbone, potentially an enhanced version of CSPDarknet or another efficient architecture, extracts hierarchical feature maps that encompass both low-level textures and high-level semantic information. These feature maps are vital for accurate object detection. Optimized for computational efficiency and accuracy, the backbone incorporates depthwise separable convolutions or similar efficient layers, reducing computational costs while preserving high representational power.

6.1.2 Neck

The neck module is responsible for refining and fusing the multi-scale features obtained from the backbone. It builds on an optimized Path Aggregation Network (PANet) [19], designed to enhance information flow across multiple feature levels. This multi-scale integration is crucial for detecting objects of various sizes, and YOLOv8 likely incorporates improvements to the original PANet to reduce memory consumption and increase computational efficiency.

6.1.3 Head

The head module generates the final predictions—bounding box coordinates, object confidence scores, and class labels—from the processed features. One notable innovation in YOLOv8 is the shift to an anchor-free approach [20] for bounding box prediction, moving away from the anchor-based methods employed in earlier versions. This anchor-free design simplifies the prediction process, reduces hyperparameter dependency, and enhances adaptability to objects with varying aspect ratios and scales [20]. By synthesizing these architectural improvements, YOLOv8 demonstrates superior performance in object detection tasks, offering advancements in accuracy, speed, and model flexibility [21]. These enhancements mark a significant step forward in object detection methodologies, enabling more efficient and precise models for practical applications.

6.1.4 Improvements in YOLOv8

• Anchor-Free Detection: YOLOv8 moves toward anchor-free detection, reducing the complexity of hyperparameters related to anchor boxes. This allows for simpler and faster training and inference.
• Network Efficiency: YOLOv8 incorporates advancements in network design, such as better feature fusion and optimized architecture, to improve both speed and accuracy.
• Model Flexibility: YOLOv8 is designed to be versatile, supporting tasks beyond object detection, such as instance segmentation and keypoint detection.
• Better Data Augmentation: The model leverages modern augmentation techniques, improving robustness against overfitting and enhancing performance on diverse datasets [18].
• Loss Function Enhancements: YOLOv8 utilizes newer loss functions and optimizations that enhance object localization and classification accuracy, particularly for smaller objects.

7 Results and Performance Metrics

This section provides a comprehensive evaluation of the vehicle classification model's performance, including key metrics such as precision, recall, and mean average precision (mAP) across different vehicle classes. This section highlights the model's robustness, demonstrating its ability to accurately detect and classify vehicles in real-time scenarios. The performance of each class—Bus, Car, Truck, Van or Mini-Truck, Two-Wheeler, Three-Wheeler, and Special Vehicles, and model performance is visualized through loss curves and precision-recall metrics. Table 5 shows the performance metrics of the model across various classes of vehicles. The model took 1.696 h for 100 epochs to run.

Few of the model's predictions are shown in the Figure 5, showcasing its effectiveness in real-world applications.

7.5 PR Curve

The PR (Precision-Recall) curve illustrates the trade-off between precision and recall for the classification model [24]. Precision measures the accuracy of positive predictions, while recall indicates the proportion of actual positives correctly identified. The curve is especially useful for imbalanced datasets. High precision with low recall means the model is conservative, making fewer but more accurate positive predictions, while high recall with low precision indicates more positive instances are identified but with more false positives. The area under the PR curve (AUC-PR) reflects overall model performance, with a larger area indicating a better ability to distinguish between classes. The PR curve for the developed model can be seen in Figure 6.

7.6 Confusion Matrix

It is a performance evaluation matrix used to visualize the outcomes of a classification model [25-28]. It is particularly helpful in understanding how well a model distinguishes between classes, especially in multi-class classification tasks [29]. The matrix is structured as a square grid, with each row representing the actual class labels and each column representing the predicted class labels. The four key components of a confusion matrix are:

• True Positives (TP): Instances correctly predicted as the positive class.
• True Negatives (TN): Instances correctly predicted as the negative class.
• False Positives (FP): Instances incorrectly predicted as positive, also known as Type I errors.
• False Negatives (FN): Instances incorrectly predicted as negative, also known as Type II errors.

By examining these values, the confusion matrix provides insights into different metrics, such as accuracy, precision, recall, and F1 score. It allows for a detailed breakdown of the model's performance, showing not only how many predictions are correct but also where the model makes mistakes, whether it confuses certain classes, and how balanced the predictions are across categories. This is particularly useful when dealing with imbalanced datasets, where a high overall accuracy might be misleading. You can see the confusion matrix for the model in Figure 7.

7.7 F1-Confidence Curve

The F1-Confidence Curve is a graphical representation that depicts the relationship between the F1 score and various confidence thresholds for a multi-class vehicle detection model. The F1 score, which balances precision and recall, is particularly useful in assessing model performance across different vehicle categories. As the confidence threshold varies, the F1 score fluctuates, reflecting the model's effectiveness in accurately detecting and classifying vehicles. A higher F1 score indicates a better balance between precision (true positive predictions) and recall (actual positive instances), essential for effective vehicle classification. In the graph as shown in Figure 8, the blue curve shows the average F1 score across all classes, peaking at 0.83 when the confidence score is 0.407. This optimal threshold enhances the model's practical utility in real-time vehicle detection systems, providing insights into the best confidence levels for deployment.

Now in Figure 9 there are some other metrics and graphs for the better understanding of the overall model and its losses.

7.8 Loss Curves Analysis

7.8.1 Box Loss Learning Curve

It tracks the model's ability to accurately predict bounding boxes for detected vehicles, as shown in Figure 10. Both the training and validation losses start high, rapidly decreasing as the model learns, and eventually stabilize around 1.00 after 100 epochs. This shows that the model has learned to predict bounding boxes effectively, with a consistent performance between training and validation sets, indicating minimal overfitting.

7.8.2 Classification Loss Learning Curve

It shows the model's performance in classifying vehicle types as shown in Figure 11. The loss decreases sharply in the initial epochs, with both training and validation losses converging towards 1.00. This indicates that the model is able to distinguish between different vehicle classes effectively, and the closeness of the training and validation curves suggests good generalization across the dataset.

7.8.3 Distribution Focal Loss Learning Curve

It addresses class imbalance, helping the model focus on harder-to-classify instances as shown in Figure 12. The training loss stabilizes near 1.00, while the validation loss remains slightly higher, suggesting that the model effectively handles imbalanced vehicle classes without major overfitting, although there may be room for improvement in dealing with certain complex cases.

In all three graphs, the training and validation losses remain closely aligned, highlighting the model's robustness and efficiency in predicting bounding boxes and classifying vehicles while avoiding overfitting.

8 Conclusion and Discussion

This research introduces a real-time vehicle classification model utilizing the YOLOv8 framework, built on a self-annotated custom dataset comprising seven vehicle categories: Bus, Car, Truck, Van or Mini-Truck, Two-Wheeler, Three-Wheeler, and Special Vehicles. The model effectively addresses the complexities of traffic management in resource-constrained environments, achieving high precision and recall through transfer learning and efficient computational performance. The comprehensive workflow of data collection, annotation, preprocessing, and augmentation highlights the rigorous methodology that underpins the model's robustness. Its successful integration into intelligent traffic management systems offers improved traffic flow and enhanced road safety, especially in low- and middle-income countries with limited infrastructure.

9 Limitations and Future Scopes

Despite achieving high classification accuracy, the proposed real-time vehicle classification system has certain limitations. First, the dataset, although diverse, may not fully represent extreme environmental conditions such as heavy fog, nighttime glare, or severe weather, which could impact model performance. Second, the resizing of images to 640 × 640 pixels using a stretch transformation may introduce distortions, potentially affecting object shape integrity. Additionally, while YOLOv8 performs well in real-time detection, its computational requirements may pose challenges for low-power embedded devices, limiting its deployment in resource-constrained environments. The salt and pepper noise applied to only 0.1% of pixels may not fully simulate real-world noise levels caused by motion blur, occlusions, or sensor defects. Moreover, the augmentation strategy does not include adversarial perturbations, which could make the model vulnerable to minor adversarial changes. Future work can address these limitations by incorporating domain adaptation techniques, higher-resolution datasets, and optimizing inference for edge devices.

Author Contributions

Tejasva Maurya: conceptualization, methodology, investigation, writing – original draft, writing – review and editing. Saurabh Kumar: conceptualization, methodology, visualization, validation, writing – original draft, writing – review and editing. Mritunjay Rai: conceptualization, validation, writing – original draft, writing – review and editing. Abhishek Kumar Saxena: conceptualization, methodology, investigation, validation, project administration, resources, writing – original draft, writing – review and editing. Neha Goel: conceptualization, validation, project administration, supervision, writing – original draft, writing – review and editing. Gunjan Gupta: conceptualization, writing – original draft, supervision, project administration, resources, validation, methodology.

Conflicts of Interest

The authors declare no conflicts of interest.