QuantumNetSec: Quantum Machine Learning for Network Security

2.1 Variational Quantum Classifier (VQC)

The VQC [18] is designed to bridge the gap between quantum computing and classical data classification methods. By integrating quantum circuits with classical optimization, the VQC is capable of tackling complex classification tasks. As depicted in Figure 2, the VQC process begins with the encoding of classical data into quantum states through a feature map. The quantum gates $$$ {U}_0(x),{U}_1(x),\dots, {U}_n(x) $$$ are employed to transform the input data $$$ x $$$ into a quantum state. This step embeds the classical data into the quantum domain, leveraging the higher-dimensional space provided by quantum states.

There are diverse types of feature maps in quantum computing, each designed to encode classical data into quantum states in a way that captures the essential features of the data. Figure 3 shows an example of a feature map circuit, where a combination of Hadamard gates (H) and parameterized Pauli-Z rotations (P) are used. The Hadamard gates create superposition. Then, the parameterized Pauli-Z rotations encode the classical input data into the quantum state by rotating the Qubits' phases according to the values of the input features. This arrangement of gates helps map the classical data into a quantum state that can be effectively utilized in QML algorithms.

The core of the VQC lies in the variational quantum circuit (also known as Ansatz), as shown in Figure 4, where quantum gates are applied to the encoded states. These gates, controlled by adjustable parameters, enable the circuit to learn complex patterns within the data. The variational circuit typically consists of multiple layers of quantum gates, which may include rotation gates such as $$$ {R}_Y\left(\theta \right) $$$, where $$$ \theta $$$ is the tunable parameter that controls the rotation angle around the y-axis of the Bloch Sphere. These quantum operations, combined with entangling gates, allow the VQC to explore a vast space of possible solutions, searching for the one that best separates the data into distinct classes.

The final stage in the VQC's process is the optimization of the Ansatz, which involves fine-tuning the parameters of the quantum circuit to minimize a cost function. This is accomplished through classical optimization techniques, where a classical optimizer iteratively adjusts the parameters to achieve the best possible classification accuracy. The measurement results are then used to compute a cost function that quantifies the accuracy of the classification. This process is essential for evaluating how well the current parameter set $$$ \theta $$$ is performing in distinguishing between different classes. Finally, the measurement outcomes are fed into a classical optimizer, which adjusts the parameters $$$ \theta $$$ of the variational circuit. This iterative process continues until the cost function is minimized, and the VQC finds the optimal parameters that define the decision boundary between different classes. By combining quantum circuits with classical optimization, the VQC leverages quantum properties to potentially enhance classification performance, particularly in areas where classical methods may be limited by computational complexity or data dimensionality.

2.2 Quantum Support Vector Machine (QSVM)

The QSVM [19] is a version of the classical support vector machine (SVM) [20] algorithm, designed to leverage the properties of quantum computing for enhanced data classification. Unlike its classical counterpart, the QSVM operates by mapping input data into a high-dimensional quantum feature space, as illustrated in Figure 5. This quantum feature space allows for the effective separation of data that may not be linearly separable in its original form. The QSVM achieves this by constructing a quantum kernel, a function that measures the similarity between pairs of data points within this quantum space. This quantum kernel is generated through specific quantum transformations applied to the quantum states that represent the input data, enabling the QSVM to tackle complex classification tasks that might be challenging for classical methods.

During the training phase, the QSVM utilizes a quantum circuit to project the training data into the quantum feature space. Within this space, the algorithm computes a kernel matrix, which captures the nuanced relationships between the training examples. The next step involves identifying support vectors, critical data points that help define the optimal hyperplane for separating different classes within the quantum feature space. This process is similar to the classical SVM but occurs within the quantum domain, potentially offering significant advantages in managing data structures and improving classification outcomes.

In the testing phase, the QSVM applies the same quantum mapping process to new, unlabeled data points. These points are classified based on their position relative to the quantum hyperplane established during the training phase. The ability to accurately classify data using quantum operations makes QSVM particularly valuable for datasets that challenge traditional classification methods.

To demonstrate the functionality of QSVM, an example can be provided involving two classes of data points: one class forms a circular pattern within a two-dimensional space, while the other class occupies the surrounding region. In this scenario, a classical SVM faces challenges in achieving linear separation within the original feature space. By contrast, QSVM addresses this limitation by mapping the data into a high-dimensional quantum feature space, where the circular pattern becomes linearly separable. Through the use of a quantum kernel specifically designed for such transformations, QSVM efficiently determines the optimal hyperplane, enabling accurate classification of the data points.

The performance of the QSVM is heavily influenced by the choice of the feature map and the design of the quantum kernel. These components are essential for capturing the key characteristics of the data in the quantum feature space, thereby enabling the QSVM to achieve superior classification accuracy.

2.3 Quantum Convolutional Neural Network (QCNN)

The QCNN [21] is a quantum-enhanced version of classical convolutional neural networks (CNNs) [22], designed to harness the power of quantum computing for advanced data processing tasks. As depicted in Figure 6, QCNNs retain the fundamental architecture of classical CNNs, consisting of convolutional layers, pooling layers, and fully connected layers, but they implement these components using quantum operations. In QCNNs, the convolutional layers are created through parameterized unitary operations applied to neighboring Qubits, analogous to the filters in classical CNNs. These quantum convolutions enable the detection and extraction of quantum features, operating on the quantum states of qubits rather than on pixel values as in classical CNNs.

In classical CNNs, layers are designed to automatically learn and refine spatial hierarchies of features from input data, typically images. Convolutional layers apply filters that slide across the input, performing convolution operations that generate feature maps. These maps highlight various aspects of the input, such as edges, textures, and other patterns. The filters in classical CNNs are trained through backpropagation and gradient descent to identify specific features relevant to the task at hand. Pooling layers follow, reducing the dimensionality of these feature maps while retaining the most significant information. This process helps in creating a more abstract representation of the data as it moves deeper into the network.

QCNNs similarly incorporate quantum pooling layers, which serve a purpose akin to classical pooling but are executed within the quantum framework. In classical CNNs, pooling layers reduce the size of feature maps by selecting the most important information, typically through operations like max pooling or average pooling. In QCNNs, this dimensionality reduction is achieved by measuring a subset of qubits, effectively decreasing the number of qubits in the system. The selection of which qubits to measure is a crucial design decision in QCNNs, as it directly influences how information is condensed and propagated through the quantum network, impacting the overall performance of the model.

Training a QCNN involves optimizing the parameters of both the quantum convolutional and fully connected layers to minimize a loss function, much like in classical neural networks. However, because the operations within QCNNs are quantum in nature, traditional optimization techniques such as gradient descent must be adapted to work with quantum circuits. This often involves using classical optimization algorithms to adjust the parameters of the quantum operations based on measurement outcomes, creating a hybrid quantum-classical learning approach. The quantum-classical interplay in QCNNs opens up new possibilities for data processing, particularly in areas where quantum features can offer an advantage over classical methods, potentially leading to breakthroughs in fields such as quantum image processing and quantum pattern recognition.

An example of QCNN functionality can be demonstrated in a classification task involving quantum states representing two distinct categories. Each category is encoded into a set of qubits, with their quantum states containing patterns that distinguish one class from the other. In the quantum convolutional layer, parameterized unitary operations are applied to neighboring qubits to extract features such as correlations or entanglements specific to each category. These extracted features are processed through quantum pooling layers, where selected qubits are measured to reduce the dimensionality of the quantum system while preserving the most significant information. The reduced quantum representation is subsequently passed through additional quantum layers, culminating in a classification outcome.

2.4 Quantum K-Means (QKM)

QKM [23] is a quantum version of the classical k-means [24] algorithm, designed to improve the efficiency and effectiveness of clustering tasks by leveraging the unique properties of quantum computing. Like its classical counterpart, QKM aims to group data into distinct clusters by identifying and separating different categories based on their similarities. The fundamental mechanism of QKM, illustrated in Figure 7, involves a quantum circuit known as the swap-test circuit. This circuit plays a crucial role in determining the distance between data points and cluster centroids, a key step in the clustering process.

In the QKM algorithm, quantum states represent the centroids of clusters and the data points to be classified. For example, the quantum state $$$ \mid \varphi \Big\rangle $$$ represents the current centroid of a cluster, while $$$ \mid \Psi \Big\rangle $$$ represents a new data point whose cluster needs to be determined. To calculate the distance between these quantum states, an auxiliary qubit is introduced in the $$$ \mid 0\Big\rangle $$$ state. A Hadamard gate ($$$ H $$$) is applied to this Qubit, placing it in a superposition state. The swap-test circuit, which involves controlled-NOT (CX) gates, is then used to calculate the distance between the quantum states $$$ \mid \varphi \Big\rangle $$$ and $$$ \mid \Psi \Big\rangle $$$. Based on this distance, the algorithm assigns the data point to the nearest cluster, updating the centroids iteratively until the algorithm converges.

The QKM algorithm operates through three main subroutines: distance calculation using the swap-test circuit, cluster update, and centroid update. After each distance calculation, the algorithm updates the clusters and centroids, repeating these steps until the positions of the centroids stabilize, indicating that the clusters have been effectively formed.

Classical k-means is a well-established algorithm used to partition data into $$$ k $$$ clusters by repeatedly assigning data points to the nearest centroid and recalculating the centroids until convergence is achieved. QKM enhances this process by utilizing quantum properties such as superposition and entanglement, which allow for potentially more efficient computations. The swap-test circuit in QKM enables the algorithm to measure the similarity between quantum states, representing the distance between data points and centroids in a quantum feature space.

4.1 Quantum Transpilation

Transpilation is a fundamental process in quantum computing that involves transforming a given quantum circuit to perform on a specific quantum device. This adaptation is crucial because each quantum device has unique features and operational requirements. Instructions, such as what gate to use, that works well on one quantum backend may not be suitable for another. Figure 9 exemplifies this process by illustrating a quantum circuit before and after transpilation.

In Figure 9a, the original ZFeatureMap circuit is shown, which consists of parameterized $$$ {R}_Y $$$ and $$$ {R}_Z $$$ rotation gates applied to Qubits $$$ {q}_0,{q}_1 $$$, and $$$ {q}_2 $$$. This initial circuit is designed without consideration for a specific quantum device. Figure 9b shows part of the circuit after transpilation, where the quantum gates and their sequence have been modified to suit the architecture of the target quantum device. During transpilation, various elements such as the device's set of basic gates, the architecture of its quantum chips, and timing restrictions are considered. The primary goal of transpilation is to enhance performance on error-prone quantum hardware by modifying and refining circuits to mitigate the effects of noise and errors. These modifications align the circuit with the target device, improving the accuracy of computational results.

4.2 Circuit Optimization

In addition to adjusting circuits to meet the specific requirements of a quantum backend through transpilation, it is also necessary to address errors caused by noise. Decoherence, a significant type of noise, causes Qubits to lose information over time, particularly when they remain idle. Increased circuit depth increases this issue, leading to more substantial information loss.

Additionally, measurement errors present challenges when Qubits are measured incorrectly—for example, reading a 0 as a 1. Gate errors further contribute to noise; quantum gates are not flawless, and errors are particularly prevalent in two-qubit gates like the CX, which tend to have higher error rates than single-qubit gates. Crosstalk errors can also increase these issues, as activating a gate on one Qubit might inadvertently affect its neighbors.

Optimizing circuit transpilation can help reduce noise in quantum circuits. This can be done through a standardized six-step workflow, as illustrated in Figure 10.

As Figure 10 shows, the initial step, initialization, involves a basic check of circuit instructions and translates multiqubit gates into simpler configurations. Following initialization, the layout step assigns the circuit's virtual qubits to the physical qubits on the quantum device.

The routing phase integrates necessary SWAP gates into the quantum circuit to align with the backend's connectivity constraints. Since qubits in a quantum device are often not directly connected, SWAP gates are critical for repositioning Qubit states, enabling the execution of two-qubit gates between qubits that are not adjacent. This repositioning is essential for ensuring that all necessary interactions can occur despite physical layout limitations. The focus during this phase is on minimizing the use of SWAP gates because they introduce additional operations that are both noise-prone and computationally expensive. Therefore, the routing phase is typically optimized alongside the layout phase to efficiently manage qubit placement and connectivity.

In the translation phase, the circuit's gates are converted into the native gate set of the target backend. While quantum circuits can be designed with a variety of gate types, quantum hardware generally supports only a limited set of native gates. This translation ensures that the circuit is compatible with the specific backend but often results in increased circuit depth and a higher overall gate count. The addition of SWAP gates in the routing phase can contribute to this increased complexity, making efficient routing and translation crucial for optimizing quantum computations.

The optimization step runs a main optimization that aims to reduce the circuit's depth and gate count by combining or eliminating gates wherever possible. The success of this optimization varies; in some instances, it significantly reduces circuit complexity, while in others, the improvements might be minimal, especially on noisy quantum devices.

Finally, the Scheduling step manages the idle times between operations by strategically inserting delay instructions and optimizing the circuit's runtime on the backend in light of physical constraints and operational capabilities.

4.3 Personalized Circuit Optimization

Each step in the transpilation optimization process incrementally refines the quantum circuit, ensuring compatibility with the intended quantum hardware. While Qiskit offers four predefined transpilation levels, ranging from no optimization (level 0) to increasingly advanced optimizations (levels 1 through 3), these default settings may not fully exploit the potential of quantum hardware. Each step in the circuit optimization process presents opportunities for further refinement by utilizing nondefault configurations.

In our proposal, we implement a personalized circuit optimization approach, where we rigorously test all possible combinations of available optimization techniques for each step. This comprehensive exploration allows us to tailor the optimization process to the specific characteristics of the quantum hardware and the circuit at hand, potentially leading to better performance than standard methods. Table 2 presents the options available at each step of the circuit optimization process.

The default circuit optimization level 0 is tailored for device characterization experiments. This level does not apply any optimizations to the input circuit, only mapping it to the constraints of the target backend. It employs the TrivialLayout ⁴ method for layout and routing, where the same numbers are assigned to both physical and virtual qubits. When necessary, it incorporates SWAP operations through StochasticSwap ⁵ to ensure the circuit functions within the backend's physical layout. Moving to optimization level 1, a light optimization is introduced, enhancing the basic mapping of circuits. Initially, TrivialLayout is used to establish a basic mapping; if this requires additional SWAPs, SabreSWAP [45] finds a layout with the minimal number of SWAPs necessary. Postlayout optimization is then applied using VF2LayoutPostLayout [46] to select optimal qubits within the graph. This level also includes InverseCancellation ⁶ to refine the circuit and 1Q gate optimization to streamline single-qubit operations.

The optimization further intensifies with Levels 2 and 3. Level 2 builds upon the foundation set by Level 1 but excludes TrivialLayout, instead employing more advanced heuristic methods with increased search depth and optimization trials. It also introduces CommutativeCancellation ⁷ to reduce redundancy. Level 3, offering the highest optimization, extends these methods further with exhaustive efforts and trials, particularly in layout and routing. It also features advanced techniques such as the resynthesis of two-qubit blocks using Cartan's KAK decomposition [47] and specific optimizations like RemoveDiagonalGatesBeforeMeasure ⁸, which streamline the circuit by altering or eliminating unnecessary gates and SWAPs in preparation for measurement.

Table 2 outlines the diverse methods available for each step of the transpilation process. By exploring these options, we aim to identify the most effective combination of techniques for minimizing circuit depth, gate count, and overall noise, thereby enhancing the reliability and efficiency of quantum computations. This approach ensures that the optimization process is not only thorough but also adaptable, maximizing the utility of the quantum hardware for specific tasks.

By systematically experimenting with these options, our personalized circuit optimization approach enables us to fine-tune the quantum circuits beyond what is possible with standard optimization levels. This method not only enhances the adaptability of the circuits to different quantum devices but also helps in mitigating the impact of various noise factors, thereby contributing to more robust and accurate quantum computations.

5.1 Datasets

To evaluate the proposed system, we conducted a comprehensive case study, leveraging four prominent network security datasets: UNSW-NB15, CIC-IDS17, TONIoT, and CICIoT23. These datasets include both normal traffic data and various types of attack network traffic data and are frequently utilized for the detection and classification of attacks using ML techniques [48, 49].

The UNSW-NB15 dataset is a benchmark network security dataset created for ML testing. It consists of 100 GB of labeled network flow data, including normal data and data from nine different types of attacks. The attack categories in this dataset are as follows: Analysis, Backdoor, DoS (Denial of Service), Exploits, Fuzzers, Generic, Reconnaissance, Shellcode, and Worms. This dataset provides a comprehensive set of network activities, making it suitable for evaluating the effectiveness of IDSs.

The CIC-IDS17 dataset was constructed by simulating the behavior of 25 users over a week. Network flow data were captured during this period, resulting in a detailed dataset that includes both normal and malicious activities. The types of attacks recorded in this dataset are BoT, BruteForce, DDoS, DoS, Infiltration, PortScan, and WebAttack. The comprehensive nature of this dataset allows for the thorough testing of IDS against various attack scenarios.

The TONIoT dataset contains a variety of IoT network attacks. The attacks included in this dataset are Backdoor, DoS, DDoS, Injection, MITM (Man-in-the-Middle), Password, Ransomware, Scanning, and XSS (Cross-Site Scripting). This dataset is particularly valuable for assessing the performance of IDS in protecting IoT networks from a wide range of cyber threats.

The CICIoT23 dataset is designed to test the robustness of IDSs in IoT networks. It includes a range of IoT-specific attacks such as BruteForce, DoS, DDoS, Mirai, Reconnaissance, Spoofing, and WebAttack. All attacks are executed by malicious IoT devices targeting other IoT devices, providing an environment for testing IDSs.

These datasets provide a diverse set of scenarios and attack types, enabling a thorough evaluation of QuantumNetSec's capabilities in different network environments. The variety of attacks and the comprehensive nature of these datasets ensure that the system is tested against a wide range of potential threats, highlighting its robustness and effectiveness in real-world applications.

5.2 Quantum and Classical ML Configuration

Four classical ML methods were selected and will serve as a comparison to the quantum counterparts results: SVM, convolution neural network, RF as a representative classifier to compare with VQC, and KNNs, all implemented using the scikit-learn⁹ and TensorFlow ¹⁰ frameworks.

To evaluate the performance of QuantumNetSec, four QML models were employed: VQC, QSVM, QKM, and QCNN, which were implemented using the Qiskit framework. For these models, the specific hyperparameters used are presented in Table 3.

We employ four feature maps from Qiskit for encoding classical data into quantum states, alongside four Ansatz and classical optimizers within the Qiskit framework. The RawFeatureVector offers a direct mapping of classical data to quantum states, establishing a baseline. The PauliFeatureMap and ZFeatureMap utilize quantum gates to simulate Pauli operators and employ Hadamard and unitary gates for transforming classical inputs into quantum states, enabling the capture of nonlinear patterns and facilitating first-order data encoding. The ZZFeatureMap extends these capabilities by incorporating second-order Qubit interactions for more complex correlation encoding [50].

The RealAmplitudes ansatz circuit consists of alternating layers of $$$ Y $$$ gate rotations and $$$ CX $$$ entanglements. It is named RealAmplitudes because the quantum states have only real amplitudes; thus, the complex part is always zero. On the other hand, EfficientSU2 circuit comprises layers of single qubit operations spanned by SU(2) and $$$ CX $$$ entanglements. The SU(2) stands for the special unitary group of degree 2, involving $$$ 2\times 2 $$$ unitary matrices with determinant 1, like the Pauli rotation gates.

Another variational circuit used is the ExcitationPreserving. It is structured with layers of $$$ Z $$$ rotations and 2-qubit entanglements. We also used the TwoLocal ansatz, in which the circuit is a parameterized structure alternating between rotation layers, which apply single qubit gates to all qubits and entanglement layers that use two-qubit gates to entangle the Qubits.

For our experiments, we selected four classical optimization techniques, each with unique strengths. ADAM [51] is a gradient-based algorithm that adapts to lower-order moments, making it effective for nonstationary and noisy gradients. COBYLA [52] is used for constrained problems with unknown derivatives, iteratively refining solutions through linear approximations. SPSA [53] is well-suited for large-scale and complex systems, using a stochastic approximation of the gradient for efficient optimization. Finally, gradient descent [54] is a fundamental method that iteratively updates parameters in the direction of the negative gradient to find the function's minimum.

5.3 NISQ Devices

We evaluated the performance of QuantumNetSec across various NISQ backend configurations. Figure 11 presents the topology (physical positioning of Qubits) and noise levels per Qubit and connection for each of three of the backends used, with darker colors indicating lower noise levels. As illustrated in Figure 11, IBM Cairo has 27 Qubits with a distinct topology compared to IBM Kyoto, which has 127 Qubits, and IBM Torino with 133 Qubits.

For this evaluation, seven different backends were utilized provided by IBM quantum platform: Cairo, Kyoto, Brisbane, Osaka, Sherbrooke, Torino, and Quebec. These environments incorporate the specific noise models, quantum logic gates, and topologies of actual quantum computers, providing a realistic assessment of quantum technologies in security applications.

6.1 Circuit Optimization Impact on QuantumNetSec

In this subsection, we present the performance of various QML algorithms on different datasets under three optimization levels: no optimization, default optimization (using Qiskit's level 1, 2, 3 optimizations)¹¹, and personalized optimization. Table 4 shows the results, where we report the F1 score percentage, a metric that balances precision and recall, for each configuration. We highlight the best results obtained across different NISQ backends in the attack detection task (binary classification). The results for configurations with no optimization and using default optimization were partially derived from our previous studies, as documented in [14] and [15].

For the UNSW-NB15 dataset, the VQC exhibited a progressive increase in performance, starting with an F1 score of 88.99% with no optimization, improving to 89.15% with default optimization, and reaching 89.9% with personalized optimization. The QSVM showed similar improvements, moving from 87.76% without optimization to 87.95% with default optimization and achieving 88.0% with personalized technique. QKMs started at 86.78% with no optimization, improved to 87.22% with default optimization, and further to 87.8% with personalized optimization. The QCNN demonstrated significant enhancement, increasing from 85.15% without optimization to 87.32% with default optimization, and achieving 89.15% with personalized optimization.

In the TONIoT dataset, the VQC showed high performance across all configurations, with an F1 score of 97.78% with no optimization, slightly increasing to 97.89% with default optimization, and reaching 97.95% with personalized optimization. The QSVM's performance improved from 96.12% without optimization to 97.15% with default optimization, and further to 97.35% with personalized optimization. The QKM algorithm displayed a similar pattern, starting at 96.11% with no optimization, improving to 97.15% with default optimization, and achieving 97.35% with personalized optimization. The QCNN also exhibited strong performance, increasing from 97.4% with no optimization to 97.95% with default optimization and reaching 98.16% with personalized optimization.

For the CIC-IDS17 dataset, the VQC's performance improved from 95.78% without optimization to 96.12% with personalized optimization, while the QSVM, QKM, and QCNN also showed similar improvements. In contrast, for the CICIoT23 dataset, the VQC saw a boost from 74.53% to 79.55%, and the QSVM, QKM, and QCNN experienced substantial gains, with QSVM reaching up to 83.70% with personalized optimization. Overall, Table 4 illustrates that across all datasets and QML algorithms, personalized optimization consistently yielded the highest F1 scores. This underscores the importance of tailored optimization strategies in maximizing the performance of QML algorithms on NISQ devices. Notably, the QCNN algorithm showed the most substantial improvements with personalized optimization.

The differences in performance across optimization levels can be attributed to the distinct strategies employed at each stage. The “No Optimization” configuration serves as a baseline, where circuits are implemented without modifications, often resulting in higher error rates due to inefficient qubit usage and excessive gate depth. Default optimization, as provided by Qiskit's transpiler, reduces gate count and circuit depth through predefined optimization techniques, leading to moderate improvements in F1 scores by mitigating noise and improving execution efficiency. Personalized optimization further refines this process by tailoring circuit modifications to the specific backend and dataset characteristics, allowing for maximized fidelity and performance. This progression explains the incremental yet consistent improvements observed across all datasets and QML algorithms, as personalized optimization exploits hardware-specific nuances and dataset requirements to achieve superior results.

6.2 Results Across Different NISQ Backends

Table 5 summarizes the performance of various QML algorithms across different datasets and NISQ backends, presenting F1 scores to highlight each algorithm's effectiveness under different conditions. Similar to Table 4, these results refer to the attack detection task (binary classification). However, this analysis exclusively considers outcomes achieved using personalized optimization. This focus allows for a clear assessment of how the personalized optimization strategy enhance algorithm performance on specific backends.

The performance of QML algorithms varies significantly across different NISQ backends for the datasets tested. For the UNSW-NB15 dataset, VQC performs best on the Kyoto backend with an F1 score of 89.9%, while QSVM shows strong results on the Osaka backend at 87.95%. QKM achieves consistent performance, peaking at 87.8% on the Osaka backend, and QCNN scores highest on the Kyoto backend with 89.15%. In the CIC-IDS17 dataset, VQC excels on both the Kyoto and Torino backends with an F1 score of 96.0%, while QSVM achieves 95.6% on the Brisbane backend. QKM performs best on the Quebec backend with 93.95%, and QCNN reaches 97.15% on both Torino and Quebec backends.

For the TONIoT dataset, VQC achieves the highest F1 score of 97.95% on the Sherbrooke backend, QSVM performs best on Kyoto with 97.35%, and QCNN demonstrates the highest efficiency on Osaka with 98.16%. For the CICIoT23 dataset, VQC reaches 79.55% on Brisbane, QSVM excels with 83.70% on Brisbane and Quebec, QKM achieves 82.99% on Sherbrooke and Torino, and QCNN stands out with 84.55% on Torino.

Overall, the results indicate significant variation in performance across different NISQ backends, reflecting the impact of each system's unique noise model, quantum logic gates, and topology. This underscores the importance of selecting and tuning QML models according to the specific characteristics of the quantum hardware in use. Notably, QCNN stands out with the highest F1 scores in several configurations, showcasing its effectiveness in handling the complexities of real quantum systems. The varying results emphasize the necessity for ongoing research and development in quantum computing to address the challenges posed by noise and other physical limitations in NISQ systems. The best results obtained in each model and backend combination will be discussed in detail in the following subsections.

6.3 QuantumNetSec: Attack Detection Results

In this subsection, we compare the performance of QML algorithms with classical ML algorithms in the context of binary classification for attack detection. The results, presented in Table 6 show the F1 scores for both QML and classical ML algorithms.

For the UNSW-NB15 dataset, the QCNN achieved the highest F1 score of 88.9%, outperforming classical ML algorithms. Among the classical ML models, CNN had the highest score of 86.72%. The VQC also performed well with an F1 score of 89.9%, surpassing the best classical result. In the CIC-IDS17 dataset, QCNN again led the performance with an F1 score of 97.15%, significantly higher than the top-performing classical model, SVM, which scored 93.78%. The VQC also demonstrated strong performance with an F1 score of 96.0%, highlighting the advantages of QML in this dataset.

For the TONIoT dataset, the QCNN achieved an high F1 score of 98.16%, outclassing all classical ML algorithms. The highest score among classical models was achieved by RF with an F1 score of 96.6%. The QSVM and QKM also performed exceptionally well, each scoring 97.35%. In the CICIoT23 dataset, QCNN again showed superior performance with an F1 score of 84.55%, compared to the best classical model, SVM, which scored 82.70%. The QSVM and QKM also performed well, with F1 scores of 83.7% and 82.99%, respectively, surpassing most classical algorithms. Overall, the results in Table 6 highlight that QML models generally outperform classical ML models across all tested datasets. The superior performance of QML, particularly the QCNN, underscores the potential of quantum computing to enhance attack detection capabilities in cybersecurity applications.

6.4 QuantumNetSec: Identification of Attacks

In this section, we present the multiclass F1 score results of various QML and classical ML algorithms for identifying different types of attacks (multiclass classification). The results are detailed in Tables 7–10, which correspond to the attack data from the UNSW-NB15, CIC-IDS17, TONIoT, and CICIoT23 datasets, respectively. The results highlight the performance of each algorithm in detecting specific attack categories.

For the UNSW-NB15 dataset, the QCNN consistently outperformed other models, achieving the highest F1 scores across multiple attack categories. Notably, QCNN scored 99.5% in both analysis and generic attack categories, indicating its superior capability in handling these types of attacks. The VQC also showed strong performance, particularly in the generic category with an F1 score of 97.05%. In contrast, classical models like SVM and RF performed well in certain categories but generally fell short of the top QML models.

In the CIC-IDS17 dataset, QCNN again demonstrated exceptional performance, particularly in the BoT and BruteForce attack categories, where it achieved high F1 scores of 99.54% and 100%, respectively. QSVM also showed strong results, with a high score in the BruteForce and Infiltration categories. These results underscore the effectiveness of QML models, especially QCNN, in identifying a wide range of attack types with high accuracy compared to their classical counterparts.

For the TONIoT dataset, QCNN maintained its leading position with high F1 scores across most attack categories. It achieved a high score in the MITM, DoS, and Password attacks, highlighting its robustness and reliability. VQC also performed well, particularly in the DoS category with an F1 score of 98.62%. Classical models like SVM and RF performed adequately, but again, QML models demonstrated superior performance in most categories.

In the CICIoT23 dataset, QCNN continued to excel, achieving the highest F1 scores in categories such as Mirai (98.36%) and DoS (97.55%). QSVM also performed strongly in several categories, including DDoS and DoS, with scores of 97.0% and 96.77%, respectively. While classical models showed competitive performance in some categories, the QML models, particularly QCNN, consistently outperformed them, demonstrating their potential for more effective attack detection in complex IoT environments.

6.5 Results Discussion

The results from both binary and multiclass classification experiments demonstrate the significant potential of QuantumNetSec. In binary classification, the QML models consistently outperformed their classical counterparts across various datasets. QCNN emerged as the most effective model, achieving the highest F1 scores in almost all scenarios. This suggests that QCNN's ability to leverage quantum properties such as superposition and entanglement provides a tangible advantage in accurately distinguishing between attack and normal data.

In the multiclass classification results, the superiority of QML models was further confirmed. QCNN maintained its leading performance, particularly excelling in complex attack categories across all datasets. For instance, in the UNSW-NB15 dataset, QCNN achieved high scores in categories such as Analysis and Generic attacks. Similarly, in the CIC-IDS17 and TONIoT datasets, QCNN consistently delivered the highest F1 scores, highlighting its robustness and versatility. These findings underline the potential of QML models to handle a diverse range of attack types with high precision.

The comparison of QML models with classical ML models in both binary and multiclass classifications underscores the impact of quantum computing in cybersecurity applications. While classical models like SVM and RF showed competitive performance in certain categories, they generally lagged behind the QML models. The enhanced performance of QML models demonstrates their ability to address the limitations of classical approaches. This is crucial for improving the overall reliability and effectiveness of IDSs.

Moreover, the results highlight the importance of optimization techniques tailored to NISQ devices. The personalized optimization strategy employed in our experiments significantly boosted the performance of QML models. This indicates that, as quantum hardware continues to evolve, further improvements in optimization techniques could lead to even greater enhancements in the efficacy of QML-based cybersecurity solutions. These findings suggest a promising future for the integration of quantum computing in network security, paving the way for more advanced and resilient IDSs.

7.1 Key Challenges

One of the primary challenges faced by QuantumNetSec lies in the limitations imposed by current quantum hardware. The constraints of NISQ devices, including noise, limited qubit counts, and restricted connectivity, present significant hurdles to achieving consistent scalability and reliability. These hardware limitations underscore the need for innovative, hardware-aware quantum algorithm development, and the incorporation of robust error mitigation techniques to bridge the gap between theoretical capabilities and practical deployment. Another notable challenge is the system's performance on complex IoT scenarios, as evidenced by the results on the CICIoT23 dataset. Although the implementation of personalized optimization techniques led to improved outcomes compared to baseline approaches, the performance still fell short of expectations. This highlights the difficulty in managing highly heterogeneous and intricate datasets typical of IoT environments and calls for further research into tailored feature extraction techniques and advanced quantum circuit designs to better address these complexities. These improvements could enable QuantumNetSec to handle such scenarios with greater accuracy and reliability.

Moreover, implementing QuantumNetSec in real-world scenarios introduces additional practical challenges. The constraints of quantum hardware, including the need for consistent operation under noisy conditions, limited access to high-performing quantum devices, and the costs associated with quantum computation, can hinder widespread adoption. Access to quantum devices is currently limited, often requiring significant investment or partnerships with providers, which may be prohibitive for many organizations. However, it is important to note that the field of quantum computing is advancing rapidly, with significant improvements in hardware capabilities and cost reductions expected in the near future. These developments provide optimism that the barriers to practical implementation will diminish, making systems like QuantumNetSec more accessible and viable for real-world applications.

7.2 Future Work

Building upon these challenges, several directions for future research can be proposed to enhance the capabilities of QuantumNetSec. The adaptability of the model to handle unknown attacks and emerging datasets is a critical area for further investigation. Integrating unsupervised learning approaches, such as quantum anomaly detection, could help identify patterns associated with previously unseen threats. Moreover, techniques like transfer learning and incremental learning should be explored to enable the model to adapt to new datasets without requiring complete retraining.

To improve performance on complex IoT scenarios, future work could refine feature extraction pipelines and develop advanced quantum circuit architectures. Hybrid approaches that combine quantum and classical preprocessing steps might better capture the intricate nature of IoT traffic. Additionally, incorporating domain-specific insights into the design of quantum feature maps and ansätze could enhance the system's capability to handle heterogeneous data.

Ensuring scalability as quantum hardware evolves will also be critical. Research should focus on optimizing quantum circuits to reduce computational overhead while maintaining or improving detection accuracy. Leveraging advancements in quantum hardware, such as error-corrected qubits and advanced transpilation techniques, will be essential in enhancing the system's applicability to large-scale, high-traffic network environments.

Finally, addressing the robustness of QuantumNetSec against adversarial attacks will be crucial for its long-term viability. Investigating the vulnerabilities of quantum-enhanced systems and developing quantum-specific defense mechanisms will be key. Techniques such as robust quantum circuit designs, adversarial training, and quantum cryptographic methods could be integrated to strengthen the system's defenses against sophisticated threats.

In conclusion, QuantumNetSec represents a significant advancement in leveraging QML for cybersecurity. By addressing these challenges and exploring these future directions, it has the potential to become a cornerstone of quantum-enhanced IDSs, paving the way for more secure and resilient digital infrastructures.