The effect of single-walled carbon nanotubes (SWCNTs) on the crosslinking network structure of Ethylene Propylene Diene Monomers (EPDM)—Polyamide elastomers (PAE) is investigated in detail using Hansen Solubility Parameter (HSP) theory. Rheological method and scanning electron microscopy (SEM) are used to investigate the effect of SWCNTs on the molecular chain movement and their interaction with the filler network of EPDM-PAE. The DIN abrasion test is conducted to obtain the effect of SWCNTs on the wear characteristics of EPDM-PAE/SWCNTs composites. The linear synergistic relationship between crosslinking density and wear volume indicates that the change in wear properties is contributed by the micro-network structure. The addition of SWCNTs increases the chemical crosslinking density and improves the wear properties of EPDM-PAE. However, the decrease in the physical crosslinking density leads to an increase in the degree of internal friction within the network structure, resulting in a decrease in the wear performance. Changes in wear resistance can be related to the micro-network structure, which provides a novel approach to studying the wear characteristics of high-performance elastomers. It is also an innovative work based on the network structure to guide the preparation of new wear-resistant elastomeric materials.