There are no existing protocols for optical diagnosis of Sporadic colonic hamartomas, which are benign polyps, using the narrow-band imaging (NBI). Efficient detection of hamartoma polyps is difficult due to the similar appearances in NBI with other polyp types. Differentiating hamartoma from adenomatous is necessary for efficient utilization of “diagnose and leave” or “resect and discard” strategies during colonoscopy procedure. To address the above challenge, we conducted a study where suitably trained AI algorithms were employed for automatic differentiation of hamartoma and adenomatous polyps. An Attention based Multi-scale CNN (AM-Net), that integrates a Multi-scale Residual Network (MRN) with a parallel attention module (PAM) was introduced in this study. The Multi-scale Residual Network (MRN) structure enables the model to capture local multi-scale features while the attention module identifies “where to focus” and “what to focus on” through channel and spatial dimensional attention. To the best of our knowledge, AM-Net is the first AI-based model designed to differentiate colonic hamartomas from adenomatous polyps using NBI colonoscopy videos. In this study the performance of AM-Net was evaluated using a real-life colonoscopy polyp video comprising 1706 NBI polyp frames collected from 45 patients at a tertiary care hospital. The dataset includes 761 frames of hamartoma polyps and 945 frames of adenomatous polyps. The results demonstrated that efficient differentiation between hamartoma and adenomatous polyps is possible using a suitably designed and trained AI network. The proposed AM-Net achieved an accuracy of 86.97%, precision of 82.84%, F1-score of 87.75%, and AUC of 0.95, outperforming existing state-of-the-art CNN architectures and attention mechanisms across all metrics by effectively capturing structural details such as polyp mucosal patterns, textures, and boundaries, showcasing its ability to substantially enhance the accurate classification of hamartoma polyps.