Automatic accurate skin lesion segmentation systems are very helpful for timely diagnosis and treatment of skin cancers. Recently, methods based on convolutional neural networks (CNN) have presented powerful performances and good results in biomedical applications. In the proposed method, a novel structure based on Mask RCNN, a proposed CNN, and a geodesic segmentation method is presented to improve the performance of the skin lesion segmentation. Lesions are detected and segmented by the Mask R-CNN in the first stage. A multi-atrous full convolutional neural network (MAFCNN) is proposed to combine outputs of the Mask RCNN and the input image to present more accurate segmentation results. To modify boundary of the lesion segmented by the MAFCNN, a geodesic segmentation method is used. Some parts of the segmentation result of the proposed CNN are utilized as labeled pixels for the geodesic method. Results demonstrate that using the proposed MAFCNN in a novel structure followed by the geodesic method significantly improves the mean Jaccard value. Experiments on five well-known skin image datasets show that the proposed method outperforms other state-of-the-art methods.