The Coral reef has been playing a vital role in the development of medicines required in various serious diseases such as HIV infections, heart disease etc. The role of coral reef has been found to be very significant towards providing the shelter to numerous species of marine life such as fishes, turtles, crabs etc. The automatic classification of coral reef species have been considered essential for monitoring the health of marine ecosystems, identifying the patterns in biodiversity, and implementing the conservation strategies to protect these vulnerable and diverse marine ecosystem. The classification of coral reef species can help the expert to identify the threatened as well as vulnerable coral species. This work has proposed an approach to detect coral reef species with deep learning techniques. This work has proposed an Xception based approach with some additional modification to detect the species of coral reef. In this work, the structureRSMAS dataset which comprises the underwater images of fourteen different species have been used. The proposed work performance has been compared with the other CNN models such as VGG16, VGG19, ResNet 101 and also with the state-of-the-art work. The classification performance of the proposed approach has exhibited the superior performance compared to the mentioned CNN models as well as the state-of-the-art works with highest accuracy of 88.46% achieved with the early stopping criterion and 86.54% accuracy has been achieved without the early stopping criterion.