Cognitive load assessment is important for optimizing task performance by designing personalized instructional materials. Entropy is a critical measure in analyzing cognitive load as it quantifies the complexity, irregularity and unpredictability of patterns present in the data. In this study, we have used EEG (electroencephalography) signals to measure the cognitive load during mental tasks and rest. For the purpose of implementation, an open-access dataset is used, where EEG brain signals are collected from 29 healthy subjects while performing mental tasks and rest. The four significant EEG frequency bands, alpha, beta, theta and delta, are extracted using bandpass filters. As significant features, three different entropy, measures, Shannon, Wavelet and Maximum likelihood, are computed for all four bands. To identify the most significant entropy measure considering four bands, three classifiers, support vector machine (SVM), k-nearest neighbor (KNN) and linear discriminate analysis (LDA) are used to distinguish the mental task and rest. The results show that the Maximum likelihood entropy of the alpha band plays a significant contribution in classifying mental tasks and rest. Entropy measures in cognitive load analysis using EEG provide a robust, sensitive, and insightful approach to understand brain dynamics and their relationship with cognitive processes.