Word sense disambiguation is the process of identifying a word sense based on its context. This article presents a novel approach for word sense disambiguation for Bengali text using unsupervised technique. The effect of clustering algorithms on 100 popular polysemous Bengali words are demonstrated. The features on which the clustering is applied are sentence embeddings and target word embeddings obtained from fastText and BERT model. Sentence embeddings are low dimensional real valued vector representation of a whole sentence. The novelty of the approach lies in the fact that instead of using hand crafted features the authors rely on the use of embeddings to form clusters. Sentences with similar semantics tend to be grouped into similar clusters. Hence the sentences in a cluster appear in similar context. Thus, it can be interpreted that a cluster effectively is associated with one particular meaning of a polysemous/homonymous word. Each of the cluster identifies a coarse grained sense distinction of the word based on its context. The evaluation of the clusters for a particular target word are done based on the random score. The results obtained are very promising. Out of a maximum possible adjusted random index of 1 the average value obtained for 100 words is 0.82 using a specific BERT implementation.