Harnessing Key Phrases in Constructing a Concept-Based Semantic Representation of Text Using Clustering Techniques

This paper introduces a modified approach for representing text as semantic vectors, building upon the Bag of Weighted Concepts (BoWC) method developed in previous research. The limitations of the BoWC method are addressed, and a proposed solution is presented. Instead of using unigrams, the authors propose extracting key phrases that best represent each document to generate high-quality concepts and reduce concept overlap. These unique key phrases are then used to construct the concept dictionary. Document vectors are created by mapping document key phrases to the concept dictionary using a modified concept weighting function that considers the weight of the key phrase within the document. To evaluate the effectiveness of the resulting vectors, they were employed in a clustering task and compared against robust baselines. Experimental studies demonstrate that the proposed modifications enhance the quality of document vector representation, as evidenced by a minimum 4% increase in clustering accuracy based on the V1 metric.