Statistical Evaluation of CESAMO Encoder for Pattern Preservation in Categorical Data

It is common to find categorical attributes in datasets used for training Machine Learning (ML) algorithms. However, most ML models are designed to exclusively handle numerical inputs. To effectively incorporate these categorical attributes, it is necessary to convert them into numerical values. Preserving the inherent patterns and information associated with the categorical attributes is essential throughout this conversion process. Any loss of information or pattern might adversely impact the performance of ML algorithms. Several encoding techniques have been proposed to handle this conversion. This paper delves into the exploration of the CESAMO encoding technique. CESAMO encoder captures the relationships between categorical attributes and other variables using what is inferred as Pattern Preserving Codes. A statistical evaluation of this encoding technique was conducted using synthetic data, comparing its performance with other encoding methods. The experimental results demonstrate that CESAMO outperforms all the other categorical encoding techniques that were compared.