Living Off the Land Binaries (LOLBins) is a cyber intrusion technique in which the attacker exploits legitimate operating system binaries to carry out criminal actions. This work proposes to compare different machine learning techniques to automatically detect LOLBins attacks. The command lines were preprocessed using Natural Language Processing (NLP) algorithms. The effectiveness of the methods was evaluated taking into consideration data balancing techniques such as Random Oversampler/Undersampler and SMOTE. Performance metrics such as accuracy and F1-Score were used to compare the different techniques. In conclusion, it was verified that the Decision Tree (DT) and Random Forest (RF) algorithms were superior to the others. Furthermore, using a command window as input generated better results than evaluating isolated command lines.