An important aspect of temporal data analysis is time-series anomaly detection. It has applications in e-commerce, healthcare, transportation, and manufacturing sectors. Unforeseen situations and potential issues can be detected with the insights gained from analyzing anomalies. Anomaly detection in time series is especially challenging because of the scarcity of labelled anomalies in the dataset. Current methods are highly domain-specific, and lack standardization of techniques. We evaluate the suitability of statistical and neural models for this task. We compare their accuracy at learning the data patterns and anomalies, on the M4 competition dataset, which is not domain-specific. Our study has found that the neural network-based Deep LSTM, and NBEATSx models have shown the best forecasts of data, and indicated anomalies in the univariate analysis scope. These models have outperformed machine learning-based models like Prophet, ARIMA, and Isolation Forest, transformer-based MOMENT model, and the large language model-based Lag-LLaMA.