Cattle detection and stock monitoring in open fields continue to pose a challenge in smart agriculture due to the low efficiency of acquiring the data and expensive human labour. With the advancement of computer vision technology, it is now possible to apply multi-modal data analysis to address the issues and implement the image based cattle monitoring applications. In this study, we take advantage of baseline object detection methods and transfer learning to detect a large herd in open fields. We also built a cattle dataset with annotated thermal images captured from a drone flying about 30 m above the ground. We use machine learning regression models to estimate the live weight of the cattle based on several manual measurements as well as image based segmentation instances. Meanwhile, the cattle detection and weight estimation have been integrated into the image based cattle monitoring system.