The incorporation of Internet of Things (IoT) devices and robotics into smart building environments has introduced new opportunities for human activity recognition (HAR). The paper introduces an innovative approach that combines IoT sensors and robotic data to create a complete dataset for Human Activity Recognition (HAR) in smart building settings. The system utilises a synchronised data collection technique to gather information from fixed IoT sensors, wearables, and smart devices, as well as from mobile robots. The dataset obtained includes a wide variety of sensor modalities, such as accelerometers, gyroscopes, microphones, motion sensors, smart plugs, and cameras (thermal, RGB-D). A proposed strategy is presented to tackle the difficulties of multi-modal data fusion and classification by fusing all sensor data into a single machine learning model. This study lays the groundwork for future works into Human Activity Recognition (HAR) systems, which involve the integration of multi-sensor settings, sophisticated data fusion algorithms, context-sensitive identification, and tailored support for individuals working or residing in smart environments. The setup of the sensors and use cases of this study were conducted in a smart home environment. Nevertheless, the integrated system can be tailored to suit the requirements of any smart building.