Since the 1970s, Programmable Logic Controller (PLC) as a typical example of sequence control system has gradually become the mainstream technology in factory automation. However, the inflexibility and low maintainability of PLC bring a difficulty to adopt a PLC in modern factory automation with agility to handle unexpected internal and external events. Recent research shows that ontology can provide better knowledge representation and reasoning, in comparison with expert system/knowledge-based system. Ontology provides high flexibility and maintainability when it is facilitated as a controller for factory automation. Examples could be to add a new machine to replace an old machine, or to add an urgent order with customized processes to the shop floor. This research proposes a preliminary ontology controller integrating with Internet of Things (IoTs), cloud computing, and sematic model and reasoners. The controller is implemented in a laboratory pick-and-place station. The results indicate the ontology-based controller can handle unexpected events without a need of interruption to adjust/re-code the program. This research intends to show a direction for the controller of factory automation. The success of the research paves a new way to develop a further large and complicated controller for factory automation.