Co-creative computer agents can be used to support designers they work alongside. However, this approach to co-creativity has not been studied in haptics. In this paper, we investigate this topic by applying an interactive reinforcement learning technique developed for sound synthesis to a two degree-of-freedom force-feedback design space. We developed a prototype authoring tool to study the interaction between designers and this type of co-creative agent. As these agents are trained on the designer’s feedback in real-time, this method is flexible and could be applied to other hardware and software configurations. Five participants were recruited to use the tool in two artistic tasks and were interviewed about their experiences. An analysis of the participants’ comments and actions indicated that co-creation in haptics can help designers explore new effects if they are sufficiently able to control the extent and direction of exploration.