ChatGPT and ChatGPT API: An Experiment with Evaluating ChatGPT Answers

This paper describes ChatGPT and how to use ChatGPT Application Programming Interface (API). The importance of knowing about ChatGPT API draws from the importance of Generative AI apps which the authors aim, with this paper, at spreading the knowledge about because they believe in the benefit of doing so. By mastering the API of ChatGPT, one can integrate it with one’s own application in a rapid system development method as opposed to building the whole application from scratch. This makes it possible to build wrappers around ChatGPT, for example, which can serve various needs. Moreover, one can benefit from the automation provided by ChatGPT API and can also utilize that to run several prompts automatically, for example, and even evaluate the returned results in a similar manner. In this research, an experiment was conducted where a list of prompts about technological inventors’ names (obtained from a dataset) were sent to ChatGPT using the ChatGPT API and the returned answers were evaluated automatically. This research has found that ChatGPT answered the questions with a high accuracy of 73.91%. Therefore, it is feasible to invest time and effort in using ChatGPT API. Since this study relies on using a prestored file of list of inventions, the limitation lies in the finite number of inventions which were queried. However, the same experiment can be applied on larger files with little effort, since the code, which has been available publicly by the authors, automates the process. This research can be used as a guideline for any researcher who is interested in using the ChatGPT API.