The usability analysis of chatbot technologies for internal personnel communications

The emergence and pervasiveness of the Internet provides opportunities for new types of communications between customers and service providers. One such technology is a chatbot: a computer program that simulates a human conversation enabled by the Internet. Chatbots are currently used for a variety of reasons, from daily weather reports to ordering pizza! In this paper we present the investigation of chatbot technologies for an industry partner to better their internal communication process between field technicians and engineers. Implementation of this technology will automate the technician-to-engineer communication process and thus will result in a much more efficient system. Our team followed a human factors engineering methodology where we compared different chatbot platforms (IBM, Pandora, Self-development Kit) via usability testing. User feedback was derived systematically from a wide range of users and usability of several platforms was tested using a cognitive walkthrough and the data was quantified using a System Usability Scale (SUS). The majority of the participants in this research study (eight out of 10) preferred IBM's Watson. This platform received an average SUS score rating of 81.9 out of 100.