| Sumario: | Most of the conversational AI platforms use the natural language understanding pipeline. An essential algorithm of the pipeline is intent recognition. It is responsible for classifying input messages into classes (intents), and this way, controlling the dialog flow. These days, many intent recognition services are currently also available as web services or as open-source alternatives. However, there is a limited number of datasets to compare their performance in a scenario of noised input. Additionally, many factors, such as CPU and memory requirements, make selecting the right approach challenging in practice. This paper presents a novel CIIRC dataset for evaluating the impact of noise (sentence disfluencies and filler sentences) on intent recognition algorithms. The data set focuses on command control as well as on social topics. We suggest criteria for selecting the best intent recognition algorithm. Finally, we use the suggested criteria and the new CIIRC dataset to compare the selected public intent recognition services with popular open-source algorithms.
|
|---|