What is the industry standard for Bot Accuracy? Is 94 are already good?
especially for BFSI industry.
in which percentile is this 94% accuracy? is it the highest or medium? or lowest?
Hi @Abdullah_Zuhud ,
Bot Accuracy as it measures the percentage of utterances that have confidence more than minConfidence/ that hits an intent.
- There is no standard metric to decide what is good if we move from 90 to 94 to 98, etc.
It depends on multiple factors for any bot: Number of users, Usecase type, whether users click a button or its open for users to type an utterance. If use cases have button type selections, the more likely the bot accuracy is because buttons will keep on invoking confidences trained for it.
For an open scenario where the user types to get through a use-case, its more likely that it might go to an unidentified utterance, the Solution to which is retraining the bot timely with unidentified utterances to ensure more coverage, which is a standard procedure.
- So it is not if its BFSI, but the nature of the usecase itself and how users behave for the particular bot instead of the industry at whole.
- At 94%, its a very good accuracy if you want a blunt answer. You can consider adding unidentified utterance to take it higher to 95+ but more or less its good.
- If you want an average, you can target 10 bots in BFSI, check percentages, add it up and divide by 10 to get an average. But as I mentioned in 1 & 2, it depends on the nature of usecase and user behaviour