Why sampling fails in IVR Customer Behavior Analysis

Enterprises all around the world are investing heavily on automated Customer Interaction systems. Some enterprises have millions of automated customer interactions per month and hence have the necessity to create metrics that reflect the performance of the IVR.

Metrics such as self service completion rates and the repeat call rate to the IVR provide but a peripheral and delayed view of its performance. Everything is hunky dory as long as the metrics show high self service completion rates and low repeat caller rate. It is when the home grown metrics showcase below par performance, that management eyebrows are raised.  Hence, root cause for failure is analyzed based on logged caller interactions within IVR. Statistical analysis based on call samples is an often misused methodology, and I have a feeling that it breaks down magnificently in analyzing call tree structures.

IVRs call volumes are distributed heterogeneously. There is a high level of randomness in both call origination and in caller behaviour within IVR. Chronological external factors such as time of day or month and unpredictable factors such as the state of the economy, factor in a random manner deciding caller behaviour within the IVR. Simple sampling methods hence break down magnificently.

There is no way to pre-know problem areas within an IVR. If IVR calls are sampled after identifying problem areas, we run the risk of delaying important business improvements which could very quickly snowball into loss of business. This rules out sampling ideas such as quota sampling and cluster sampling.

Shaky sampling techniques raised to complete lack of knowledge of IVR performance render it impossible for caller behaviour to be unequivocally represented in any sample set. This would hence further impact significantly in recommendations for IVR performance improvement.

Business decisions based on incorrectly represented data, directly results in lowered customer service satisfaction levels and increase maintenance costs. It is hence imperative that we utilize the raw data utilized in call the calls to aid business decisions. In these days of increased computing power, the term "it has been statistically proved" loses significance in a lot of applications. IVR interaction is one of them.