Shades of Grey in calculating Completion

Completion is one of the key metrics used to monitor performance of self service systems. There are wide discrepancies in the way organizations measure and make sense of this metric.

Some organizations calculate this to be the share of callers not transferring to agents. Some take this as the share of callers reaching the Main Menu. Others take into account the callers reaching the most basic of self service choices (say, hearing Account Balance for a Financial Services firm, or Recharge Amount left for a Mobile Service Provider).

All these methods have their merits in aiding understanding of the performance. But they do not tell us the true measure of self service. Method I tells us the hang-up rate of the IVR, but does not differentiate between callers who hang-up after succeeding in their task and those who hang-up after being unable to complete their objective. Method II is useful in that it tells us the number of people who reached the Main Menu and hence its complement, the share of callers who drop even before reaching the Main Menu. Method III is one of the most popular ways, and this method gets closer to actual self service performance. But, it mostly uses a default readout such as Account Balance, which callers automatically hear after getting authenticated. This means that callers may or may not have wanted to hear their Balance, and in assuming that all callers hearing their Balance to be self served is an error.

If all these methods have some issues, what is a better method? The more appropriate method would be to understand the number of callers succeeding in serving themselves after starting a particular process.  The following are some examples:

• Callers requesting for roaming activation on the IVR, and getting it activated right there
  
• Callers recharging their prepaid mobile by keying in the requisite codes
  
• Callers requesting for a Checkbook and the system accepting their request
  
• Callers starting a funds transfer on their IVR and completing it
  
• Callers requesting for a sports package for their satellite TV, and getting it added
  

Organizations could have high hang-up rates, high Main Menu rates or high rates of reaching Account Balance. But, none of these methods take into account the intention of the caller. This method enables us to understand their self service performance better, in that it takes into account only callers who started a process, and of those callers, how many succeed. That is the true measure of Self Service Completion.