Call Calibration Revisited: Subjectivity
September 16, 2008 by Arthur Mays, Quality Assurance in Servers and Solutions, Tech Stuff, The Planet
A couple of weeks ago, Flora posted a few questions in the comments section of our Call Calibration blog and asked for more information on our call-monitoring process: “I’m interested to know more about calibration and call monitoring. When representatives are appraised by different appraisers, how to ensure fairness? How to eliminate the effect of subjectiveness [sic] in call monitoring?”
As the Contact Center QA manager, fairness is always a main concern for me, so I’d like to take a few minutes to share a little more about how we’ve set up our system.
In the Call Calibration blog, you got a glimpse of what happens during our weekly calibration meetings: The management staff, supervisors, team leads and randomly selected agents monitor and review a sample selection of calls. We discuss the call, and each team member shares their view on how they interpreted the customer-agent interaction. The call is reviewed from a process-and-procedure viewpoint, measuring how it impacts customer satisfaction.
Subjectivity in call monitoring takes two distinct forms: the subjective goals when working with each customer and the subjectivity inherent in peer reviews of performance. The question of the qualitative or subjective aspects in each call is very important: An agent can hit all the points on the evaluation form and the customer’s needs may still not be fully met. Our goal is to satisfy our customers, and if we aren’t hitting that goal with these objective metrics, these sessions are counterproductive. As such, our call calibration grading sheets have evolved over the course of our reviews, acknowledging the importance of — and the subjectivity inherent in — one-on-one interaction.
For instance, empathy is probably the most subjective quality of a call, so we needed to learn the best ways to “grade” when is a call truly sincere and when it sounds forced and unattached. Is there an objective way to ensure this subjective goal is met? We want to move beyond responses such as, “I’m sorry to hear that,” to responses like, “I’m very sorry to hear that you are experiencing X issue. Let me see what I can do to help.” We work with all of our agents to get their input on the most organic, genuine ways to communicate help to our customers, and as we create these hosting call-center best practices, we get additional feedback that helps us build and shape the way we run our organization.
We know that it’s impossible to separate reviewers from their relationships with the agents being graded, so subjectivity will always be an implicit part of the review process. We’ve made the environment very positive, and we follow up with each individual reviewer after our calibration sessions if his or her perception of the call differs from others. Even on the calls scored very poorly, the agents being reviewed should never feel like they are being picked on or talked down to … Every review offers an opportunity to learn how to perform better — it’s not intended to harp on what was done wrong.
When we first introduced the calibration program, there were supervisors who did not want their teams monitored. The fingers always pointed to someone else’s team, and no one really wanted to hear what they did wrong on a given call. By making these calibration sessions much more casual and positive, the tide turned, and agents are now anxious to be selected for review. We made sure to establish the underlying objective of how our interactions affect The Planet’s overall customer experience, and that team-centric goal goes a long way in opening everyone to feedback and criticism.
Calibration is fun now. It’s an engaging process showing us how we did from the customer’s standpoint at the end of the day. If you have any additional questions about the call calibration process, please let us know!
-Arthur
September 17th, 2008 at 11:48 am
Good efforts to expermentalize on call samples via monitoring and reviewing the efficiency of the services.
September 22nd, 2008 at 10:51 am
““I’m very sorry to hear that you are experiencing X issue. Let me see what I can do to help.”
PLEAAAAAAAAAAAAAAASEEEEEE do NOT train your staff to give such a corny, canned, useless answer. 99.99% of the time the staff member (who would normally be a member of billing staff as the first people to answer) can do nothing about the issue nor even have the faintest idea what the customer is talking about if its a technical issue. The only thing they usually d is refer it to someone else. Its a terrible canned answer.
September 24th, 2008 at 10:04 am
Hi James, Thanks for your comment.
It’s not a matter of scripting canned answers … the example was more to convey a shift from being a little distanced with the response (I’m sorry to hear that) to acknowledging/confirming the problem and doing everything we can to help.
Our billing staff does not answer technical support phone calls, they handle a completely separate call queue specific to billing issues.
I personally don’t know how to fix a server or give full technical responses to customers in tickets, but I know exactly who would know how to best respond … Because I can’t answer the question, I shouldn’t convey sympathy and understanding? I shouldn’t reinforce the fact that I’m here to help? It might sound corny or canned, but it’s the truth, and it’s much better than going to the other extreme and not saying anything.
We can look at the scenario to an extreme: Your server’s mail is not delivering correctly, so you create a ticket (like 5128112PLNT) AND call into support to get it working (which you didn’t have to do for that ticket). The first thing out of the representatives mouth can either be “Okay. *Long Pause while beginning to work on it*” or a repeat of the problem to confirm exactly what is going on with a reminder that we are here to help. It takes an extra 30 seconds to ensure that we begin working on the correct problem (which is why we confirm what we’ve heard), and regardless of the perceived utility of “Let me see what I can do to help,” it reinforces the fact that our customer support representatives are here to help you.
In a call center environment, representatives should be talking to the customer, not to a script. Because customer experience can be tracked as a whole, we can see traits of good calls and find ways to work those traits into all of our calls … you’d be hard-pressed to call that useless.
January 22nd, 2009 at 6:09 am
I think monitoring is essential for training, guidence and improvements.