Call Calibration Revisited: Subjectivity
September 16, 2008 by Arthur Mays, Quality Assurance in Servers and Solutions, Tech Stuff, The Planet
A couple of weeks ago, Flora posted a few questions in the comments section of our Call Calibration blog and asked for more information on our call-monitoring process: “I’m interested to know more about calibration and call monitoring. When representatives are appraised by different appraisers, how to ensure fairness? How to eliminate the effect of subjectiveness [sic] in call monitoring?”
As the Contact Center QA manager, fairness is always a main concern for me, so I’d like to take a few minutes to share a little more about how we’ve set up our system.
In the Call Calibration blog, you got a glimpse of what happens during our weekly calibration meetings: The management staff, supervisors, team leads and randomly selected agents monitor and review a sample selection of calls. We discuss the call, and each team member shares their view on how they interpreted the customer-agent interaction. The call is reviewed from a process-and-procedure viewpoint, measuring how it impacts customer satisfaction.
Subjectivity in call monitoring takes two distinct forms: the subjective goals when working with each customer and the subjectivity inherent in peer reviews of performance. The question of the qualitative or subjective aspects in each call is very important: An agent can hit all the points on the evaluation form and the customer’s needs may still not be fully met. Our goal is to satisfy our customers, and if we aren’t hitting that goal with these objective metrics, these sessions are counterproductive. As such, our call calibration grading sheets have evolved over the course of our reviews, acknowledging the importance of — and the subjectivity inherent in — one-on-one interaction.
For instance, empathy is probably the most subjective quality of a call, so we needed to learn the best ways to “grade” when is a call truly sincere and when it sounds forced and unattached. Is there an objective way to ensure this subjective goal is met? We want to move beyond responses such as, “I’m sorry to hear that,” to responses like, “I’m very sorry to hear that you are experiencing X issue. Let me see what I can do to help.” We work with all of our agents to get their input on the most organic, genuine ways to communicate help to our customers, and as we create these hosting call-center best practices, we get additional feedback that helps us build and shape the way we run our organization.
We know that it’s impossible to separate reviewers from their relationships with the agents being graded, so subjectivity will always be an implicit part of the review process. We’ve made the environment very positive, and we follow up with each individual reviewer after our calibration sessions if his or her perception of the call differs from others. Even on the calls scored very poorly, the agents being reviewed should never feel like they are being picked on or talked down to … Every review offers an opportunity to learn how to perform better — it’s not intended to harp on what was done wrong.
When we first introduced the calibration program, there were supervisors who did not want their teams monitored. The fingers always pointed to someone else’s team, and no one really wanted to hear what they did wrong on a given call. By making these calibration sessions much more casual and positive, the tide turned, and agents are now anxious to be selected for review. We made sure to establish the underlying objective of how our interactions affect The Planet’s overall customer experience, and that team-centric goal goes a long way in opening everyone to feedback and criticism.
Calibration is fun now. It’s an engaging process showing us how we did from the customer’s standpoint at the end of the day. If you have any additional questions about the call calibration process, please let us know!
-Arthur
