How to Build a Quality Assurance Scorecard

Quality assurance helps call centers understand whether calls are being handled well. At the center of most quality programs is a QA scorecard, a structured checklist used to review and score calls. A good scorecard makes quality reviews fair, consistent, and useful for coaching. A poor one can turn reviews into a box-ticking exercise that frustrates staff and does little to improve calls.

This guide explains how to build an effective QA scorecard, step by step.

What Is a QA Scorecard?

A QA scorecard is a set of criteria used to evaluate calls, with a scoring method that produces a quality score. Reviewers listen to calls, or read transcripts, and assess them against each item on the scorecard.

Step 1: Define the Purpose

Start by deciding what the scorecard is for. Common goals include:

• Improving customer experience.
• Supporting coaching and development.
• Ensuring compliance with laws and policies.
• Maintaining consistency across the team.
• Identifying trends and training needs.

Most scorecards serve several purposes, but being clear about priorities helps decide what to include and how to weight it.

Step 2: Understand What Good Looks Like

Before writing criteria, define what a good call looks like for your business. Useful sources include:

• Listening to excellent calls, and noting what made them good.
• Customer feedback, showing what customers value.
• Complaints, showing what goes wrong.
• Business goals, such as resolution or sales.
• Compliance requirements.
• Input from experienced agents and supervisors.

Step 3: Group Criteria Into Categories

Organize the scorecard into a few clear sections. Common categories include:

• Opening: greeting, introduction, required statements.
• Verification and compliance: identity checks, disclosures, data protection.
• Understanding the customer: listening, questioning, confirming the issue.
• Resolution: accurate information, correct process, solving the problem.
• Communication: clarity, tone, empathy, avoiding jargon.
• Call control: managing holds, transfers, and time effectively.
• Closing: summary, next steps, checking for further needs.
• Documentation: accurate notes and records.

Step 4: Write Clear, Observable Criteria

Each item should describe something a reviewer can clearly see or hear on the call.

Vague: "Agent was empathetic."

Clear: "Agent acknowledged the customer's situation or feelings before moving to a solution."

Vague: "Good call control."

Clear: "Agent asked permission before placing the customer on hold and checked back if the hold lasted more than a minute."

Clear criteria make scoring more consistent between reviewers and give agents specific guidance.

Step 5: Choose a Scoring Method

Yes / No / Not applicable

Each item is marked as met, not met, or not relevant to the call. This is simple and consistent, and works well for clearly defined behaviors.

Rating scales

Items are scored on a scale, such as 1 to 5. This allows more nuance but can lead to more variation between reviewers.

Combination

Many scorecards use yes/no for compliance and process items, and scales for softer skills.

Including "not applicable" is important, since not every item applies to every call. For example, a transfer item doesn't apply if no transfer was needed.

Step 6: Weight What Matters Most

Not all items are equally important. Weighting gives more value to the items that matter most, such as resolving the issue or giving accurate information, compared with less critical items.

Critical items

Some items may be considered critical, such as required legal disclosures, protecting payment details, or verifying identity before sharing account information. Missing a critical item may cause the call to fail regardless of other scores, or may be reported separately. Use critical items sparingly, for genuinely essential requirements.

Step 7: Include the Customer Outcome

A call can follow every step and still leave the customer without help. Include items that focus on outcomes, such as:

• "Was the customer's issue resolved, or were clear next steps agreed?"
• "Was the information given accurate?"

Step 8: Leave Room for Comments

Scores show what happened. Comments explain why. Include space for reviewers to note specific moments, strengths, and suggestions, which makes the review far more useful for coaching.

Step 9: Keep It Manageable

Long scorecards with dozens of items take longer to complete, are harder to score consistently, and can encourage box-ticking. Focus on the items that genuinely reflect quality and matter most to customers and the business.

Step 10: Test the Scorecard

Before rolling it out:

• Score sample calls using the new scorecard.
• Check whether scores reflect reality. Do excellent calls score highly and poor calls score low?
• Ask agents and supervisors for feedback.
• Refine unclear items.

Step 11: Calibrate

Calibration sessions bring reviewers together to score the same calls independently and then compare results. Differences reveal items that are interpreted differently, which can then be clarified. Regular calibration keeps scoring fair and consistent over time.

Step 12: Connect Scores to Coaching

A scorecard is most valuable when it leads to improvement. Share reviews with agents, discuss specific moments in the call, recognize strengths, and agree on one or two areas to work on.

Step 13: Review the Scorecard Regularly

As products, processes, customer expectations, and rules change, the scorecard should change too. Review it periodically to make sure it still reflects what good looks like.

Example Scorecard Structure

Scorecards for AI Calls

As AI voice agents handle more calls, many businesses review automated calls using similar scorecards, checking whether the AI gave required statements, understood the caller, provided accurate information, handed over appropriately, and resolved the issue. The criteria may differ slightly, but the principles are the same.

Common Mistakes

• Too many items, making reviews slow and superficial.
• Vague criteria that reviewers interpret differently.
• Ignoring outcomes, focusing only on process.
• Overusing critical items, making minor mistakes feel like failures.
• No calibration, leading to inconsistent scores.
• No link to coaching, so scores don't lead to improvement.
• Never updating the scorecard.

Measuring What Matters

A well-designed QA scorecard turns the question "Was that a good call?" into a clear, consistent, and fair assessment. By defining what good looks like, writing clear criteria, weighting what matters most, calibrating reviewers, and linking results to coaching, call centers can use their scorecards not just to measure quality, but to improve it.

Related articles

• Call Quality: What It Means and How Businesses Measure It
• What Is Speech Analytics?
• Monitoring Staff Fairly: Balancing Oversight and Trust