AI Conversation Quality Specialist

Who We Are:

At ContactPoint 360 , we put people first . As a global customer experience company, we partner with top brands to deliver outstanding service through meaningful interactions. Our Durban team is growing, and we’re looking for passionate, people-focused individuals who thrive in a fast-paced, hospitality-driven environment.

You’ve spent years listening to calls, scoring interactions, and coaching agents on what good sounds like in any contact center. Now there’s a new kind of agent that needs your expertise — an AI one.

The Opportunity:

You’ll be working with a deployed AI agent handling real customer interactions for some of the largest cable, telecom, and utility providers in the US. As an AI Conversation Quality Specialist, your job is to make this AI agent better — by applying the same QA instincts and coaching standards you’ve used on human agents for years.

This is not a data entry role. It’s applied quality work with direct impact on millions of customer interactions.

What You’ll Do:

Key Responsibilities

Quality Evaluation

• Score the AI agent’s responses across real and synthetic conversations using structured rubrics — assessing tone, accuracy, empathy, and escalation appropriateness
• Provide written rationale for every rating, the same way you’d document a coaching observation for a human agent
• Participate in calibration sessions to maintain consistency across the QA team
• Flag guideline gaps that are causing scoring inconsistency and propose rubric updates

Training Data & Coaching Input

• Identify where the AI agent’s responses would fail a QA scorecard if delivered by a human agent — document the gap with specificity
• Write gold-standard prompt-response pairs that reflect how a top-performing US agent would actually handle an interaction
• Design edge-case scenarios — billing disputes, repeat callers, escalation-prone situations — that expose gaps in the AI agent’s current behavior

Stress Testing

• Submit challenging, real-world customer inputs to surface cultural blind spots, over-hedging, or inappropriate escalation in the AI agent’s responses
• Simulate frustrated or non-standard customer personas that reflect actual US contact center patterns
• Log and categorize failure modes systematically so engineering teams can prioritize fixes

Continuous Improvement

• Track and report on annotation volume, accuracy, and throughput against weekly targets
• Collaborate with Conversation Design and ML teams to translate QA findings into actionable model feedback

Nice to Have

• QA Lead or Team Lead experience within a BPO environment
• Frontline agent training or L&D background
• Prior experience with AI annotation platforms (Scale AI, Surge, Appen)
• Familiarity with AI failure modes — hallucination, robotic phrasing, over-hedging, inappropriate escalation
• Bilingual English/Spanish

What You Bring:

• Matric / Grade 12 (essential)
• Clear criminal record (essential)
• Excellent communication skills (written and verbal) (essential)- English level C1/C2
• Computer literacy and ability to work on multiple platforms
• 2+ years in contact center QA — scoring agent interactions against rubrics, identifying coaching opportunities, and translating US consumer expectations and success factors into the South African context
• Experience in cable, telecom, internet, or utilities verticals strongly preferred
• Strong written communication skills — ability to clearly articulate why a response works or doesn’t
• Comfortable with structured rubrics, scorecards, and annotation guidelines
• High attention to detail; ability to maintain quality across high daily annotation volumes
• Ability to work night/ shift hours

What We Offer:

• Competitive base salary + performance-based incentives/commission
• Paid training and continuous coaching
• Career growth opportunities within the company
• Supportive and energetic team environment

Location:

This is an on-site position based at our Durban- Mount Edgecombe Offices.