Agent disputes
A dispute is a challenge to a specific evaluation criterion score. When an agent disagrees with how they were scored, they can submit a dispute explaining why the score should be different. Disputes trigger a review process where managers or QA leads examine the evaluation and either uphold the original score or change it.
Disputes are important for quality assurance. They identify potentially inconsistent scoring, unclear criteria definitions, or evaluator calibration issues. Tracking dispute patterns helps improve your evaluation process over time.
When agents can dispute
Section titled “When agents can dispute”Agents can dispute evaluations when:
- The evaluation is published (draft evaluations cannot be disputed)
- The agent is the assigned handler of the interaction
- The user has appropriate permissions (typically agents can dispute their own evaluations)
Users with evaluation management permissions can also initiate disputes on behalf of agents or to challenge scores during calibration exercises.
Submitting disputes
Section titled “Submitting disputes”To dispute an evaluation, open the interaction and navigate to the evaluation section. Click Dispute Evaluation to open the dispute interface.
The dispute screen shows the complete evaluation alongside the full interaction transcript. This split view lets agents compare scores to the actual conversation while building their dispute.
Selecting disputed criteria
Section titled “Selecting disputed criteria”Click any criterion score to propose a different rating. You can dispute multiple criteria in a single submission. For each disputed criterion, you must:
- Select the score you believe is correct (the proposed score)
- Choose a reason label explaining why you are disputing
- Optionally add context notes with specific details
Dispute reason labels
Section titled “Dispute reason labels”Happitu provides standard reason labels for disputes:
Score misapplied: The evaluator applied the criterion incorrectly based on the definition. Use this when you met the criterion requirements but were scored as not meeting them.
Missing context: The evaluator missed important context from earlier in the interaction. Use this when something earlier in the conversation affects how a later moment should be scored.
Partial credit: The criterion should have received partial credit rather than full fail or full pass. Use this for criteria where you partially met requirements.
Tech/audio issue: Audio quality, transcript errors, or technical problems affected the evaluation. Use this when technical issues prevented fair scoring.
Other: None of the standard reasons apply. Requires a detailed note explaining the situation.
Adding context notes
Section titled “Adding context notes”Notes are optional for most reason labels but required when selecting Other. Provide specific details that support your dispute:
- Reference timestamps in the interaction
- Quote relevant parts of the conversation
- Explain your reasoning clearly
- Reference specific criteria definitions if applicable
Good notes help reviewers understand your perspective and make fair decisions.
Submitting
Section titled “Submitting”Once you have selected disputed criteria, proposed scores, and chosen reason labels, review your dispute carefully. Verify that:
- You are disputing the correct criteria
- Your proposed scores are realistic
- Your reason labels accurately describe the situation
- Your notes are clear and specific
Click Submit for Review to send the dispute to your QA team. You cannot edit a dispute after submission, so review carefully.
Reviewing and resolving disputes
Section titled “Reviewing and resolving disputes”Managers and QA leads with appropriate permissions can view and resolve disputes.
Finding disputed evaluations
Section titled “Finding disputed evaluations”Disputed evaluations appear throughout Happitu with visual indicators. In interaction lists, evaluations with open disputes show dispute badges. In evaluation detail views, disputed criteria display prominently.
Use filters in Explore to find all interactions with open disputes. Filter by Has dispute to show only disputed evaluations. Export disputed evaluations for offline review using the export feature with the hasDispute column.
The resolution interface
Section titled “The resolution interface”Open a disputed evaluation to see the dispute details. For each disputed criterion, you will see:
- Original score (what the evaluator assigned)
- Proposed score (what the agent believes is correct)
- Reason label (why the agent is disputing)
- Agent notes (their explanation)
Review the interaction transcript alongside the dispute. Listen to or read the relevant portions to understand the agent’s perspective.
Making resolution decisions
Section titled “Making resolution decisions”For each disputed criterion, select a final score. You can:
- Uphold the original score: The original evaluation was correct
- Accept the proposed score: The agent was right, change to their proposed rating
- Select a different score: Neither the original nor proposed score is correct; assign a new rating
Add a resolution note explaining your decision. Good resolution notes are specific about why you made your choice, referencing the criterion definition or interaction evidence.
Completing resolution
Section titled “Completing resolution”After scoring all disputed criteria, submit the resolution. Happitu updates the evaluation scores to reflect your decisions and marks the dispute as resolved. The evaluation score recalculates automatically based on the new ratings.
Agents receive notification when their disputes are resolved, including the final scores and resolution notes.
Dispute analytics and patterns
Section titled “Dispute analytics and patterns”Tracking disputes helps improve your quality program. Review dispute patterns regularly to identify:
Calibration issues
Section titled “Calibration issues”If multiple evaluators receive disputes on the same criteria, they may be scoring inconsistently. Use disputes to identify evaluators who need calibration or criteria that need clearer definitions.
Criteria problems
Section titled “Criteria problems”High dispute rates on specific criteria indicate the criterion is unclear or unrealistic. Review disputed criteria monthly. If a criterion generates disputes consistently, revise its definition or scale.
Training opportunities
Section titled “Training opportunities”Disputes can reveal agent training needs. If agents frequently dispute criteria they should understand, they may need additional training on that quality standard.
Process improvements
Section titled “Process improvements”Disputes about missing context or transcript errors may indicate process issues. If agents frequently cite tech/audio issues, improve recording quality. If they cite missing context, ensure evaluators review complete interactions.
Best practices
Section titled “Best practices”For agents disputing
Section titled “For agents disputing”- Be specific: Reference timestamps and quote the interaction
- Know the criteria: Review criterion definitions before disputing
- Choose appropriate reasons: Select the label that best describes your situation
- Dispute promptly: Submit disputes while the interaction is fresh in your mind
- Accept outcomes: Not all disputes will be resolved in your favor; use feedback to improve
For managers resolving
Section titled “For managers resolving”- Review objectively: Set aside who submitted the dispute and focus on the evidence
- Explain decisions: Write clear resolution notes so agents understand your reasoning
- Look for patterns: Track which criteria and evaluators generate disputes
- Respond quickly: Resolve disputes within a few days to maintain trust in the process
- Use for calibration: Share anonymized disputes in calibration sessions
For the organization
Section titled “For the organization”- Track metrics: Monitor dispute rate, resolution time, and overturn rate
- Review regularly: Analyze dispute patterns monthly for trends
- Update criteria: Revise criteria that generate excessive disputes
- Train evaluators: Use disputes to identify evaluators needing calibration
- Communicate: Share dispute learnings with the QA team