How to Write a Scoring Rubric for an Interview Competency

The most common rubric failure is anchoring levels to adjectives instead of behaviors. A rubric where level 5 means “outstanding” and level 3 means “satisfactory” is not a rubric. It is a label system for a judgment that the interviewer still has to make independently.

A working rubric describes observable behavior at each level, the principle behind behaviorally anchored rating scales. The interviewer matches what the candidate said to a behavioral description. Here is how to build one.

Step one: define the competency

Before you write any anchor, you need a precise behavioral definition of the competency for this specific role, ideally drawn from job analysis. Not a generic definition from a competency library: a definition grounded in what strong performance on this dimension actually looks like in this job.

For a customer success manager role, “manages client expectations” means something specific. The definition should state: what task or situation this competency applies to, what effective behavior looks like in this role’s context, and how you would recognize it if you saw it.

If your team cannot agree on this definition, the rubric development process itself is valuable: it surfaces disagreements about what the job requires before you build an interview around assumptions.

Step two: collect critical incidents

The best anchors come from real examples of effective and ineffective performance collected from people who know the job.

Ask supervisors and high performers to describe specific situations where they observed someone handling the competency well, and situations where they observed it handled poorly. Collect at least ten to fifteen incidents per competency before you try to write anchors.

The incidents do two things: they ground the anchors in real work rather than abstract ideals, and they reveal the range of behaviors you need to cover across rating levels.

Step three: sort incidents by performance level

Group the incidents you collected by performance level: clearly strong, clearly weak, and in between. Do this sorting before you write anchors, and do it with multiple people independently to check for consensus.

If two people consistently disagree about whether an incident belongs at the high or middle level, the incident is probably at a boundary and can help you define where one level ends and the next begins.

Step four: write behavioral anchors

Now write the anchor descriptions, one per level, based on the sorted incidents. Each anchor should:

• Describe behavior, not outcomes (“candidate described presenting a status update that included risks and their mitigation plan” rather than “candidate communicated well with stakeholders”)
• Be observable from interview responses, not dependent on knowledge the interviewer cannot have
• Be distinct enough from adjacent levels that an interviewer can tell the difference

A five-point scale with behavioral anchors might look like this for a senior-level “technical communication” competency:

5: Candidate described anticipating the specific knowledge gaps their audience would have before the presentation, structuring the explanation to address those gaps, and checking for understanding in a way that invited clarification without being condescending.

3: Candidate described adjusting their explanation based on audience background, using analogies to make technical concepts accessible. Did not describe proactively anticipating gaps or verifying comprehension.

1: Candidate described technical communication that included significant jargon, or did not tailor the explanation to the audience’s background. No evidence of adapting to feedback.

Step five: have SMEs retranslate the anchors

The retranslation method is a quality check on anchor clarity. Give a group of SMEs the behavioral descriptions without their level labels. Ask each person to independently assign each description to a rating level.

If there is high agreement (most people assign anchor A to level 5), the anchor is clearly written. If there is significant disagreement, the anchor is ambiguous and needs revision.

Step six: calibrate before deployment

Before using the rubric in live interviews, run calibration sessions with interviewers. Show two or three example candidate responses and have each interviewer score independently. Review disagreements and discuss what evidence maps to which anchor.

Calibration does two things: it surfaces rubric ambiguities you can still fix before the cycle begins, and it builds a shared reference point across interviewers so that a level-4 score means the same thing regardless of who is scoring.