Building fair, defensible interview rubrics

Start with the outcomes of the role

A rubric is a shared definition of what your team is looking for. Without one, interviewers can ask similar questions and still evaluate very different things. One person rewards confidence, another rewards technical detail, and another prioritises a career history that feels familiar. The final discussion then becomes a negotiation between preferences.

Begin by naming the work the hire needs to accomplish. “Communicate clearly” is broad. “Explain a technical trade-off to a non-technical stakeholder and check their understanding” is observable. Translate each important outcome into a criterion that an interview, work sample, or other appropriate assessment can actually explore.

Describe what different levels look like

A scale of one to five does not create clarity on its own. Reviewers need anchors: short descriptions of the evidence associated with each level. Keep those descriptions focused on the candidate’s reasoning and actions, rather than personality labels such as “dynamic” or “a natural leader.”

For a problem-solving criterion, a developing answer might describe an action but not explain why it was chosen. A strong answer could identify constraints, compare options, explain a decision, and reflect on the result. These anchors are examples, not universal standards; they need to reflect the seniority and responsibilities of your particular role.

• Use job-relevant actions rather than impressions.
• Distinguish missing evidence from weak evidence.
• Avoid rewarding presentation style unless it is genuinely required for the work.
• Leave a place for the reviewer to explain their score.

Keep observations separate from conclusions

Interview notes should capture what the candidate actually said or demonstrated. “Named two options and explained the cost of delaying the decision” is more useful than “good strategic thinker.” The first can be discussed and checked. The second is already a conclusion, with the supporting evidence hidden.

After the conversation, connect those observations to the rubric. If a question did not produce enough information, mark that gap rather than inventing certainty. A targeted follow-up may be the right next step. This makes the decision trail more understandable for reviewers who were not in the interview.

Calibrate before the real interviews

Give the hiring team a short sample answer and ask everyone to assess it independently. Compare the reasoning, not just the numbers. If one interviewer treats brevity as confidence and another treats it as a lack of depth, that difference should be resolved before it affects a candidate.

Repeat the exercise when the role changes or a new interviewer joins. Do not force artificial agreement on every answer. The useful outcome is a clearer understanding of which evidence matters, where judgement is required, and when a reviewer should record uncertainty instead of a confident rating.

Use AI as a drafting aid, not an authority

AI can help turn a job description into an initial set of questions and criteria. It can also reproduce vague requirements, inappropriate assumptions, or criteria that sound polished but have little connection to the work. A human owner should review every proposed criterion and remove anything the team cannot justify.

A structured rubric can support consistency; it does not guarantee the absence of bias or make a process automatically compliant with employment law. Review the rubric against actual job needs, accessibility considerations, and local requirements. Keep the final decision with accountable people who can explain both the evidence and the limits of their assessment.