Incremental Validity: Does Your Assessment Battery Add Up?

A common assumption in hiring is that more assessment is better. Phone screen, then technical assessment, then cognitive test, then structured interview, then panel interview. Each layer feels like it is adding information.

Sometimes it is. Often it is not.

Incremental validity is the measure of how much a new assessment adds to prediction beyond what you can already predict from assessments already in the battery. It is one of the most practically important and least-discussed concepts in selection science.

The basic problem

Many assessments that predict job performance measure overlapping constructs. A cognitive ability test and a structured problem-solving interview both measure reasoning. If you already have a strong cognitive test in your battery, adding a structured problem-solving interview may add very little additional predictive power because the two measures share most of their variance.

The two-assessment battery is more expensive to administer than the one-assessment battery. If the second measure adds no incremental validity, the cost is not justified by improved prediction.

What the research shows

Schmidt and Hunter’s meta-analytic work provides a useful framework. Their analysis of selection combinations found that:

• Cognitive ability + structured interview is one of the strongest two-predictor combinations, with a multiple validity around .63
• Adding a personality measure (conscientiousness) to that battery adds incremental validity because personality and cognitive ability measure distinct constructs
• Adding a second cognitive measure to a battery that already includes one adds very little incremental validity, regardless of how the second test is labeled

The constructs that tend to add incremental validity to a cognitive-based battery: conscientiousness, emotional stability, and job-specific knowledge (which taps domain expertise that general cognitive ability does not).

Practical incremental validity issues in hiring

The redundant screen: Many organizations run a phone interview, then a video screen, then an in-person interview. If all three cover the same territory with unstructured questions, the three rounds produce one data point three times. The process is expensive for candidates and employers, and no more predictive than one round done well.

The unvalidated add-on: Organizations sometimes add an assessment because it feels insightful or because a consultant recommended it. If the new assessment correlates highly with existing measures and has no independent validity evidence, it adds noise, not signal.

The battery with no design: A hiring process that has accumulated assessments over time without evaluation often has significant redundancy. The structured interview asks about problem-solving. The cognitive screen tests problem-solving. The case study presentation evaluates problem-solving. You have measured one thing three ways and have no data on other relevant constructs.

Building a battery that actually adds up

The design goal is a set of assessments where each component measures something the others do not, in proportion to how much that construct matters for the job.

Practical steps:

1. Start with job analysis to identify the most critical KSAOs
2. Select assessments that map to distinct KSAOs
3. Check the construct correlations between assessments before finalizing the battery
4. Remove or replace any assessment that correlates so strongly with another that the incremental contribution is near zero
5. Weight assessment scores in proportion to their validity and incremental contribution

This is not complex in principle. It requires upfront discipline in design rather than adding tools because they seem useful.