Blog / Why Selected Scores Can Change an Observed Correlation

Why Selected Scores Can Change an Observed Correlation

Selection can narrow score variation, changing the relationship observed in the selected sample without proving the wider-population relationship has the same size.

October 20, 2026 · 6 min read · By Cepingku Editorial Team

When a study reports that score X and outcome Y have a weak or strong correlation, the coefficient is not the whole story. The people in the study may already have been made similar by a selection rule: they may be applicants, people who passed an earlier screen, volunteers, or people in a narrow score band. If selection narrows the observed variation in X, the calculated correlation describes that selected sample. It does not automatically equal the relationship in the broader group from which the sample came.

Range restriction is a selection question

Sackett and Yang’s original paper describes the common problem of estimating a population correlation from an observed correlation in a sample restricted by selection. Their typology distinguishes scenarios by the variable on which selection occurs, whether unrestricted variances are known, and whether a third selection variable is involved. That is why ‘range restriction’ is not one universal condition that can be fixed with a button: the selection mechanism and available information determine what can be estimated.

A fully fictional calculation

Use these five invented pairs only to inspect the arithmetic. Let X = [0, 2, 4, 6, 8] and Y = [0, 1, 5, 4, 10]. These are not assessment scores, job-performance data, or observations from this site. For the sample Pearson correlation, the means are x-bar = 4 and y-bar = 4. The sum of cross-products of deviations is 46. The sums of squared deviations are 40 for X and 62 for Y. Therefore r = 46 ÷ square root of (40 × 62) = 0.924, rounded to three decimals.

Now retain only the last three pairs because X is at least 4. This selection is deliberately artificial; it simply imitates selecting on X. The retained values are X = [4, 6, 8] and Y = [5, 4, 10]. Their means are 6 and 19/3. Their cross-product sum is 10, while the squared-deviation sums are 8 and 62/3. Using the same formula gives r = 10 ÷ square root of (8 × 62/3) = 0.778, again rounded to three decimals.

Fictional sampleObserved X range and correlation
All five invented pairsX runs from 0 to 8; r = 0.924.
Only pairs with X at least 4X runs from 4 to 8; r = 0.778.

This example does not show that selecting high scorers must turn .924 into .778. It shows a checkable mechanism: after selection, both the variation in X and the paired observations used in the calculation changed, so the observed correlation changed. If Y lay exactly on a straight line with X, a narrower range could still produce a very high correlation. Selection on a third variable can create a different pattern again. No invented calculation can identify the population relationship or justify a correction for a real study.

Do not apply a correction without the selection facts

Before treating a correlation as distorted by range restriction, identify whether a real selection process occurred; whether it selected on X, Y, or another variable; what information is available about unselected people; and whether the relationship is approximately linear with comparable spread across the relevant range. Without those conditions, reporting a casual ‘corrected correlation’ may be more misleading than reporting the unadjusted result as a sample description. APA’s Guidelines for Psychological Assessment and Evaluation says that, when results are interpreted in populations or settings that differ from those studied by a developer, measurement effects such as restrictions in range should be accounted for and reported. The same guidance evaluates validity in the context of setting and purpose.

Read a correlation with its sampling frame

Ask who entered the sample, what rule determined entry, whether the result is only a within-sample description or a claim about unselected people, and which assumptions supported any adjustment. Those questions do not turn correlation into causation or make a personal score predict an outcome. They prevent a relationship measured in a narrowed group from being presented as though it had the same strength for everyone.

Career Decision Readiness is a self-reflection entry point for exploring how a person gathers information, makes choices, and turns a possible direction into next steps. This article does not study any association between the site’s results and employment, education, or another external outcome. It does not claim that this site has range restriction, calculate a correction for it, or make the assessment suitable for hiring, admission, or major career decisions.

Want to see where you land?

Take the Career Decision Readiness and get your results in minutes.

Take the Career Decision Readiness →

← Back to the blog