Blog / Why Translated Questionnaire Wording Does Not Guarantee Comparable Scores
Why Translated Questionnaire Wording Does Not Guarantee Comparable Scores
Matching wording or response scales can be a necessary start, but cross-language score interpretation requires evidence about meaning, context, and score behaviour.
October 22, 2026 · 6 min read · By Cepingku Editorial Team
Ready to start?
Start the Career Decision Readiness →Two questionnaire versions can look sentence-by-sentence equivalent, use the same one-to-five response scale, and still fail to support a direct comparison of their scores. A word may evoke advice, performance evaluation, criticism, or an informal conversation. A behaviour that is ordinary and feasible in one setting may be unfamiliar or constrained in another. Similar text can therefore prompt different experiences, contexts, and response processes.
The International Test Commission’s Guidelines for Translating and Adapting Tests says that adaptation applies whenever tests move from one cultural setting to another, whether translation is required or not. Its second edition organizes 18 guidelines around pre-conditions, test development, confirmation or empirical analyses, administration, score scales and interpretation, and documentation. Translation is one part of the work; it is not the endpoint of cross-cultural comparability.
One fictional item, three different questions
Imagine a fictional questionnaire intended to capture a planning habit. Its English source item says: ‘Before I choose a direction, I seek feedback from people who know the work.’ A literal Chinese version may preserve the word feedback, yet respondents may interpret it as a formal performance appraisal, general advice, negative criticism, or an informal conversation with a practitioner. Even if the translation is grammatical and bilingual editors find it natural, three different questions remain.
| Work stage | Question it answers, and what it does not yet establish |
|---|---|
| Translation | Does the target wording convey the source sentence’s linguistic and pragmatic meaning? It does not establish that respondents invoke the same construct. |
| Cultural adaptation | Are the behaviour, roles, timing, and setting familiar, feasible, and relevant to the intended construct in the target context? It does not establish that items work statistically the same way. |
| Empirical equivalence | Do appropriate data support the proposed cross-language interpretation of item and score structure? It does not prove every observed group difference is a real attribute difference. |
If the intended construct is actively obtaining work information from knowledgeable people, a Chinese adaptation might use wording closer to ‘ask people who understand the industry or role about actual work information.’ That is not a universally correct rewrite. Whether to retain feedback, use advice, or give examples must follow the pre-specified construct, target population, and evidence from review. It should not rest on one translator’s preference.
Check interpretation before claiming score equivalence
An inspectable adaptation process can document the construct definition, source-item intent, and elements that must remain constant; use reviewers familiar with both languages and target contexts; and conduct cognitive interviews about how people understand key terms, recall experiences, and select response options. If respondents consistently hear feedback as ‘my manager’s appraisal’ while the source item was meant to ask about seeking occupational information, that is evidence of a response-process mismatch. Editing prose alone does not resolve that mismatch.
Expert agreement and cognitive interviews remain different from empirical score-equivalence evidence. The ITC Guidelines place the collection of evidence about equivalence, reliability, and validity across multiple languages and cultures in their Confirmation section. When language versions are to be linked to a common reporting scale, the Guidelines call for an appropriate equating design and data-analysis procedures. The appropriate method depends on item format, sample, use, and intended interpretation; it cannot be inferred from a bilingual text alone.
State the intended comparison
‘The languages are comparable’ is not one claim. A version may be useful for personal reflection within its own language. Comparing mean scores across language groups, linking both versions to one percentile reference, or making the same threshold decision are progressively stronger claims. Each requires a defined population, version, administration conditions, and directly relevant evidence. If one item behaves differently, that alone does not prove a cultural cause: wording, translation, sample composition, response mode, and genuinely different experiences can all contribute.
Career Decision Readiness is a self-reflection entry point for exploring how a person gathers information, makes choices, and turns a possible direction into next steps. This article does not audit the site’s Chinese or English items, or claim they have completed ITC-described translation, cultural-adaptation, empirical-equivalence, score-linking, norm, or cross-language-comparison validation. A single score in either language is not suitable for hiring, admission, or a major career decision.
Want to see where you land?
Take the Career Decision Readiness and get your results in minutes.
Take the Career Decision Readiness →