There are “three types of measurement validation”: content, convergent/discriminant, and nomological/construct validation. Content Validation is preoccupied with the questions whether or not key elements are “omitted from the indicator” and whether or not “inappropriate elements are included in the indicator.” Although content validation is an “important contribution” to measurement validity, its chief limitation is that “alone it is incomplete, for two reasons”: “first, although a necessary condition, the findings of content validation are not a sufficient for establishing validity” and second “the trade-off between parsimony and completeness that arises because indicators routinely fail to capture the full content of a systemized concept.”
…show more content…
Convergent/discriminant is concerned with questions such as “Are the scores (level 4) produced by alternative indicators (level 3) of a given systemized concept (level 2) empirically associated and thus convergent?” “Do these indicators have a weaker association with indicators of a second, different systematized concept, thus discriminating this second group of indicators from the first?” (540). As for concerns about this type of validation, the first “is that scholars might think that in convergent/discriminant empirical findings always dictate conceptual choices” and the second “arises over the interpretation of low correlations among indicators.” …show more content…
(74)
Putnam constructs a summary Index of Institutional performance that “displays the full list of indicators and shows how each is correlated with this summary index.” He defends this choice by saying that “combining these diverse indicators into a single index reduces the idiosyncratic impact of any single measure… our composite measure is both comprehensive and internally consistent.” (74)
With that being said, there is a validity issue with the indicators in Making Democracy Work. Validity means that the measurement is indeed capturing what the concept is intended to make, and a perfect matching between attribute and indicators. However, validity issue is when the underlying attribute is one notion, the indicator is touching something else, and such a discrepancy can be witnessed for the Policy Implementation category because it is the central government, not the regional governments, that designed and funded the policies. Having neither funded or created these policies, the regional governments only relation to the these polices is just its implementation. Suffice to say, the indicators for Policy Implementation is touching on something else, instead of the underlying
Convergent/discriminant is concerned with questions such as “Are the scores (level 4) produced by alternative indicators (level 3) of a given systemized concept (level 2) empirically associated and thus convergent?” “Do these indicators have a weaker association with indicators of a second, different systematized concept, thus discriminating this second group of indicators from the first?” (540). As for concerns about this type of validation, the first “is that scholars might think that in convergent/discriminant empirical findings always dictate conceptual choices” and the second “arises over the interpretation of low correlations among indicators.” …show more content…
(74)
Putnam constructs a summary Index of Institutional performance that “displays the full list of indicators and shows how each is correlated with this summary index.” He defends this choice by saying that “combining these diverse indicators into a single index reduces the idiosyncratic impact of any single measure… our composite measure is both comprehensive and internally consistent.” (74)
With that being said, there is a validity issue with the indicators in Making Democracy Work. Validity means that the measurement is indeed capturing what the concept is intended to make, and a perfect matching between attribute and indicators. However, validity issue is when the underlying attribute is one notion, the indicator is touching something else, and such a discrepancy can be witnessed for the Policy Implementation category because it is the central government, not the regional governments, that designed and funded the policies. Having neither funded or created these policies, the regional governments only relation to the these polices is just its implementation. Suffice to say, the indicators for Policy Implementation is touching on something else, instead of the underlying