http://www.statisticshowto.

com/statistical-conclusion-validity/

Statistical Conclusion Validity

Statistics Definitions > Statistical Conclusion Validity

You may find it helpful to read this article first: Reliability and Validity in Research.

What is Statistical Conclusion Validity?

Are you making the right conclusion about your data?

Statistical Conclusion Validity(SCV), or just Conclusion Validity is a measure of how

reasonable a research or experimental conclusion is. For example, lets say you ran some
research to find out if two years of preschool is more effective than one. Based on the data,
you conclude that theres a positive relationship between how well a child does in school and
how many years of preschool they attended. Conclusion validity well tell you how reliable
that conclusion is.

Conclusion validity is only concerned with the question: Based on the data, is there a
relationship or isnt there? It doesnt delve into specifics (like reliability tests) about what
kinds of relationship exist. It can be used for qualitative research as well as quantitative
research. That said, if you use the term statistical conclusion validity, thats usually taken as
meaning theres some type of statistical data analysis involves (i.e. that your research has
quantitative data).

Its important to realize that theres no such thing as perfect validity. Type 1 errors and Type
2 errors are a part of any testing process, so you can never be 100% certain that your
conclusions are correct. However, SCV refers to reasonable conclusions based on your data
not perfect ones.

Threats to Statistical Conclusion Validity

Threats lead you to make incorrect conclusions about relationships. They include:
 Fishing (mining the data and repeating tests to find somethinganything!
significant): can result in incorrectly concluding there is a relationship when in fact
there is not.
Low statistical power can cause you to incorrectly conclude there is no relationship
between your variables.
Poor reliability of treatment implementation: if you havent used standard
procedures and protocols, it could cause you to underestimate effects.
Random irrelevancies in the setting: this means any distraction, from weather thats
too hot to dealing with cantankerous people.
Restriction of range: can also lead to incorrect estimates.
Unreliable measures: can result in over- or underestimating the size of the
relationship between variables.
Violated assumptions for tests: can cause a multitude of problems including
overestimating or underestimating effects.

Other Types of Validity

Three other types of validity are used to analyze research and tests:

External Validity: the test or research can be applied to other areas.

Internal Validity: the test or instrument is measuring what its supposed to.
Construct Validity: the research/tests are well-constructed using established standards
and methods.