When the results of a study can be generalized this is called?
|
Department of Psychopharmacology, National Institute of Mental Health and Neurosciences, Bengaluru, Karnataka, India Show Find articles by Chittaranjan Andrade Author information Copyright and License information Disclaimer Department of Psychopharmacology, National Institute of Mental Health and Neurosciences, Bengaluru, Karnataka, India Address for correspondence: Dr. Chittaranjan Andrade Department of Psychopharmacology, National Institute of Mental Health and Neurosciences, Bengaluru - 560 029, Karnataka, India. E-mail: moc.liamg@cedardna Copyright : © 2018 Indian Psychiatric Society - South Zonal Branch This is an open access journal, and articles are distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 License, which allows others to remix, tweak, and build upon the work non-commercially, as long as appropriate credit is given and the new creations are licensed under the identical terms. AbstractReliability and validity describe desirable psychometric characteristics of research instruments. The concept of validity is also applied to research studies and their findings. Internal validity examines whether the study design, conduct, and analysis answer the research questions without bias. External validity examines whether the study findings can be generalized to other contexts. Ecological validity examines, specifically, whether the study findings can be generalized to real-life settings; thus ecological validity is a subtype of external validity. These concepts are explained using examples so that readers may understand why the consideration of internal, external, and ecological validity is important for designing and conducting studies, and for understanding the merits of published research. Key words: CATIE, ecological validity, external validity, internal validity, reliability DID CATIE HAVE EXTERNAL VALIDITY?The answer is both yes and no. CATIE[1] was designed as an effectiveness study; that is, a study with relevance to real-world settings. The CATIE findings are relevant to clinical practice in the USA but are of questionable relevance in India. One reason is that, in the USA, where CATIE was conducted, the primary outcome, time to all-cause treatment discontinuation, is substantially patient-influenced, whereas in India, where families supervise treatment, it is largely caregiver-determined. Another and more important reason is that the healthcare delivery system in clinical practice is strikingly different in the two countries. Thus CATIE has good external validity for clinical practice in the USA but not in India. RELIABILITY AND VALIDITYReliability and validity are concepts that are applied to instruments such as rating scales and screening tools. Validity describes how well an instrument does what it is supposed to do. For example, does an instrument that screens for depression do so with high sensitivity and specificity? Reliability describes the consistency with which results are obtained. For example, if an instrument that rates the severity of depression is administered to the same patient twice within the span of an hour, are the scores obtained closely similar? Different types of reliability and validity describe desirable psychometric properties of research and clinical instruments.[2,3] Validity can also be applied to laboratory and clinical studies, and to their findings, as well, as the sections below show. INTERNAL VALIDITYInternal validity examines whether the manner in which a study was designed, conducted, and analyzed allows trustworthy answers to the research questions in the study. For example, improper randomization, inadvertent unblinding of patients or raters, excessive use of rescue medication, and missing data can all undermine the fidelity of the results and conclusions of a randomized controlled trial (RCT). That is, the internal validity of the RCT is compromised. Internal validity is based on judgment and is not a computed statistic. Internal validity examines the extent to which systematic error (bias) is present. Such systematic error can arise through selection bias, performance bias, detection bias, and attrition bias.[4] If internal validity is compromised, it can occasionally be improved, for example, by a modified plan of analysis. However, biases can be often fatal as, for example, if double-blind ratings were not obtained in an RCT. EXTERNAL VALIDITYExternal validity examines whether the findings of a study can be generalized to other contexts.[4] Studies are conducted on samples, and if sampling was random, the sample is representative of the population, and so the results of a study can validly be generalized to the population from which the sample was drawn. But results may not be generalizable to other populations. Thus external validity is poor for studies with sociodemographic restrictions; studies that exclude severely ill and suicidal patients, or patients with personality disorders, substance use disorders, and medical comorbidities; studies that disallow concurrent treatments; and so on. External validity is also limited in short-term studies of patients who need to be treated for months to years. External validity, like internal validity, is based on judgment and is not a computed statistic. ECOLOGICAL VALIDITYEcological validity examines whether the results of a study can be generalized to real-life settings.[5] How is this different from external validity? External validity asks whether the findings of a study can be generalized to patients with characteristics that are different from those in the study, or patients who are treated in a different way, or patients who are followed up for longer durations. In contrast, ecological validity specifically examines whether the findings of a study can be generalized to naturalistic situations, such as clinical practice in everyday life. Ecological validity is, therefore, a subtype of external validity. The ecological validity of an instrument can be computed as a correlation between ratings obtained with that instrument and an appropriate measure in naturalistic practice or in everyday life. The ecological validity of a study is a judgment and is not a computed statistic. DISCUSSIONEcological validity was originally invoked in the context of laboratory studies that required to be generalized to real-life situations.[5] Thus, laboratory studies of the neuropsychological and psychomotor impairments produced by psychotropic drugs have poor ecological validity because what is studied in relaxed, rested, and healthy subjects tested in a controlled environment is very different from demands that stressed patients face in everyday life. In fact, these cognitive and psychomotor tests, especially when based on computerized tasks, have no parallel in everyday life. How much less ecological validity, then, would research in animal models of different neuropsychiatric states have for patients in clinical practice? This explains why drugs that work in animal models often fail in humans.[6] On a parting note, a good understanding of the concepts of internal, external, and ecological validity is necessary to properly design and conduct studies and to evaluate the merits and applications of published research. When the results of a study can be generalized?The degree to which the results of a study can be generalized to other participants and conditions is its external validity.
What does it mean to generalize the results of a study?Generalizability is the degree to which you can apply the results of your study to a broader context. Research results are considered generalizable when the findings can be applied to most contexts, most people, most of the time.
What type of study can be generalized?Experimental research, for example, often produces generalizable results.
What is generalized data in research?Data generalization is the process of creating a more broad categorization of data in a database, essentially 'zooming out' from the data to create a more general picture of trends or insights it provides.
|
