How do you combine results of different studies?
Paper by van Dijk & Schatschneider, 2020, available under a CC By 4.0 license. For some research questions, it is better to have large sample sizes. Instead of going out to collect these
samples, researchers can combine data from existing studies. There are two main ways to combine existing data: through meta-analysis of summary statistics, and through Integrative Data Analysis using individual participant data. See this white paper for more information and suggested readings for both. This methodology is based on the summary statistics of groups provided in research reports. With a meta-analysis, researchers aim to find the summary effect size of
their construct of interest. The summary effect size is represented by a weighted average of the effects included. Knowing the effect size across studies can help inform us about the true impact of a phenomenon. A second approach for meta-analysis is to understand the relations between multiple constructs. This can be done with meta-analytic structural equation modeling. Instead of focusing on single effect sizes, researchers compare and combine correlation matrices that are later used to
estimate path analytic models. Steps
Additional reading on meta-analysisMethodological
Examples from Psychology and Education
Integrative Data AnalysisThis methodology is based on raw, individual participant data. Unlike meta-analysis, in which the sample size is the number of studies included, IDA pools all individual data and creates one new data set. The goal of IDA is to create scaled scores on the constructs of interest across all independent data samples that will then be used as variables in subsequent statistical analysis. For each construct of interest, this is accomplished by selecting representative items from measures that are representative of this core construct and then modeling these items to create a valid and reliable scaled score. The items selected do not all have to be identical across the samples, and each sample can have unique items in addition to some items that are common across the samples. One method to create these scaled scores for IDA is using Moderated Nonlinear Factor Analysis (MNLFA). To estimate scaled scores across the independent data samples, MNLFA tests for measurement invariance across potential influential covariates at both the factor (intercept and variance) and item (intercept and variance) level. The following is an example of a path diagram for a MNLFA model on student behavior: Steps
Additional reading on IDA:Methodological
Examples from education
Benefits of IDA over meta-analysis:While potential more labor intensive, IDA has several advantages over meta-analysis. Benefits of meta-analysis over IDA:The major advantage of meta-analysis over IDA is that it allows for a much broader sampling of the research conducted on a given topic. Not all data from all available studies will be made available. Meta-analysis provides the ability to conduct a more comprehensive research synthesis. It is also more flexible, in that the results obtained via IDA could be incorporated into a meta-analysis. That is, it would be possible to obtain datasets from studies that would inform a meta-analysis but the results from that project were not reported in a way to make them usuable for a meta-analysis. One could conduct an IDA analysis from those projects and incorporate the effect size estimates into a large meta-analysis that also includes the effects obtained from published studies. How do I combine data from different studies?Instead of going out to collect these samples, researchers can combine data from existing studies. There are two main ways to combine existing data: through meta-analysis of summary statistics, and through Integrative Data Analysis using individual participant data.
What combines results of many studies?A meta-analysis is a statistical analysis that combines the results of multiple scientific studies. Meta-analyses can be performed when there are multiple scientific studies addressing the same question, with each individual study reporting measurements that are expected to have some degree of error.
Is a method for statistically combining the results of multiple studies?Meta-analysis is a statistical method to combine results of different studies, especially those with small sample size or with conflicting results. Meta-analysis is often an important component of systematic reviews.
Which type of study would combine results from multiple studies to come to an overall summary of how human health is affected?In many medical specialties, it is common to find that multiple studies have been conducted to answer similar questions about the effectiveness of a treatment, or the incidence or mortality rates for a particular disease, for example.
|