How does increasing the level of confidence affect the size of the margin of error, e?

Remember that there is variability associated with your outcomes and statistics.

When you calculate a statistic based on your sample data, how do you know if the statistic truly represents your population? Even if you've selected a random sample, your sample will not completely reflect your population. Each sample you take will give you a different result.

Let's Look at an Example:

Suppose that you want to compare the mean age for those with and without an IV in the prehospital setting. You review the ambulance runs for the past two weeks and calculate a mean age of 10.4 years for those with an IV and 8.5 years for those without an IV. The difference between the two means is 1.9 years. From this, you might conclude that those receiving an IV were older on average.

Suddenly, it's not clear that there's an important difference in age between these two groups. Now suppose you collect the same data over the next six weeks. This time the average age for those with an IV is 9.2 years and the average age for those without an IV is 8.9 years, for a difference of 0.3 years. Suddenly, it's not clear that there's an important difference in age between these two groups. Why did your different samples yield different results? Is one sample more correct than the other?

Remember that there is variability in your outcomes and statistics. The more individual variation you see in your outcome, the less confidence you have in your statistics. In addition, the smaller your sample size, the less comfortable you can be asserting that the statistics you calculate are representative of your population.

Providing a Range of Values

A confidence interval provides a range of values that will capture the true population value a certain percentage of the time. You determine the level of confidence, but it is generally set at 90%, 95%, or 99%. Confidence intervals use the variability of your data to assess the precision or accuracy of your estimated statistics. You can use confidence intervals to describe a single group or to compare two groups. We will not cover the statistical equations for a confidence interval here, but we will discuss several examples.

Example
  • Average pulse rate = 101 bpm; Standard Deviation = 50; N = 200
  • 95% Confidence Interval = (94, 108)
    We are 95% confident that the true pulse rate for our population is between 94 and 108.
    Margin of error = (108 – 94) / 2 = ± 7 bpm

The confidence interval in the above example could be described at 94 to 108 bpm (beats per minute) or 101 bpm ± 7 bpm. Here the number 7 is your margin of error. For confidence intervals around the mean, the margin of error is just half of your total confidence interval width.

Sample Size and Variability

The precision of your statistics depends on your sample size and variability. A larger sample size or lower variability will result in a tighter confidence interval with a smaller margin of error. A smaller sample size or a higher variability will result in a wider confidence interval with a larger margin of error. The level of confidence also affects the interval width. If you want a higher level of confidence, that interval will not be as tight. A tight interval at 95% or higher confidence is ideal.

Examples:
  • Average Scene Time = 5.5. mins; Standard Deviation = 3 mins; N = 10 runs
  • 95% Confidence Interval = (3.6, 7.4)
    Margin of Error = ±1.9 minutes
  • Average Scene Time = 5.5 mins; Standard Deviation = 3 mins; N=1,000 runs
  • 95% Confidence Interval = (5.4, 5.6)
    Margin of Error = ± 0.1 minutes
  • Average Scene Time = 5.5 mins; Standard Deviation = 15 mins; N=1,000 runs
  • 95% Confidence Interval = (4.6, 6.4)
    Margin of Error = ± 0.9 minutes

How does increasing the level of confidence affect the size of the margin of error, e?

rev. 04-Aug-2022

Statistics 101
  Data Analysis and Statistical Inference

In-class problems on confidence intervals
 



Answers to conceptual questions on confidence intervals

Decide whether the following statements are true or false.  Explain your reasoning.

Problems:

a)  For a given standard error, lower confidence levels produce wider confidence intervals.

False.   To get higher confidence, we need to make the interval wider interval.  This is evident in the multiplier, which increases with confidence level.

b)  If you increase sample size, the width of confidence intervals will increase.

False.   Increasing the sample size decreases the width of confidence intervals, because it decreases the standard error.

c)  The statement, "the 95% confidence interval for the population mean is (350, 400)", is equivalent to the statement, "there is a 95% probability that the population mean is between 350 and 400".

False.   95% confidence means that we used a procedure that works 95% of the time to get this interval.  That is, 95% of all intervals produced by the procedure will contain their corresponding parameters.  For any one particular interval, the true population percentage is either inside the interval or outside  the interval.  In this case, it is either in between 350 and 400, or it is not in between 350 and 400.  Hence, the probabliity that the population percentage is in between those two exact numbers is either zero or one.

d)  To reduce the width of a confidence interval by a factor of two (i.e., in half), you have to quadruple the sample size.

True, as long as we're talking about a CI for a population percentage.   The standard error for a population percentage has the square root of  the sample size in the denominator.  Hence, increasing the sample size by a factor of 4 (i.e., multiplying it by 4) is equivalent to multiplying the standard error by 1/2.  Hence, the interval will be half as wide.  This also works approximately for population averages as long as the multiplier from the t-curve doesn't change much when increasing the sample size (which it won't if the original sample size is large).

e)  Assuming the central limit theorem applies, confidence intervals are always valid.

By "valid," we mean that the confidence interval procedure has a 95% chance of producing an interval that contains the population parameter.

False.  The central limit theorem is needed for confidence intervals to be valid.   However, it is also necessary that the data be collected from random samples.  Confidence intervals will not remedy poorly collected data.

f)  The statement, "the 95% confidence interval for the population mean is (350, 400)" means that 95% of the population values are between 350 and 400.

False.  The confidence interval is a range of plausible values for the population average.   It does not provide a range for 95% of the data values from the population.  To find the percentage of values in the population between 350 and 400, we need to look at a histogram of the data values and determine what percentage of observations are between 350 and 400.

g)  If you take large random samples over and over again from the same population, and make 95% confidence intervals for the population average, about 95% of the intervals should contain the population average.

True.   This is the definition of confidence intervals.

h)  If you take large random samples over and over again from the same population, and make 95% confidence intervals for the population average, about 95% of the intervals should contain the sample average.

False.   The confidence interval is a range for the population average, not for the sample average.  In fact, every confidence interval contains its corresponding sample average, because CIs are of the form:  sample avg. +/- multiplier SE.  So, the sample average is right in the middle of the CI.

i)   It is necessary that the distribution of the variable of interest follows a normal curve.

False.   It is necessary that the distribution of the sample average follows a normal curve.  The data values of the variable, however, need not follow a normal curve, because if the sample size is large enough the central limit theorem for the sample average will apply.

j)  A 95% confidence interval obtained from a random sample of 1000 people has a better chance of containing the population percentage than a 95% confidence interval obtained from a random sample of 500 people.

False.  All 95% confidence intervals have the property that they come from a procedure that has a 95% chance of yielding an interval that contains the true value.   The confidence interval method automatically accounts for sample size in the standard error.   A 95% CI with n=1000 will be narrower than a 95% CI with n=500, but both CIs will have 95% confidence of containing the population percentage.

k)  If you make go through life making 99% confidence intervals for all sorts of population means, about 1% of the time the intervals won't cover their respective population means.

True.  Since 99% of the intervals should contain the corresponding population mean, 1% of them will not.


How does the level of confidence and sample size affect the margin of error E?

Answer: As sample size increases, the margin of error decreases. As the variability in the population increases, the margin of error increases. As the confidence level increases, the margin of error increases.

Why does higher confidence mean higher margin of error?

As the confidence level goes up the left endpoints become smaller and the right denpoints become larger. That means the widths of the intervals become larger as the confidence level increases. Since the margin of error is half the width of the interval, the margin of error increases as the level of confidence goes up.