Margin of error: How is it calculated in a survey?

The margin of error is something that is not always mentioned when explaining the results of a survey, yet it is crucially important for a good reading of the data. But what is sampling error, why is it important, and how is it calculated?

What is the margin of error in a survey?

The margin of error is a basic concept in market research because it helps to understand how accurate the survey results will be.

As you know, surveys work with samples that represent the study populations. Interviewing everyone in the universe is generally not economically feasible, but with a sufficient and well-selected sample, the results will be fairly close to what they would be if we had interviewed the entire population.

The margin of error indicates the range within which the actual results are expected to fall. Roughly speaking, if a survey says that 50% of consumers prefer a product with a 3% margin of error, that means that in reality between 47% and 53% of consumers (the study universe) prefer that product. The smaller the margin of error, the closer the sample responses will be to what we would have obtained by interviewing the entire universe.

Why is the margin of error important?

When reading the results of a survey, it is important to know the margin of error. This is why it should always be indicated both in the proposals and in the fact sheets of the studies. The reason is that with a known margin of error, the quality and representativeness of the survey sample can be assessed.

The margin of error is particularly important when assessing significant differences. In the comparison of different market segments or between different options, the margin of error helps to determine whether the observed differences in the data are large enough to know that they are indeed different values, or whether they could be due to chance. For example, if two products have a difference in preference of 2%, but the margin of error is 5%, that difference would not be considered significant.

For these reasons, it is important to always take this into account when making decisions based on the data. A smaller margin of error generally indicates a larger and more representative sample, which gives more confidence in the results to make decisions based on them.

What does the margin of error depend on?

The margin of sampling error depends on several key factors:

• Sample Size (n): as the sample size increases, the estimate becomes more precise. Therefore, it seems that the best way to avoid risk is to have as large a sample as possible. But be careful, because this would increase the costs of the study. Moreover, adding more sample is increasingly inefficient. Above a certain level, reducing the margin of error becomes more and more expensive. That is why we always look for an appropriate balance that allows us to make decisions at reasonable costs.
• Standard deviation (σ or p): is a statistical measure that quantifies the dispersion or variability of a set of data in relation to its mean (average). It indicates how much the data, on average, deviate from the mean of the set. If the data in the population are highly varied, it is more difficult to make precise estimates. The larger the standard deviation, the greater the margin of error.
• Confidence level (Z): probability that the confidence interval includes in repeated sampling the true value that would be obtained by interviewing the entire population. It is commonly expressed as a percentage (e.g. 90%, 95%, 99%). For example, a 95% confidence level indicates that if 100 different samples were taken and confidence intervals were calculated for each sample, approximately 95 of those intervals would be expected to contain the true value that would result from interviewing the entire population. The higher the level of confidence we wish to have, the higher the margin of error.

The confidence level is related to the ‘critical Z-value’. It is a value of the standard normal distribution that corresponds to a specific confidence level. Common confidence levels and their corresponding Z-values are:

• 90% Confidence Level: Z ≈ 1.645
• 95% Confidence Level: Z ≈ 1.96
• 99% Confidence Level: Z ≈ 2.576

These Z values are obtained from the standard normal distribution tables and correspond to the points that leave a specific area at the extremes of the distribution. For example, for a 95% confidence level, the Z-value of 1.96 leaves 2.5% at each tail of the distribution (totalling 5%).

How is the margin of error of a sample calculated?

The margin of error is calculated from the sample size and the variability of the data. Its basic formula is:

Where:

• Z is the critical z-value corresponding to the desired confidence level (e.g. 1.96 for a 95% confidence level).
• σ is the population standard deviation.
• n is the sample size.

Now that we know more details, a more accurate reading example would be this: if a survey indicates that 60% of respondents prefer a product with a margin of error of ±3% at the 95% confidence level, this means that if the survey were repeated multiple times, the actual result would be between 57% and 63% 95% of the time.

What sample size do you need?

Calculating the sampling error helps you to know how many respondents you will need to have data you can trust. If you like, you can find more details in our article on how to calculate the sample size of a market research study. Or if you prefer, get in touch with our research experts who will help you make the best decisions so that you have the optimal sample base to make confident decisions.