3 Causes of Bad Data in Survey Research

Survey research is a way to learn about a large population by studying a small random sample of that population.  Survey Research is used for many reasons including reducing a company’s risk of making wrong decisions, providing companies with a “picture” of its clients or customers, and “weighs” the public’s perception of a particular issue.  In order to make data driven decisions, you first have to ensure that the data that you are basing your decisions on is both valid and reliable. 

Below are three common sources of bad data and how you can avoid them. 

1. Sampling Bias- A good sample is one that is representative of the population as a whole. For example, if you are doing a survey of Stark County adults and you know that 48% of Stark County adults are women, then a good final sample will be 48% women. Sampling bias is introduced when a sample does not accurately represent the population. An example of sampling bias is undercoverage.  This occurs when some members of the population (like young adults or minorities) are underrepresented in a sample. How to avoid sampling bias: Monitor your sample demographics throughout the data collection process so that you can ensure that the final sample is representative. 
    
2. Questionnaire flaws- A questionnaire is a series of questions asked to your sample to obtain information about a particular topic. How a questionnaire is designed can have an effect on the quality of data that is collected. There are many, many ways that a questionnaire can be flawed. Some of the most common flaws include questions that are written above or below the knowledge level of the sample, leading or biased questions, double-barreled questions, long questions with a long list of response choices, and questions with response choices that are not mutually exclusive. How to avoid questionnaire flaws: Make sure that your final questionnaire is short, clear and understandable to your target sample. There are many resources available on-line and at the local library on how to write a good survey instrument. Also, it is important to pre-test your survey instrument prior to full implementation.
   
   
3. The Interviewer Effect- The Interviewer Effect is the variation in answers associated with the person conducting the interview. The Interviewer Effect can happen when interviewers do not read questions thoroughly, direct respondents to certain answers, or record answers incorrectly. How to avoid the Interviewer Effect: Make sure that your interviewers are properly trained on the importance of consistency, standardization, and neutrality.

Remember, if you are trying to make data-driven decisions, you must make sure that you are not using bad data. Making decisions using bad data is many times worse than making decisions on no data at all.

St. Patrick's Day by the Numbers

Top of the morning to you! St. Patrick’s Day is celebrated each year on March 17^th, which is date of the death of St. Patrick. For those of you who don’t know, St. Patrick is the person who is credited with bringing Christianity to Ireland. Many people wear green on St. Patrick’s Day. The green represents spring, shamrocks, and Ireland.

In the spirit of the holiday, below are some interesting and surprising numbers associated with St. Patrick’s Day.

• 4.2 Million- The total population of Ireland·         
• 36.9 Million- Number of U.S. residents who claimed Irish ancestry in 2009.
• $4 Billion- The amount that Americans spend on St. Patrick’s Day each year.
• 46%- Percent of adults who plan on celebrating St. Patrick’s Day.
• 20%- Percent of adults who plan on having a drink to celebrate.
• 100 pounds- Pounds of green dye added to the Chicago River in 1962
• 1 out of every 10,000- The chance of a clover having 4-leafs instead of 3.
• 1976- The first year that St. Patrick’s Day was officially recognized.  

What are you doing to celebrate St. Patrick’s Day this year?

IS EMPLOYMENT FINALLY ON THE UPSWING?

Throughout the Northeast Ohio region, residents have a negative opinion of job opportunities in their county, although the levels varied significantly by county. Stark County had the highest negative rating with 62% rating the availability of job opportunities as poor or very poor. The negative rating is also alarmingly high in Cuyahoga and Summit Counties with more than half of respondents in each county rating the availability of job opportunities negatively.  Medina County had the lowest negative rating with just 26% rating job opportunities negatively. 

When looking at how the rating of the availability of job opportunities has changed over the past five years, some clear trends emerge. All three counties had a drop in favorable rating between 2009 and 2010, with 2010 being the lowest rating in the past five years in all counties. Since 2010, the ratings in all three counties has slowly started to rise again, albeit at a much slower rate than the rate at which they dropped in preceding years.

Note: Cuyahoga is not included in this chart due to only having 1 year of data. Most recent Medina County data is from January 2012 (not 2011).

County unemployment rates show similar trends as the data from the collaborative polls.  Medina County has the lowest unemployment rate for all of the past five years while Stark County as had the highest. In all four counties, the unemployment rate rose steadily between 2007 and 2009, remained mostly stagnant between 2009 and 2010, and started to decrease between 2010 and 2011.

¹Source: Ohio Labor Market  

The Collaborative Poll program is a research service offered by CMOR to organizations, agencies and businesses with a county-wide interest as an affordable method of collecting information on opinions & attitudes of county residents. A Collaborative Poll is an annual telephone survey that includes questions posed by multiple organizations in which each participant pays for only those questions necessary to meet their information and research needs.

 

The Center for Marketing & Opinion Research (CMOR) is a full service public opinion research firm located in the Portage Lakes area providing innovative research solutions to non-profits, government agencies, social service organizations, colleges and universities, and hospitals. CMOR utilizes multiple modes of data collection including telephone, web and mail surveys and focus group administration as well as offering comprehensive consulting services including consultation on research and survey design, program evaluation, grant writing, data management, analysis and report writing.

QUALITY OF HEALTHCARE IN REGION

Throughout the Northeast Ohio region, residents have a favorable opinion of the quality of healthcare services available in their county, although the level of favorability varies significantly by county. Medina County had the highest favorability rating with 88% rating the quality of healthcare in the county as excellent or good. This is higher than Summit County (home of Summa Health System, Akron General Medical Center, and Akron Children’s Hospital), Cuyahoga County (home of the Cleveland Clinic and University Hospitals), and Stark County (home of Aultman Hospital and Mercy Medical Center). While favorability rates in Summit and Stark Counties have remained largely unchanged over the past five years, favorability ratings in Medina County have increased from year to year from 80% in 2007 to 88% in 2011. 

Note: Cuyahoga is not included in this chart due to only having 1 year of data. Most recent *Medina County data is from January 2012 (not 2011).

Data referenced in this post was collected as part of the collaborative poll program at the Center for Marketing and Opinion research (CMOR). The Collaborative Poll program is a research service offered by CMOR to organizations, agencies and businesses with a county-wide interest as an affordable method of collecting information on opinions & attitudes of county residents. A Collaborative Poll is an annual telephone survey that includes questions posed by multiple organizations in which each participant pays for only those questions necessary to meet their information and research needs.

The Center for Marketing & Opinion Research (CMOR) is a full service public opinion research firm located in the Portage Lakes area providing innovative research solutions to non-profits, government agencies, social service organizations, colleges and universities, and hospitals. CMOR utilizes multiple modes of data collection including telephone, web and mail surveys and focus group administration as well as offering comprehensive consulting services including consultation on research and survey design, program evaluation, grant writing, data management, analysis and report writing.