5 Questions to ask Before Selecting a Research Firm for a Phone Survey

Telephone surveys continue to be the most reliable source of public opinion data. Telephone surveys are invaluable tools for community and business leaders and serve as the basis for making data driven decisions. 

By asking these five questions, you will help ensure that your next telephone survey project provides data that is valid and reliable and that your research objectives are achieved.

1. Do you include cell phones in your sampling frame?
   Why this is important: When cell phones were first gaining popularity, several research studies comparing the population with landlines to the population using only cell phones were conducted and found that respondents answered questions similarly and therefore concluded that not including cell phones in a sampling frame was acceptable because, at the time, there was no evidence that there were differences in the populations. Times have changed: 25% of US households no longer have landlines. When you add in households that are use cell phones primarily and keep landlines for purposes other than engaging in conversations, the percentage jumps to 35%. This means that only 65% of households can be reached using traditional telephone survey methods- developing a sample frame based on residential landlines. In order to obtain a representative sample of the general public, it is now important to include a cell phone sample- if at all possible. While this may increase the cost of data collection, the data will be better quality and will yield more accurate results.
2. How do you keep up to date with new and emerging technologies in the field?
   Why this is important: There have been many important changes in the survey research industry over the past 30 years; there have been many important changes over the last 2 years. As technology and communication has changed over the last few years, so has the survey research industry. There are professional associations in the survey research industry such as the American Association for Public Opinion Research (AAPOR) that have a Code of Ethics that include things such as information that needs to be disclosed to potential respondents, keeping client data proprietary, and a commitment to use the methodology that is best suited to the research problem at hand. These organizations and industry publications keep research firms up to date with trends, emerging technology and new methodologies.
3. Do you use the same survey instrument for all of your clients?
   Why this is important: While some survey questions are appropriate to repeat across different surveys, there needs to be some degree of customization of each survey instrument. Every project and every client is different. While some organizations may have similar objectives, such as passing a levy, there are different questions that must be used to meet that objective based on the individuality of the client.
4. Will the final sample be representative of the population?
   Why this is important: The value of survey research findings depends heavily on the quality of the sample. It is important for a sample to be representative so that the views of the whole population are counted and not biased by one particular segment of the population. Many times when conducting telephone surveys, the final sample may under or over represent an important demographic area such as gender. Monitoring the data as it is collected allows a research firm to take measures to adapt the sampling frame if the sample is not matching the targeted population.
   
5. What type of on-going support do you supply after the report is delivered?
   Why this is important: There is always a chance that you may have a question about the data or report after the project is complete and the report is delivered. Or you may need data broken down in a different way or need to make a presentation to important stakeholders. What type of support does your research partner say they will provide after the report has been delivered? Is there an additional charge for this support? Will your research partner walk you through the report when it is delivered? Who within the company will you have access to? All important questions to consider BEFORE signing the initial contract.

3 Ways that Survey Research is Different from Telemarketing

Although there are distinct differences between survey research and telemarketing, people often use the terms interchangeably. Many times when I have told a person that my company conducts survey research, I hear, “Oh, you’re a telemarketer.” My answer is always the same: “No, we are not telemarketers and this is why. . .”

Below are 3 ways that Survey Research is different from Telemarketing:

1. Surveys researchers want to know your opinion. Survey researchers elicit information from the public. The information collected provides organizations with the public's opinions of products and services or social and political issues.
2. Telemarketers want to sell you something. Telemarketers solicit to raise funds, or sell products and services.
3. Survey research is exempt from the Do Not Call requirements, telemarketers are not. Telemarketers must abide by Federal regulations regarding "National Do Not Call", survey researchers do not.

In a nutshell, a survey researcher should never try to sell you on anything other than the importance of the research they are conducting. If they try to sell you a product or a service, they are not survey researchers, they are telemarketers. Telephone survey researchers and telemarketers both use telephones and need to have strong communication skills in order to be successful in their jobs. However, overall, they are very different.

3 Causes of Bad Data in Survey Research

Survey research is a way to learn about a large population by studying a small random sample of that population.  Survey Research is used for many reasons including reducing a company’s risk of making wrong decisions, providing companies with a “picture” of its clients or customers, and “weighs” the public’s perception of a particular issue.  In order to make data driven decisions, you first have to ensure that the data that you are basing your decisions on is both valid and reliable. 

Below are three common sources of bad data and how you can avoid them. 

1. Sampling Bias- A good sample is one that is representative of the population as a whole. For example, if you are doing a survey of Stark County adults and you know that 48% of Stark County adults are women, then a good final sample will be 48% women. Sampling bias is introduced when a sample does not accurately represent the population. An example of sampling bias is undercoverage.  This occurs when some members of the population (like young adults or minorities) are underrepresented in a sample. How to avoid sampling bias: Monitor your sample demographics throughout the data collection process so that you can ensure that the final sample is representative. 
    
2. Questionnaire flaws- A questionnaire is a series of questions asked to your sample to obtain information about a particular topic. How a questionnaire is designed can have an effect on the quality of data that is collected. There are many, many ways that a questionnaire can be flawed. Some of the most common flaws include questions that are written above or below the knowledge level of the sample, leading or biased questions, double-barreled questions, long questions with a long list of response choices, and questions with response choices that are not mutually exclusive. How to avoid questionnaire flaws: Make sure that your final questionnaire is short, clear and understandable to your target sample. There are many resources available on-line and at the local library on how to write a good survey instrument. Also, it is important to pre-test your survey instrument prior to full implementation.
   
   
3. The Interviewer Effect- The Interviewer Effect is the variation in answers associated with the person conducting the interview. The Interviewer Effect can happen when interviewers do not read questions thoroughly, direct respondents to certain answers, or record answers incorrectly. How to avoid the Interviewer Effect: Make sure that your interviewers are properly trained on the importance of consistency, standardization, and neutrality.

Remember, if you are trying to make data-driven decisions, you must make sure that you are not using bad data. Making decisions using bad data is many times worse than making decisions on no data at all.

St. Patrick's Day by the Numbers

Top of the morning to you! St. Patrick’s Day is celebrated each year on March 17^th, which is date of the death of St. Patrick. For those of you who don’t know, St. Patrick is the person who is credited with bringing Christianity to Ireland. Many people wear green on St. Patrick’s Day. The green represents spring, shamrocks, and Ireland.

In the spirit of the holiday, below are some interesting and surprising numbers associated with St. Patrick’s Day.

• 4.2 Million- The total population of Ireland·         
• 36.9 Million- Number of U.S. residents who claimed Irish ancestry in 2009.
• $4 Billion- The amount that Americans spend on St. Patrick’s Day each year.
• 46%- Percent of adults who plan on celebrating St. Patrick’s Day.
• 20%- Percent of adults who plan on having a drink to celebrate.
• 100 pounds- Pounds of green dye added to the Chicago River in 1962
• 1 out of every 10,000- The chance of a clover having 4-leafs instead of 3.
• 1976- The first year that St. Patrick’s Day was officially recognized.  

What are you doing to celebrate St. Patrick’s Day this year?

IS EMPLOYMENT FINALLY ON THE UPSWING?

Throughout the Northeast Ohio region, residents have a negative opinion of job opportunities in their county, although the levels varied significantly by county. Stark County had the highest negative rating with 62% rating the availability of job opportunities as poor or very poor. The negative rating is also alarmingly high in Cuyahoga and Summit Counties with more than half of respondents in each county rating the availability of job opportunities negatively.  Medina County had the lowest negative rating with just 26% rating job opportunities negatively. 

When looking at how the rating of the availability of job opportunities has changed over the past five years, some clear trends emerge. All three counties had a drop in favorable rating between 2009 and 2010, with 2010 being the lowest rating in the past five years in all counties. Since 2010, the ratings in all three counties has slowly started to rise again, albeit at a much slower rate than the rate at which they dropped in preceding years.

Note: Cuyahoga is not included in this chart due to only having 1 year of data. Most recent Medina County data is from January 2012 (not 2011).

County unemployment rates show similar trends as the data from the collaborative polls.  Medina County has the lowest unemployment rate for all of the past five years while Stark County as had the highest. In all four counties, the unemployment rate rose steadily between 2007 and 2009, remained mostly stagnant between 2009 and 2010, and started to decrease between 2010 and 2011.

¹Source: Ohio Labor Market  

The Collaborative Poll program is a research service offered by CMOR to organizations, agencies and businesses with a county-wide interest as an affordable method of collecting information on opinions & attitudes of county residents. A Collaborative Poll is an annual telephone survey that includes questions posed by multiple organizations in which each participant pays for only those questions necessary to meet their information and research needs.

 

The Center for Marketing & Opinion Research (CMOR) is a full service public opinion research firm located in the Portage Lakes area providing innovative research solutions to non-profits, government agencies, social service organizations, colleges and universities, and hospitals. CMOR utilizes multiple modes of data collection including telephone, web and mail surveys and focus group administration as well as offering comprehensive consulting services including consultation on research and survey design, program evaluation, grant writing, data management, analysis and report writing.