Whether you’re conducting an online survey for academic purposes or doing market research for a new business venture, your study’s success lies in adept data collection. But how does one navigate this crucial phase in a way that leads to meaningful insights? This article takes you through our top tips for data collection in online surveys.
Start by clearly defining your research question and strategizing how you’ll effectively measure each key variable. Take the following example:
Example:
You want to explore the impact of smoking cigarettes on athletic performance. The biggest question arises: How do you measure smoking habits among participants? Would a simple binary “Yes, I smoke” or “No, I don’t smoke” suffice? Would you ask participants how many cigarettes they smoke in a day? Perhaps you’d ask them to place their smoking habits on a scale of one to five – 1 being “I never smoke” and 5 being “I am a chain smoker”.
It’s important that the questions you pose prompt answers that truly relate to the behavior or event you’re investigating, and that it leads to accurate responses. For example, in our example study above, avoid asking participants if they believe smoking impacts their athleticism. Instead, ask precise questions that’ll lead you to your conclusion. Start by exploring their smoking habits, like how often and how frequently they smoke. Then inquire about their athletic activities and challenges. You could also include questions about their own views on how smoking impacts their abilities, keeping in mind that their answers may be biased and cannot be used as the sole indicator of how smoking affects athleticism.
Did you know? Depending on your analysis needs, you could opt for straightforward descriptive statistics (e.g. "30% of the respondents smoke daily, while 20% report athletic challenges"), which doesn’t require complex questioning as test statistics does. Test statistics involve exploring direct cause-and-effect relationships (e.g. answering the question “Does smoking significantly impact athletic performance?” as in the example study above).
Craft questions that are clear, concise, and easy to understand. Where possible, avoid open-ended questions and stick to closed-ended questions. Why? Because closed questions are much easier to analyze, saving you tons of time and effort when sorting through your data. In addition to this, avoid using jargon and ensure each question addresses a specific aspect of your research.
Example:
Vague question: How do you feel about your current job?
Improved question: On a scale of 1 to 10, how satisfied are you in your current job role?
Well-structured questions facilitate accurate and relevant responses, contributing to the overall quality of your survey data.
Conduct a pretest with a small group of participants before launching the survey to identify any design issues or confusion. Participant feedback during the pretest allows for necessary revisions, ensuring the survey is optimized for a larger audience.
How can you do a pretest?
Invite a small group of potential participants to go through the survey and provide their feedback. If participants find a question confusing or ambiguous, refine the wording or provide additional clarification before your survey gets launched.
Implement specific pre-screening questions or disqualifiers at the beginning of your survey to ensure that the respondents are relevant to your research objectives and eligible to take your survey. This step helps filter out individuals who do not meet the necessary qualifications or characteristics, ensuring that the collected data is targeted and meaningful.
Example:
You are conducting a survey on consumer preferences for a new line of fitness products in Germany. To pre-screen your participants effectively, include questions like:
List 10-12 countries, one of them being “Germany”. If a respondent indicates that they live in Germany, they qualify to continue the survey. Avoid posing yes/no questions when pre-screening (e.g. “Do you live in Germany?”), as this could result in you screening in bad actors, which defeats the purpose. Don’t make it too obvious from your title that you’re looking for German participants, either. A title such as “Understanding Consumer Preferences for Our New Product Line” is sufficient.
List 10-12 hobbies, with one of these hobbies being “fitness activities”. If a participant marks “fitness activities” as one of their hobbies, they qualify to continue the survey.
This question helps identify individuals who are actively involved in fitness activities, ensuring that your survey captures insights from the target audience.
This question checks whether respondents have recent experience with the fitness product market, ensuring they have the necessary qualifications to continue your survey.
Good to know: Pre-screening your participants not only ensures that respondents qualify to answer your survey, but it also helps save both your and the participant’s time by avoiding irrelevant or unqualified responses.
Include strategically-placed qualification and attention checks in your survey to assess participants’ eligibility and attentiveness. These quality checks filter out unreliable data, which maintains the validity of your results and encourages thoughtful participant engagement.
Attention Check Example:
Include a question like "To ensure you're paying attention, please select 'Strongly Agree' for this question" halfway through your survey. If a participant fails this attention check, it signals a lack of attentiveness and signals that the respondent’s answers may be unreliable.
Qualification Check Example:
If you’re looking for responses from full-time students, include a question like “How much time do you spend on academic activities in a typical week?” with answer options such as “>30 hours”, “20-30 hours”, “10-20 hours” and “”. This allows you to identify whether the participant is actually studying full time and whether they are representative of the typical full-time student or not.
Explore reputable survey exchanges and online panels to broaden your audience and reach niche demographics that might not be represented in your typical audience.
Professional Paid Panels:
Be sure to offer fair payment that’s in line with your desired residence country’s minimum wage requirements when making use of paid panels – offering a substandard payment will undoubtedly lead to sub-par responses. Also keep in mind that certain audiences might require higher remuneration than others. Someone who is very budget-focused might gladly spend their time answering a 20-minute survey for $5. A highly financially-successful individual, on the other hand, most likely would not. The more valuable the person’s time is, the more you’ll have to pay for their response.
Free Survey Exchanges:
Don’t have a budget available? Then a survey exchange platform is the way to go. Since these platforms operate on mutual exchanges, you’ll typically find they offer good data quality, since other users on the platform are in the same boat as you are; they need high quality answers to their surveys, so they’re likely to offer the same in return. Plus, most of these platforms have quality measures in place to ensure you get real, usable data.
Design a concise and engaging survey (keep it under 10 mins if possible) to prevent participant fatigue – something that could lead to participants providing incomplete answers or be discouraged to complete the survey at all. Focus on maintaining visual appeal, easy navigation, and participant interest throughout the survey.
How to keep respondents engaged:
Communicate clear instructions for survey completion, including explanations or examples where needed. Clear instructions reduce confusion, which ensures that participants understand what is needed of them, allowing them to provide accurate responses.
Example:
Vague instruction: What are your thoughts about psychologists and school?
Improved question: Do you believe that all schools should have an on-site psychologist? Please answer either “yes” or “no”, and in one sentence, provide a reason why.
Always adhere to ethical guidelines, prioritizing participant privacy and confidentiality. Obtain informed consent, and reassure participants that their responses will remain anonymous and be used for research purposes only. Respondents are more likely to provide honest and accurate answers if they know that their responses will remain confidential.
Pro Tip: If you need your respondents’ email addresses for optional participation in incentives or to send out optional follow-up info, only ask for this at the end of the survey. This way, after already taking the time to answer your questions, participants are more likely to provide you with their information and submit their answers. However, if you’re wanting to do a follow-up survey, it’s always best to disclose this at the start of your survey to manage expectations. If you need personal/identifiable information such as email addresses for follow-up studies, though, you should always include a consent form at the beginning of your survey.
Check your survey responses every now and then to spot any interesting patterns, inconsistencies, or outliers. Do this for the first 20 responses, and again at the halfway mark – that should be enough to identify any hiccups.
Time-saving Tip: For a quick preview, run a preliminary data analysis with just the initial 20-or-so respondents using tools like SPSS, R, or PowerBI. Save your syntax or analysis steps, and when your full dataset is in, simply use the same syntax to get your results straight away.
Your online survey’s success lies in meticulous planning and adherence to key principles. From defining clear objectives to implementing quality checks and ethical considerations, each step plays a crucial role in collecting both reliable and useful data. By employing the strategies set out in this guide, you're bound to gather meaningful insights from your survey. Happy surveying!
1. Should I pretest my survey before launch?
Yes! Pretesting your survey will help you to identify design issues and points of confusion, allowing for necessary revisions to be made before reaching your larger audience.
2. What are attention checks in surveys, and why are they important?
Attention checks are strategically placed questions used to filter out unreliable data by assessing respondents' attentiveness. These checks are important in maintaining data quality and thoughtful participant engagement.
3. How often should I monitor and analyze survey responses during data collection?
Check the first 20 responses, and monitor your data once more at the halfway mark. . This’ll help you to identify any patterns, inconsistencies, or outliers in survey responses, ensuring accurate and meaningful data analysis.
P.S. If you’re on the hunt for real responses from real people, SurveySwap’s the place to be. Sign up to start collecting your survey responses.