Survey advice for potential researchers?

[IaninSheffield]

Hi all,

CC folks tend to be generous coves who are more than willing to help out a researcher trying to improve cycling. Unfortunately, as we’ve all doubtless experienced, those questionnaires/surveys are sometimes poorly designed. We regularly get surveys submitted by GCSE students who are clearly keen on their subject, but understandably less familiar or experienced in survey design.

I wondered if it might be helpful to knock together some suggestions for those intending asking CC for help with their surveys? Perhaps mods might consider pinning the result at the top of this sub?

I’ll offer the starting point below, but acknowledge that I’m likely to have missed things. If there are other bits of advice you think should be added, or if there’s anything that should be omitted or edited, please chime in. It's clearly not exhaustive, but of course if it gets too long, folks are less likely to read it.
Or maybe this is a complete non-starter? Any thoughts?


Thanks for reaching out to CycleChat as a place to ask for help with your research. As you might appreciate, members on this forum are often asked their opinion about cycling related matters, and whilst they are keen to help, unfortunately some surveys are less well thought out and therefore difficult to answer.

We appreciate that some researchers may have less experience in designing surveys, so we’d like to offer the following suggestions you may wish to consider.

1. Background - who are you, what is your background, what problem is your survey aiming to resolve, and why is it important that CycleChat members should respond? For example:

   I’m Alex and am conducting research into breathable cycle clothing as part of my GCSE Design and Technology course. Although some garments are described as ‘breathable’, they still trap moisture when the wearer is exercising hard. I’m aiming to design features in a cycling jacket which allow moisture to escape through methods other than ‘wicking’. By responding you will be contributing to research seeking to produce better products for cyclists … and helping me to a better grade in my GCSE.

2. Who is your target audience? Bear in mind that most, though not all, CycleChat members live in the UK. Are your questions relevant to this context? For example, if you’re designing a product and would like to find out how much people might be willing to pay, questions with £ rather than $ make more sense.
3. A poorly designed survey is likely to produce poor data. Furthermore, poorly chosen questions may also frustrate your respondents, causing them to drop out before completing. There is plenty of advice online to help you avoid these issues; here are some points to get you started.
4. Have you set questions which “require an answer” before allowing respondents to move on? Sometimes this is important to your survey, but will also mean people can’t skip questions and again might cause them to drop out of the survey prematurely.
5. Have you ‘piloted’ your survey? In other words, have you had some people try it out and test your questions? Doing this might help identify any problems which need addressing before you need to start collecting ‘real’ data.
6. Feedback for you. After posting a link to your survey, be aware that members might have a question for you. Do please return and provide answers when needed. When people are comfortable their queries are being addressed, they are more likely to feel positively towards your survey and your response rate will improve.
7. Feedback for us. When your survey is complete, it is only polite to pop back and provide a brief summary of what you found. If members responded to the survey, it’s likely they would be interested in the results.

Thanks for reading and we look forward to answering your survey.

 

[Faratid]

I'm surprised at the consistently poor quality of surveys and questionnaires posted here. It's hard to believe that these students are having anything in the way of project supervision, tutoring, or even being taught the basics of research methodology and questionnaire design. To be fair, the same applies to your average questionnaire from various companies, such as energy providers.
None of these questionnaires would be considered valid by any academic researcher; you can't just sit down and think "what do I want to find out about xxxx?" Questionnaire design, such as a semantic differential, has to be based on content analysis of structured interviews with the relevant cohort, from which the subject of the questions is derived.

 

[matticus]

Perhaps they have a go first without any training - that's what we see.

Then when they prepare properly, one aspect is that some channels are better than others ...

 

[HobbesOnTour]

@IaninSheffield I admire your dedication to helping the students (and others), although I think you're overly optimistic!

I don't like to be too critical of the students because putting a questionnaire together is bloody hard! And a good one? Harder still.

I do know other fora require surveys to be preapproved, but that's a lot of work for mods.

I will say that I like how posts always get responses, even if not to the liking of the ops. It shows the forum in a positive light, I think.

 

[snorri]

Many of the surveys appear to have been submitted by pupils/students with little knowledge of cycling or cyclists, or the compilation of surveys. I think it's up to their teachers/tutors to advise them and assist those who have achieved little from their survey results.
Surveyors often show a lack of awareness of the many facets of cycling, eg leisure, off road, commuting, touring, competitive etc. and that cyclists may engage in more than one of these activities, and have more than one bicycle!

 

[IaninSheffield]

Nice idea, but I suspect most surveyors just create an a/c, post their survey, don't read anything in the forum, go away and never come back.

I hear ya ... but some might

 

[IaninSheffield]

I'm surprised at the consistently poor quality of surveys and questionnaires posted here. It's hard to believe that these students are having anything in the way of project supervision, tutoring, or even being taught the basics of research methodology and questionnaire design. To be fair, the same applies to your average questionnaire from various companies, such as energy providers.
None of these questionnaires would be considered valid by any academic researcher; you can't just sit down and think "what do I want to find out about xxxx?" Questionnaire design, such as a semantic differential, has to be based on content analysis of structured interviews with the relevant cohort, from which the subject of the questions is derived.

True, but would you expect the average 15 year old to understand "content analysis of structured interviews with the relevant cohort, from which the subject of the questions is derived."?
Maybe they'd appreciate a little help?

 

[IaninSheffield]

I think it's up to their teachers/tutors to advise them and assist those who have achieved little from their survey results.

It is, but I'm sure they wouldn't turn down a little help from us ... especially in the current climate.

 

[HobbesOnTour]

It is, but I'm sure they wouldn't turn down a little help from us ... especially in the current climate.

I agree 100% but I'd imagine the questionnaire is well completed before they come here.

 

[Ming the Merciless]

@IaninSheffield Maybe time to become an influencer, fly to Dubai , and post some videos wearing a mankini on the beach , explaining how to do surveys?

 

[IaninSheffield]

@IaninSheffield Maybe time to become an influencer, fly to Dubai , and post some videos wearing a mankini on the beach , explaining how to do surveys?

Ooo, now there's an idea. I have a nice gold lamé number I haven't worn since lockdown began.
[begins searching Covid entry requirements for UAE & flights to Dubai]

 

[sheddy]

Researcher: What am I trying to measure ?

 

[slowmotion]

I was good at passing exams, but when I look back on my school days I shudder at how little I knew about the real world and how to deal with people. Quite a few of the surveys are posted by school children. Just give them a break FFS. It must be hugely dispiriting for the poster to get all the "it's a crap survey" comments. We were all crap once.