• Our booking engine at tickets.railforums.co.uk (powered by TrainSplit) helps support the running of the forum with every ticket purchase! Find out more and ask any questions/give us feedback in this thread!

Post Graduate Research... Can you Help?

Status
Not open for further replies.

DaveNewcastle

Established Member
Joined
21 Dec 2007
Messages
7,387
Location
Newcastle (unless I'm out)
. . . . the omission of a "None" button will lead to highly false results as respondents have to put something and introduces the coin toss problem and a massive potential shift within the results, as coin toss theory rarely holds, resulting in a large skew of data and wasting a lot of your time later.

Ie. A respondent wants to put, "None of the Above" or "Change Nothing" but without being provided that button, will randomly stab any buttons to complete the survey to carry out their moral obligation to do so and/or gain a sense of satisfaction for helping you by completing the questionnaire; this will happen a lot more than you think, resulting in a massive bias and utterly useless results, that WILL be shot down by anyone reading the report that has any sense at all between their ears.
Exactly!
The results WILL be meaningless for that very reason.

A little more intellectual rigour is required to render the results of an open ended survey meaningful.

A broader cross-section of respondents would help too, but one particular sub-section would seem to be necessary to provide useful outcomes: i.e. on-board surveys of the First Class passengers on a few long-distance TOCs whose preferred style of travel is ostensibly being analysed.
Without asking the users, and identifying their/our responses, then we're not revealing the opinions of the people whose market choices would be affected by the proposals.

But as Nym says, the biggest distortion of any survey dataset is the lack of a 'none of the above' option. Basic stuff.
 
Sponsor Post - registered members do not see these adverts; click here to register, or click here to log in
R

RailUK Forums

Nym

Established Member
Joined
2 Mar 2007
Messages
9,173
Location
Somewhere, not in London
There are so many much 'better' ways to have this survey completed, most don't involve an open on-line survey, the most simple to implement being the most obvious, speak to passengers...

Psudoqualitative quantitative research at the beginning of research for a proposal is never really a good idea, unless it's a very simple set of questions, something like this needs to be purely qualitative in the initial stages, there are countless avenues to find such information, as I'm sure some readers of this forum know. This qualitative research can yield a very good indication of further avenues of potential pursuit.

One of Dave's suggestions is where I would probably be starting, I'd also be speaking to pax on other modes of transport to see what they would like to see...

Also, one would think it's something very basic, but evidently not...
 

bb21

Emeritus Moderator
Joined
4 Feb 2010
Messages
24,151
Contrary to some of the opinions above, I do not believe the results are totally meaningless without the inclusion of the "None of the above" option.

The question is worded in such a way as to guaging relative popularity of the three possible developments suggested. (Here I assume that the wording of the question has not been changed.) The results are in no way an indication as to whether people prefer things to change, so the inclusion of that fourth option is irrelevant.

If these results are to be presented in support of a change in train design then further research would need to be carried out, however without knowing the purpose of this questionnaire, I hesitate to claim that the results are false or misleading.
 

DaveNewcastle

Established Member
Joined
21 Dec 2007
Messages
7,387
Location
Newcastle (unless I'm out)
The question is worded in such a way as to guaging relative popularity of the three possible developments suggested. (Here I assume that the wording of the question has not been changed.) The results are in no way an indication as to whether people prefer things to change, so the inclusion of that fourth option is irrelevant.
Its interesting that you have come to that conclusion. Clearly I didn't.

I have just revisited the survey, and will also assume that the wording has not been changed. Doing so reminded me that I was already frustrated by the third question: "Which of the following best describes your use for the Railway Service?" whose 3 permitted responses were already so constrained to exclude my accurate response!
But on to the crucial Q.4:
"This section is aimed to find out what you, as a potential passenger of the railway service would most like to see in the future.

Please read the following three scenarios carefully and choose which you would be most interested in.
"
We've established that the OP has not provided the "none of the above" possible response.
It is not an option.
And I detest all three proposals.
So what do we lack in order to provide meaningful responses?

The easy answer is: none of the above.
The more sophisticated, but easliy analysed answer comes from 2 questions or from further options: I'd suggest firstly an indicator of the respondent's wish for change / or not; then either, the 3 abitrary pre-concieved suggestions with a 'none of the above' option, or, perhaps more usefully, probing to determine which elements of the offerings the repondent was more or less amenable to changing without impacting negatively on their demand for the service.

As it stands, there is no reponse I can give which indicates that all 3 proposals would reduce my own demand for rail travel, and I am reasonably confident that the OP would want to quantify that loss of revenue if the results were to be of any value in determining future rail travel facilities.
 

bb21

Emeritus Moderator
Joined
4 Feb 2010
Messages
24,151
The more sophisticated, but easliy analysed answer comes from 2 questions or from further options: I'd suggest firstly an indicator of the respondent's wish for change / or not; then either, the 3 abitrary pre-concieved suggestions with a 'none of the above' option, or, perhaps more usefully, probing to determine which elements of the offerings the repondent was more or less amenable to changing without impacting negatively on their demand for the service.

Precisely. What has been suggested before was that the question be expanded to account for all possible scenarios, which really comprise of two parts, the first of which missing as the emphasis of the research could be to compare amongst the alternatives as opposed to against the status quo.

Inclusion of "None of the above" in the existing manner brings its own problems as it would be impossible to distinguish those who do not wish to see any change and those who wish to see changes but not any of the three options originally displayed, for example.

Anyway I think I risk going completely off-topic. :)
 

Nym

Established Member
Joined
2 Mar 2007
Messages
9,173
Location
Somewhere, not in London
bb21 is just enforcing the "Poorly thought out" implications of my previous posts there, and thanks for noting that, there should have been at least two additional options.

"I would like changes, but not any of those you have provided."
"I would not like any changes"
"I would not like to answer this question"

With the addition of a box for elaboration by qualitative data would have been nice...
 

michael769

Established Member
Joined
9 Oct 2005
Messages
2,006
I note that when I tried the survey just now I was offered a "None of the Above" option.
 
Status
Not open for further replies.

Top