CaRT satisfaction survey methodology

Published: Friday, 27 December 2019

I HAVE been involved in statistics environment in a number of ways over the years, writes Bill Ridgeway.

One job in the Civil Service was to collect national statistics of the catering trade (which included cafes, fish and chips shops, pubs, restaurants and staff canteens).  I have also been involved in visiting respondents for a series of social surveys (for which training was required). I am not a statistician and, therefore, I can speak only from experience not training.

Design a sampling frame

One of the first tasks in a survey is to design a sampling frame which determines the smallest number of respondents by type to ensure a fair representation across the country.  Another initial job is to design the questionnaire.  This requires a great amount of thought to ensure the questions are unambiguous and unbiased.

Yet another job is to build a system of issuing forms, receiving returned forms and recording information and statistical analysis.  Only then can the forms be issued.  Even in a very small scale enquiry (such as the CaRT survey) it is not as simple as sending out someone with a clip-board of questions.

The concept of GIGO

Designers of a survey (of whatever size) should have in mind the concept of GIGO (Garbage In, Garbage Out) in which the quality of output is determined by the quality of the input.  For example, the order of questions, the actual words and delivery of questions (whether on paper or orally) and environment can bias the outcome of the survey rendering it useless.

I wonder how much thought went into the methodology of the CaRT satisfaction survey?