Monday, July 19, 2010

Top 5 mistakes that people make with MR

1) Over sampling or under sampling – This is the biggest mistake people make with research ie., have sample sizes which are not robust enough or are hugely inefficient. While 2 FGDs are not enough to conclude on anything, usually beyond 20 FGDs, you cannot add value to any single question. More than 20 FGDs would be required only if you are researching a portfolio of questions. It might be better to split the research into separate projects, for better focus. Similarly in quantitative research, those prone to over-sampling do not understand that sample sizes depend on variability and not universe size.

2) Not giving due importance to translations – this can create a complete hara-kiri for the best designed questionnaires. Researchers spent hours / days in designing questionnaires – all the ‘to and fro’ with the client and all that included. Incorrect translations take away the complete essence. Sometimes, the issue is not even incorrect translation but language incompatibility. It is so easy to reel off ‘neither important nor unimportant’ in English. Try translating this into Hindi ! The questionnaire can become complete garbage going through the translation process.

3) Not pre-piloting questionnaire – simplest of questions should be piloted. No new questions should be put into any research without at least a small / simple pilot. People have multifarious ways of interpreting questions and unless piloted, the response might be different from what we are interpreting it

4) Selection of methodology – Usually, there is not much issue in selection of quantitative vs. qualitative. But the issues usually creep in DI vs. Group OR CLT vs. In Home. Categories where people might have any concern with sharing data in public forums, should be handled as DIs / or In home. Or whenever the research needs to understand, knowledge levels, it should be done as one-on-one.

5) In case of data not fitting pre-conceived notions, assuming that there is a problem in fieldwork. While, there are cases of fieldwork goof-ups, mostly, fieldwork is ok. Try looking at the data again, are there patterns / reasons you do not know / think of. If not, try looking at data prep, did something go wrong there? Or in data analysis? If nothing explains, call back the respondents and ask them to explain the data. Usually, this solves most of the problems.

Written by: Shubhra Misra