Researchers should be aware of the problems with observational data

Posted by Ray Poynter, 18 May, 2018 The world is shifting from asking questions to utilising observational data (mostly for very good reasons) and this is creating a new set of problems that researchers need to recognise and address. What is observational data? Observational data refers to information gathered without the subject of the research (for example an individual customer, patient, employee, etc.) having to be explicitly involved in recording what they are doing. For example, collecting data without people having to respond to a questionnaire, without having to take part in a depth interview, and without having to maintain a research diary. Most big data is observational data, for example, the transaction records from a bank, people’s viewing habits on a video streaming service, or posts in social media. But, observational data can also be small data (based on just a few people). For example, participant ethnographic methods, used to to study people in their everyday lives, collects observational data, that is clearly not ‘big data’. Observational data can be based on census or it can be based on sample. For example, a few years ago a leading mobile phone company was able to sell very detailed data about […]

The dog that didn’t bark – a great way to find insight in information

Posted by Ray Poynter, 10 May 2018 In the Sherlock Holmes story ‘The Adventure of Silver Blaze’ the key to finding the story (i.e. the person who committed the crime) is the curious incident of the dog in the night. When the horse (the Silver Blaze) was being stolen, the dog in the stable yard did not bark, and that was what was curious. This clue led Holmes to deduce that the theft was an inside job; conducted by somebody the dog already knew. Over the years, I have found the ‘dog that did not bark’ idea to be a useful tool when trying to find key messages and stories from a set of information. Thinking about what has not been said by research participants can be as revealing as what has been said. When analyzing the data, ask yourself what is not there? Consider the following examples: You read some references for a new employee, and they all stress team player, effort, innovative and cheerfulness. But none of the references address quality of work or accuracy – so perhaps the person’s work is not great – something you need to check. You look at a set of NPS data […]