CCTV camera

Problems in Identifying Causality in Observational Data – BigSurv18

Posted by Ray Poynter, 27 October, 2018 Earlier today I gave a presentation at the BigSurvey18 Conference in Barcelona, Spain. The conference was themed “Big Data Meets Survey Science” and comprised one day of short courses and two days of presentations. The majority of attendees and presenters were from academia, which perhaps explains why it included a Saturday. I enjoy attending some conferences outside of my comfort zone, I feel I can learn different things and help break down the silos between disciplines. You can download a copy of my presentation “Problems in Identifying Causality in Observational Data” by clicking here. Although I am a fan of observational data (all the way from Big Data through to ethnographic investigations), we need to be aware that observations based on the real world can produce findings that are difficult to unpick. This is also the topic in a blog I wrote last week “Does Running Damage Your Heart? Another example of the problems of using observational data to infer cause and effect.” Interesting topics that arose at the conference inlcuded: Rene Bekkers from the Vrije Universiteit Amsterdam shared results from many studies looking at trust in society. What he showed is that using a 10-point scale […]

Photo of Ray Poynter trail running

Does Running Damage Your Heart? Another example of the problems of using observational data to infer cause and effect.

Back in 2012, there were several research papers that suggested that running for more than about 20 miles a week was either not giving any benefit, or, worse still, it could be damaging hearts and making death more likely. The key sources were a piece of research by Dr Duck-chul Lee with 50,000 patients (presented at the American College of Sports Medicine (ACSM) 59th Annual Meeting 2012, see The Not-So-Long Run: Mortality Benefit of Running Less Than 20 Miles per Week) and heart findings from Dr James O’Keefe that looked at issues such as fibrosis, calcified arteries, and arrhythmias. This was picked up by a joyful media, with stories about how running was bad for you, and that anything other than a small amount of exercise was either useless or damaging. However, as is often the case with observational data (as opposed to control and test experiments), there were several problems with the conclusions. The key problem with Lee’s study was identified in 2013 by Dr Thomas Weber. In the sample of 50,000 people there were some long distance runners, for example, marathon runners. Lee wanted to assess these people against the non-runners and the occasional runners. However, he needed […]

Chart showing ice cream sales

What is the Counterfactual? – Why do we need to assess it?

Posted by Ray Poynter 19 October 2018 If we are told that a before a marketing campaign the sales were at 10,000 units a month, and after the campaign sales are 20,000 units a month, then it is easy to assume that the campaign has increased sales by 10,000 units a month, or by 100%. However, this is an example of the classic fallacy “Post hoc ergo propter hoc” (which is Latin for ‘after this, therefore because of this’. Consider the chart below, an edited and anonymised version of a presentation I saw at a conference last year. The chart was presented by the head of social media to the head of insight in a company selling ice cream. The head of social media protested that the spend on social media advertising should be maintained at a high level. He pointed out that when social media advertising was increased sales, increased, and when the advertising budget was cut, the sales dropped. Therefore, the advertising budget should be increased, so that the advertising could be maintained at the higher level. However, the head of insight took out her pen and scribbled onto the chart three words and images, as in the […]

New Skills

Courses and Workshop with NewMR – 2018

Posted by NewMR, 17 October, 2018 Sue York and Ray Poynter are widely involved in a wide range of training initiatives and consultancy, with NewMR and a variety of organisations including trade bodies, client-side companies and research suppliers. In this post, we highlight some of the courses we are currently offering to your company or organisation. Using Social Media to Build Your Brand Australian-based Sue York, who is one of the most connected insights professionals in the APAC region (see here and here), will show you how you can use Twitter and LinkedIn to build your brand. The workshop covers: Creating an impactful and memorable profile; How to find the relevant discourses; Finding your voice; and, using tools to increase the impact and reduce the workload. The workshop can be delivered as a half or full-day session at your offices (in Australia or within the APAC region), as a series of online lectures, as an e-learning course, or as consultancy. For more information about this course, or to find out about availability and costs, email admin@newmr.org.   Five Courses for Insight Professionals UK-based Ray Poynter, has selected the five courses/workshops that are currently the most requested: Find and Communicate the Story […]

Chart showing Donald Trump Popularity and Unpopularity over time

How to Use Comparisons to Understand Data

Posted by Ray Poynter, 15 October 2018. Most individual numbers do not mean very much. In many cases, in order to see the real meaning in most data, you need comparisons. For example, if I tell you that the Belgian cyclist Eddy Merckx won 11 Grand Tours, you will no doubt guess that he was a good rider. But, when I tell you that 11 is the most any rider has ever won, and that only one other rider has won 10, and only one other rider has even won 8, then you start to get a sense of how special Eddy Merckx’s was. So, this post focuses on how to use comparisons to understand the story in the data, and how to use comparisons to communicate the story in the data. How Popular/Unpopular is Donald Trump Nate Silver’s FiveFiftyEight.com provides a wealth of data on US sports and politics and provides a really good example of how to use comparisons in their regularly updated series ‘How popular/unpopular is Donald Trump?’ The chart below shows the picture on 12 October 2018, 631 days after Donald Trump took office in January 2017. Note, in the US the election for a new […]

Lights in the trees

Bias Runner 2049

Story-with-a-meaning post by Ray Poynter, 11 October 2018 Tom Torquemada was looking forward to his interview today, he was off to the Global Broadcasting Corporation to talk about his work, and he loved his work. Tom was a Bias Runner, one of a team that hunted down and retired errant AI systems. Tom had been thinking overnight about the best, non-technical, way to describe what an errant AI system was and how he and his colleagues identified them. The scale of the problem was clear to everybody – nearly everything today, in 2049, was determined by AI. Machines and bots determined who got a job, who got the next home loan, who might commit the next crime, and whether in this brave new world your schooling/conditioning would result in you being a labourer or artist. But with this transfer of power to the AI machines came a fear, a fear that the machines might not play fair, they might be biased or simply error-prone. The job of the Bias Runners was to find the biased or error-prone machines and ‘retire’ them. There were two key types of problems that the Bias Runners were looking for ‘biased machines’ and ‘unstable machines’. […]