Should Market Research Still Be Using Significance Testing?

Over the last few years there have been many calls for market researchers to stop using significance testing based on assumptions of random probability testing to measure the potential impact of sampling error. For example, Annie Pettit writing in The Huffington Post asked “Stop Asking for Margin of Error in Polling Research”. But, despite the concerns about the correctness of using this technique, it seems to still be in common use. In this post, I briefly explain what significance testing is (experts can jump this bit), why it doesn’t do what people seem to think it should do, and the way I think we should be using it in the future. What Is Significance Testing? The type of testing I am talking about in this post relates to sampling error. In quantitative research, a sample is taken from a population and one or more statistics are calculated. These statistics are then used to estimate the values for the total population. For example, assume 1000 people are selected at random from a population of 20 million. Assume that 50% of the sample are female. The inference from this study is that it would be expected that 50% of the total population […]

You Can Combine Standardisation with Creativity

One of the key tools for increasing productivity, reducing errors, and facilitating automation is standardisation. Unfortunately, many people think there is an association between standardisation and a lack of creativity. However, standardisation does not need to create a cookie-cutter approach, indeed it can promote creativity when done the right way. Whitworth Standard – 1841 For most of the Industrial Revolution, every blacksmith and workshop used its own dimensions for nuts and bolts. If you bought a steam engine from one workshop, you had to go back there if you needed any work doing on your machine. This slowed down the adoption of new machines and tools. In 1841, British engineer Joseph Whitworth introduced the British Standard Whitworth system, the first standardised system for engineering in the modern world. This innovation led to changes in how things were manufactured, for example it was now easy to buy parts from several companies, it was easier to scale-up production, which led to a golden age in manufacturing and industrial creativity. Avoiding Procustes The worst type of standardisation is often referred to as cookie-cutter, or more scathingly as procrustean (definition: enforcing uniformity or conformity without regard to natural variation or individuality). Procustes was a […]