Only a ‘few’ new products can succeed each year

There seems to be broad agreement that in the large developed markets, for example the US, about 80% to 90% of new products fail. The definitions of success vary but they tend to centre around things like achieving good sales and distribution in year 2.

The key point about new products is that there is a relatively absolute limit to how many products can be successful in a given year and there is no, practical, limit to the number of new products that can be launched. The reasons for this are covered below in this post, but the consequence of a relatively fixed number of successes and a growing number of new products is a declining success rate for products.

Behavioural Economics and New Products
One of the things we have been reminded of by the likes of Daniel Kahneman and Dan Ariely is that making choices takes effort, changing behaviour takes effort, and the energy for these choices and changes is in limited supply. Consider products like your regular/normal brand of coffee, detergent, or toothpaste, changing requires a decision, not changing simply requires the heuristics of repeat behaviour.

Households don’t buy an endless supply of brands/products
In ‘Differentiate or Die’ Jack Trout estimated that most US households cover about 85% of their needs by repeatedly buying 150 SKUs. Unless consumers are willing to buy a growing range of products, the growing number of SKUs will increase faster than people can adopt them, and the success rate will fall.

According to Insight Out of Chaos, 2001, in 1980 there were 2,689 new food launches, by 2000 this had grown to 16,390, and increase of over 600% in 20 years. Growth in GDP and waistlines can increase the number of food products bought but there is still a limit to what consumers can buy and consume. New non-food categories, such as male grooming, will increase the absolute number of successful products/brands in shoppers’ repertoires, but the impact is modest in absolute terms.

How many times do we change brands/products?
In order to explore product/brand changes in the UK, I partnered with Vision Critical’s Spingboard panel to collect a broadly representative sample of 2004 adults (in October 2013). Using a shopping basket of frequently bought branded products, we asked the sample whether they had changed the brand, flavour, or size they buy in the last month. For example, had they changed their brand/flavour/size of Tea or Toilet roll.

The table below shows the findings.

63% of the sample said they had not made any changes, on these nine products, in the last month. The lowest figure was for Frozen food, with 7% saying they had made a change in the last month, a category where shops own brand is very strong. The highest category was Toilet roll, which might be a result of the substantial price promotions that are taking place in UK retail in this field, which impacts both brand and size.

The data showed very few demographic differences. The main difference was between the younger respondents (aged under 25 years) where 53% said they had not changed any of their products, and people 25 and over, where 64% said they had not changed any of these nine in the last month.

When considering these changes, we need to remember that many of the changes will be to an existing brand, not necessarily to a new brand.

The table below shows how many people made 0 changes, 1 change, through to 9 changes.

The data shows that there is a power law type distribution for making changes. Most people don’t change, of the rest, 40% make just one change, of people who make more than one change, over 30% made two changes. The slight blip in the size of the blue bar at ‘9 changes’ probably represents the limited range used in the study, some of these people would have made 10 or more changes.

In total 53% of the product changes in the study came from the 9% of people who made four or more changes in the last month. However, these easy win people might also be easy lose people?

The table below stretches the data quite a bit, but it makes an interesting picture and suggests potential for further research. The table shows the results of multiplying the number of changes in one month by 12 to get an annualised figure.

The table suggests that the average consumer is likely to make just under one change to the Frozen food brands they buy in a typical year, and just under two changes to the regular brand/size of toilet paper. The changes are likely to be distributed with some people making very few changes, and a few people making many changes. If these numbers are broadly correct, then a typical product category is going to have about one change per consumer per year, which means the number of changes is limited to number of people in the market. The proportion between changes to existing brands and new brands will, in turn, govern the number of new product launches that will be successful.

Summary
There are good theoretical reasons for believing that the number changes in the brand/products purchased that most people will make in a year has an upper limit, based on the cognitive effort required to make changes. There is data to suggest that the number of products any individual house uses is relatively low (most needing only needing 150 SKUs to meet most of their needs). There is plenty of data showing that the number of new products launched is large and has been growing over the decades.

These factors suggest that the reason that 80% to 90% of new products fail is a result of the way markets work, rather than to do with the products themselves. Even if every new product was yummy or effective, and well-designed, only a relative few could be successful.

When we tested this hypothesis, with the help of Vision Critical’s Springboard panel, we found, as predicted, that relatively few changes were made by most people. And, many of those changes would be to existing brands, not to new brands.

The quality of a new product and its marketing is going to have an impact on whether it is one of the 10% which are successful, rather than the 90% which are less successful (more on this in another post). But, the overall ratio of successful to unsuccessful product launches does not have to be related to the quality or marketing of those products. The absolute number of successes is likely to be related to the size of the market in people and GDP terms, and the maturity of the market. If the number of new products launched keeps increasing, then the failure rate will (all other things being equal) increase.

Thoughts? Comments? Data?