Will AI Steal Our Jobs?

Artificial IntelligenceRay Poynter, 21 April 2023

At every event I go to, I hear, at some stage in the discussion, ‘Will AI steal our jobs?” So, here are my thoughts about that question and my views on the prospects for people working in the insights and research industry.

Well, the short answer is yes, but maybe not before you change role, retire, or retrain.

Change is the natural state of employment in modern society
The first thing we all need to remember is that change is normal. Few things stay the same for long, and no sector or group of people have an automatic right to a secure future. The table below shows the top categories of employment in the UK for 1921 and 2021, from the UK Census.

UK 1921 (population 43 million 2021 (population 67 million)
1 Coal Mining Retail
2 Agriculture Education
3 Building & Construction Human health activities
4 Central civil government and defence Construction of buildings
5 Railway transport Public administration and defence
6 Engineering Social work activities
7 Local government Food and beverage
8 Marine engineering & Shipbuilding Computer programming & consultancy
9 Cotton manufacture Wholesale trade (excluding vehicles)
10 Private personal service (e.g. domestic servants) Services to buildings & landscapes

Looking at the changes, we can see the impact of technology; for example, farming now employs very few people but produces much more food. We can also see the impact of macroeconomic trends, for example, the shift from coal to oil and to electricity, and also the shift away from shipbuilding in the UK to shipbuilding in other countries.

We can also see a shift to people-intensive sectors, such as education and human health activities, facilitated by higher levels of wealth. We can also the see positive aspects of new technologies, with computer programming and consultancy being the 8th largest category in 2021.

Technology has already claimed over a million jobs in research
The current concern is about artificial intelligence, but we should note that an enormous number of jobs have been lost in research over the last five decades. In 1973, most research was face-to-face; there were interviewers all over the world, knocking on doors, intercepting people on streets, and working in central locations. There were people typing and printing questionnaires, posting them to interviewers, and punching the completed questionnaires into tabulating machines and computers. Many of these jobs were lost when there was a shift to telephone interviewing, a trend that accelerated when we moved to computer-aided telephone interviewing (CATI). The rise of online surveys led to the loss of most of the people working in research call centres, and innovations like Word and PowerPoint led to the demise of teams such as the typing pool.

Perhaps the main difference between other technology changes and AI is that the jobs that might be at risk now are more senior jobs, the jobs of people more likely to read things like this blog.

Not all technology destroys jobs (especially in the short term)
The last five years have seen an explosion in the number and power of platforms. Almost every aspect of insight and research can be conducted on one platform or another. However, the main impact has been to increase the amount of research being done rather than reduce the number of people working in the research industry. Indeed, in 2023 there is a talent shortage in the insights profession. There has been an increase in the number of people working in IT, sales and customer success management in the wider insights sector over the last ten years, largely because of technology, and revenues have boomed.

The data from ESOMAR, for example, the Global Market Research Report, the Global Prices Study and the Buyers and Users Study show that more research is being conducted, that the average prices of projects are falling, that about 50% of all research is ‘no questions research’ (e.g. passive monitoring, social media listening, data analytics etc), and that about 50% of research is conducted internally by end clients.

Technology has been making research more efficient, and the key change has been an increase in the amount of research conducted and an increase in the number of salespeople, IT people, and customer success managers.

Sometimes the change in employment is hard to see. In the past, we might have seen a clientside insights person working five days a week and working on nothing but research. Now we might have five people, each spending four days a week as a brand manager, a designer, etc, and one day a week conducting insights. The employment in both cases is one FTE (full-time equivalent), but in the second case, it looks as though there is less employment in the ‘insights’ domain than in the first case.

However, it is also clear there have been hard times for some agencies, with closures and forced sales not being a rare occurrence, especially for small to middle-sized agencies.

So, what about AI?
Insights and research have been using AI for years, and until now, it has largely been additive. We have used it to auto-translate text, to help us analyse text, to produce adaptive surveys, to transcribe audio, to help optimise travel, to create research designs (e.g. for choice models), and in less visible ways such as spotting fraudulent research participants.

In general, the uses of AI today has made researchers more efficient, allowing them to do more and to do tasks that would otherwise not be done.

However, I believe we are on the cusp of some major changes that have been foreshadowed by ChatGPT and other platforms. There are two main thrusts that I think will cause AI to reduce the total employment in insights and research and one countertrend.

  1. AI will make existing providers much more efficient, allowing them to operate with smaller teams. This change is analogous to when research when from human interviewers to online and mobile surveys.
  2. AI will increasingly remove the need for clients to go to agencies to conduct research. The special stuff will still require agencies, and the things that need client success managers will still need agencies, but most of the ad hoc, simple research will be done directly by clients. This disintermediation is like the change that happened to travel agents.
  • The countertrend is that amount of research conducted will continue to grow. In a customer-centric, experience-driven economy, insights are becoming more vital, and the changes in ResTech mean they are available faster, easier, and cheaper.

The Winds of Change
So, what are the changes that will impact employment? Here is my list of key things to watch out for.

  1. The rise of secondary / desk research. AI is getting better and better at searching for answers from existing information. The breakthrough that large language models are making (things like ChatGPT and Bard) is that they can produce answers rather than data. Somebody who needs to make a business decision can ask AI a natural language question and get a natural language answer. They can then ask for the answer to be expanded or summarised further. This will massively increase the use of evidence in making decisions, but it will also reduce the need for primary research. I expect this to start reducing demand for some types of research in 2024 and onwards.
  2. Smart DIY. Self-serve platforms have been growing exponentially in number and power for years. AI is going to add a magic ingredient to these platforms, better design and better analysis. The best research for the foreseeable future will come from bespoke research conducted by researchers with training and skill. But the self-serve platforms will soon be creating research designs (e.g. discussion guides and quant questionnaires) that are as good as the median researcher, for zero marginal cost, and almost instantly. This will increase the use of research, but it will accelerate the shift from agencies doing the work to in-house teams being able to do the research. This has been happening for a while and will accelerate in 2023, 2024 and 2025, and beyond, until Smart Self-Serve becomes the norm for most research.
  3. Virtual Research. Virtual research refers to systems that estimate what real people would have said if they had been asked. There are already some systems that do this, for example, virtual eye tracking, which estimates what real eye-tracking would have shown. ChatGPT is already being used for this. For example, what would regular users of brand X say if I were to increase the price by 10% (see example at the end of this post). Note, in the short term, it does not matter whether these systems are reliable – they will be plausible, fast, and cheap, and that is enough to take a share of the insights business. These systems will start to steal business from real systems from 2024 or 2025 onwards. Virtual research will tend to benefit from concerns about the quality of research conducted via online panels and from declining response rates.
  4. AI and qual. There have been primitive forms of bots that have done qual interviews for a few years. The large language models are going to jump this forward, partly because they will be more capable and partly just by being plausible. There are three aspects to AI in qual. The first is to make qual researchers more productive, to allow them to process more textual and video material faster, and to help them summarise information and create outputs. The second is that it will enable more clients to do their own qual research by suggesting things like discussion guides and providing an analysis of the resulting material. The final one is full-blown AI-qual, with the AI conducting and analysing questions. My feeling is that these changes will only have a marginal impact on employment over the next three years, after which they will start to negatively impact head counts.
  5. Social Media and Beyond. AI has been used for years in gathering and analysing social media and for other investigations of the world, for example, automated semiotics. I expect to see another leap forward. In the short term, I expect most of the changes to be more research being conducted, with the researchers being more efficient. However, as the tools improve, more of this research will shift to non-researchers.
  6. Virtual Consultants and Virtual Presenters. Most non-researchers, such as brand managers or product designers, can only go so far with self-serve platforms. The key limitation is knowing what is possible and knowing how to turn the results into an engaging story. Developments in AI suggest that within five years, AI will be able to have a natural language conversation with, for example, a brand manager, suggest and implement a research design, and then present the findings and recommendations. The thinking in this area is around ‘virtual agents’. I think we will see this develop over the next five years, but I suspect it will be closer to ten years before this reduces the number of consultants and presenters the insights sector will need.

The Net Impact
The amount of research conducted, or more precisely, the use of evidence-based decision-making, will increase massively as AI options increase. However, the cost per project will continue to fall, and the number of people-hours per project will fall.

Over the next few years, I expect the headcount at agencies to fall, perhaps by 2% a year, but I expect that rate to accelerate from 2025 or 2026 onwards – towards a decline of perhaps 5% to 8% a year in headcounts in the supplier side.

In ten years, I expect the number of people employed on the supplier side to be about 50% of what we might have expected the employment level to be. Note that I expect insights and research to grow, and that will offset some of the losses. At the same time, I expect some increase in the number of people employed by clients who have some role in insights.

In terms of revenues, I expect the ResTech sector to be worth much more than the more traditional providers. But I also expect the traditional providers to acquire more and more of the ResTech companies.

So, the long answer is ‘Yes’; AI will take many people’s jobs. It will probably be five years before we really notice it, and in ten years, there will have been a major change. But, over a ten-year period, plenty of researchers will have retired, moved to work for clients, moved to ResTech, or retrained and will be taking advantage of what AI offers.

Webinar

Want to hear more about what people are doing with AI right now?
NewMR has a webinar on 14 June to look at AI in action in insights and research.

We have invited four people along to show how they are using AI right now. The webinar is free and you can sign up by clicking here.

Postscript – A Virtual Research Example
Using ChatGPT 4, I asked ChatGPT to describe different personas for buyers of Pepsi Max. From the personas it suggested I chose the Fitness Enthusiasts. I first asked ChatGPT what impact a price increase of 10% might do to my market share with Fitness Enthusiasts. After some reluctance, it opined a reduction of 5% to 20%. I then asked it to consider a new environmentally-friendly container. I asked for three verbatim comments from three hypothetical Fitness Enthusiast to the question, “What do you like about this new environmentally-friendly container?”. The answers were:

  1. “I’m really impressed with Pepsi Max’s commitment to sustainability. This new container is not only stylish but also eco-friendly. It makes me feel good about enjoying my favorite zero-calorie beverage, knowing that I’m contributing to a healthier environment too. I’m definitely going to recommend this to my friends!”
  2. “I love the fact that Pepsi Max is now using environmentally friendly containers. As a fitness enthusiast, I care about my health and the planet’s well-being, and it’s great to see a brand that shares my values. This new container is a game-changer, and it makes me even more loyal to Pepsi Max as my go-to beverage choice.”
  3. “This new container for Pepsi Max is fantastic! Not only does it show that the company cares about the environment, but it also reinforces my belief in making sustainable choices in my everyday life. I appreciate that they’re making an effort to reduce their carbon footprint, and it’s definitely a selling point for me when choosing a beverage.”

Note, I have no reason to believe that any of these responses are likely to be correct, but they have superficial plausibility. That plausibility is the challenge of large language models. If ChatGPT and Bard keep being talked up by the media, they do not need to be right to be successful; they only need to be plausible.