(metamorworks/Shutterstock)
There has been a lot of fear, uncertainty and doubt (FUD) about the potential of generative AI to take jobs away from people. The ability of large language models (LLMs) to answer questions and handle digital tasks when asked has caught people’s attention, for better or worse. But what are the chances that LLMs will actually replace human workers? A new study from Indeed sheds some light on that question.
The digital job board Indeed It recently conducted a test to determine the effectiveness of LLMs in managing basic job skills. Indeed Hiring Lab enrolled in GPT-4o, OpenAI’s latest LLM, and asked you to perform more than 2,800 job skills tracked in Indeed’s database, from office jobs like account management and insurance claims to more physically demanding, like a bus. driver and cook.
For each job skill, Indeed Hiring Lab created a way to measure how successfully the LLM completed the task. They created sophisticated 1,000-word prompts for each task, which required a lot of trial and error. After finally settling on the best message, Hiring Lab workers ran the message through GPT-4o 15 times and then aggregated the result. GPT-4o was asked to evaluate its own ability in each indication and the results were validated by human researchers.
Source: “AI at work: why GenAI is more likely to support workers than replace them”
The Hiring Lab focused on three main areas with the experiment, including GenAI’s ability to provide theoretical insights related to skill; GenAI’s ability to solve problems using skill; and GenAI’s determination of the importance of physical presence in the utilization of that skill. GPT-4o tested your own ability to use these attributes with a given job on a five-point scale. The researchers tabulated the results and published them last week in a paper titled “AI at Work: Why GenAI is More Likely to Support Workers Than Replace Them,” which you can download. here.
The title is a big clue to Indeed’s findings with the GenAI experiment. The report’s authors, Annina Hering and Arcenis Rojas, write that it is “very likely” that none of the 2,800 job skills will be replaced by GPT-4o or any other LLM. In fact, Indeed found that nearly 69% of skills are “very likely” or “unlikely” to be replaced by GenAI.
Clearly, no job that requires hands-on execution or the application of physical force, such as bus driver or emergency room nurse, will be replaced by GenAI, which is after all just software (autonomous buses and robot-assisted surgeries are real, but they also require much more technology than just GenAI). Considering that more than half of the jobs included in this report required some form of physical execution, the prospects for a complete replacement of GenAI seem quite bleak.
But that doesn’t mean there won’t be any benefits. Indeed says that even for jobs like a bus driver or nurse, GenAI could help with repetitive tasks like documentation, which will “allow workers to refocus on the core skills needed in these roles,” Hering and Rojas write.
The researchers concluded that about 29% of jobs could “potentially” be replaced by GenAI “as it continues to improve and if certain changes occur in workplaces and/or labor standards in the future,” the researchers write. researchers. The jobs where GenAI will have the greatest impact are “more stereotypical office jobs,” the researchers write.
Through the three central measures of the study: theoretical knowledge; problem solving; and physical job skills: GenAI excels most in theoretical knowledge, closely followed by problem solving. In fact, theoretical knowledge was the only attribute for which GenAI gave a 5, the highest score, thanks to LLMs’ extensive training in large amounts of information on the Web and the ability to use search engines.
GPT-4o also scored decently on problem solving. He rated himself a 3 on 70% of the skills he tested, and on 28% of those tasks he said it was “possible” he could replace a human. It also received several 4s and was rated “likely” that it could replace a human in 3% of tasks.
Source: “AI at work: why GenAI is more likely to support workers than replace them”
GenAI is very likely to replace humans in office jobs and jobs that are predominantly done on the computer. For example, the researchers concluded that it was “possible” or “likely” that GenAI could replace a human in more than 71% of the skills commonly found in software development job postings. Similarly, it was “possible” or “likely” that GenAI would replace humans in 78% of the skills commonly found in a typical accounting occupation, the report says.
GenAI is less likely to replace humans in jobs that require more problem solving than theoretical knowledge. This is an area where GenAI developers and data scientists may want to focus their efforts.
“If GenAI models improve their problem-solving capabilities to gain more skills in more jobs,” Hering and Rojas write, “it is likely that the share of skills that can eventually be replaced in those jobs will also increase.
There are things companies can do to help them prepare for GenAI. In the accounting field, for example, investments in electronic recordkeeping and digitization will go a long way toward preparing a company to successfully use GenAI.
Adjusting (no pun intended) personal interaction with GenAI can also produce better results. For example, an LLM may interpret a vague suggestion in several ways, and is likely to give different answers each time it is asked. More advanced tasks will require better writing and engineering skills to get the most out of GenAI, the authors write.
At the end of the day, it seems likely that GenAI will replace at least some of the tasks that human workers are doing now, with a lot of variation by industry and position. However, Indeed researchers do not see a time in the near future when GenAI will replace humans en masse, simply because GenAI, as it exists today, cannot function without humans.
“Even as GenAI evolves and learns to complete demanding tasks,” Hering and Rojas write, “the humans who monitor, guide, and correct GenAI-derived results will not be easily replaced.”
Related articles:
GenAI is revolutionizing the labor market, says BCG
AI also arrives for administrative jobs
The 10 most in-demand GenAI skills
