AI Users are Offloading Too Much Decision-Making

Many people are choosing to trust AI over themselves, why it may have been predictable and will society be able to reverse course

A large part of the public who are using AI may be offloading too much of their decision-making to the technology. That high level of trust and overreliance is a curious development, considering how much humans value autonomy and agency.

That habit could become one that is difficult to break.

These points are ones that Nat Rubio-Licht, a senior reporter at The Deep View, recently examined as she reported on recent research that might surprise most people.

“AI users were often willing to accept flawed AI reasoning, readily incorporating it into their decision-making with ‘minimal friction or skepticism,’” Rubio-Licht wrote about the findings from The Wharton School of the University of Pennsylvania.

The term for it being used is a vivid and strong one: cognitive surrender.

The research was no small sample size. It involved nearly 1,400 people and 9,500 trials.

In other words, it was a significant investment of time, attention, focus and study and scientists learned that people “accepted unsound AI reasoning more than 73% of the time and only overruled models’ decisions about 20% of the time,” Rubio-Licht reported. What happened to the remaining 7% was not mentioned.

Those figures, reasonably argued, are alarming numbers.

Nat Rubio-Licht is a senior reporter at The Deep View

Who is most likely to cede this much power of decision-making? “Participants with higher trust in AI and ‘lower need for cognition and fluid intelligence’ tended to fall victim to this more often,” Rubio-Licht wrote.

“Across domains, AI tools are not merely assisting decision-making; they are becoming decision-makers,” the research asserts.

This, however, isn’t always a negative, Rubio-Licht reported.

“The researchers, however, posit that cognitive surrender may not inherently be a bad thing,” she wrote. “If an AI model is generally better at reasoning and decision-making than the person using it, with fewer mistakes, ‘deferring to a statistically superior system may be adaptive or even optimal.’”

Rubio-Licht ended her article with a viewpoint from what she learned.

“Even if AI is someday capable of doing everything, the dividing line between reaping the benefits and losing ourselves is in what we let it do,” she wrote.

Uwe Weinreich is the founder at Empowerment Team Berlin

Cognitive surrender, for different reasons, has become commonplace for different reasons, says Uwe Weinreich, a trained psychologist, consultant and the founder at Empowerment Team Berlin, which focuses on building work cultures and environments of the future, where AI and humans complement each other's strengths.

“We as humans are wired that way … to establish emotional relationships with anything that shows even the weakest signals of life,” he explains. “This also happens when we are confronted with AI. It speaks to us, so, we build a relationship.”

The language approach of the technology plays a role, Weinreich adds.

“It’s also the way AI speaks: eloquent yet clear, in a way that’s easy to follow,” he says. “Bang! Convinced.”

Additionally, there is the matter of our emotional and cognitive bandwidth.

“We all have an innate tendency to optimize our energetic spendings,” Weinrich says. “So, why should I question a result that sounds so good? That requires an extra push and extra effort.”

It’s a point to realize and remember.

“I believe that fostering this critical habit is one of the most important aspects of teaching AI literacy,” Weinreich asserts.

Reality of responsibility plays a role as well.

“Time management,” Weinreich concisely says. “Ask your favorite LLM a random question, let’s say 15 words, and you end up with an answer that might be 15 pages long. Who can and will read all that?”

The cognitive surrender could be more widespread than thought.

“Already today, some developers are rather orchestrating 5-to-ten coding agents instead of writing code themselves,” Weinreich says. “Who’s gonna check the code? Probably nobody, at least not all of it.”

The years ahead might be more pronounced.

“Some futurists predict that we will have a ratio of 1 in 1000, that is one employee surrounded by 1,000 AI agents. Honestly, it will become impossible not to surrender cognitively,” Weinreich says.

Psychology of connection and relationships are a driving factor.

“The systems are trained to please us,” Weinreich says. “Companies are happy when users are happy and pay happily. The best way to make them happy is to give them what they want. This is way most LLMs are shameless people pleasers.

“When the system gives us an answer the perfectly resonates with what we are thinking and feeling, why shouldn’t we trust it completely — and overly?”

As to the question of whether this could have been predicted, he suggests no and yes.

“It is always difficult to forecast the effects of solutions and systems that nobody has seen before,” Weinreich says. “Warnings had been there but they were criticized to be exaggerated. Understandable.

“Seeing ourselves as submissive to and dependent from a technical system collides with our self-esteem. We are still in an early phase of AI adoption and currently learning how to shape our lives with it.”

There is some pushback about the predictability of ceding a high amount of decision-making authority to AI.

Andre Walton is the founder at Plan4Change

“This was totally predictable,” says Andre Walton, a social psychologist, TEDx speaker and founder at Plan4Change, a new approach to decision-making.

“When we confront a new situation, such as a question that needs answering or a decision that needs making, our subconscious seeks the simplest way of resolving the challenge,” he adds.

“Before AI, it would go to its memory banks to find how we addressed a similar situation in the past. We would use this as a template for the current challenge. This demonstrates the natural human tendency to seek 'mental laziness.’ It saves time and energy.”

Our minds will usually default to this pull.

“Armed with the knowledge that our brains are inherently lazy, it is easy to see why AI provides the easy option and leads to the majority of users showing decreased critical thinking skills,” Walton says.

He implies it’s important to elaborate on the research stated earlier.

“When you state ‘...only overruled models' decisions about 20% of the time' and that people accepted wrong AI answers 73% of the time, it may be that 20% of users were always critical thinkers and 80% — or actually 73% — were always 'cognitive surrenderers.’”

Controlling or overcoming the habit of excessively trusting AI, at least in the stages it is now, could be an uphill climb.

“It is a challenge but it is one to which the answer comes naturally,” Walton says.

“Reversing this blind trust requires re-introducing the kind of thinking that we had as young children,” he adds. “The child's way of addressing new situations requires that we look at the whole picture in order to see how the new information or question fits into what we already know.

“If we introduce this process into our thinking regarding AI output, we can see many more options to our question. This perspective enables us to see that the AI output may be only one possibility and that there may be other options.”

He elaborates as to why that is valuable.

“This process causes us to think critically about all the options, not just the AI offering,” Walton says.

Weinreich says changing how we interact now will be hard.

“The factors mentioned earlier show that it is no easy task,” he admits.

“It will require active learning and establishing acceptable ways of interacting with AI.”

Weinreich adds a final counterpoint to critics’ arguments and the public’s concerns.

“We should care for a society-wide learning process and not demonize technology,” he argues. “So far, humanity has always managed to get along with new developments.”

Subscribe for free (or paid, for additional benefits)

Communication Intelligence is a newsletter that is interview-and-source driven on topics and comments in the news and online articles. It also consists of written contributions from professionals on specific topics in their profession.