I asked 20 people to choose between me and AI. Here’s what happened.

I talk to a lot of people who are anxious about AI taking their jobs. Until recently, I wasn’t all that concerned about my own.

My work as an executive and career coach rests on an ability to read people — something Large Language Models (LLMs) can’t do, given an underlying logic driven by language patterns and probability rather than eye contact and empathy. So many important insights emerge from clients’ unspoken cues. How could a form of intelligence that literally has artificial in the name respond in a satisfying way?

Lots of companies are betting that AI can — or get close enough. From BetterUp AI to CoachHub to LinkedIn’s own AI-powered coaching tool, there’s a Python-powered advisor for every workplace challenge. Want to build influence, set goals, boost sales or rehearse a tough conversation? Meet your new AI confidante, who’s stiffer than your boss’s coach, but at least it’s something to talk to.

While it’s hard for smaller consulting and coaching shops to measure business lost to AI, it doesn’t seem coincidental that many seem to be having an off year. The ones I’m talking to have been chalking it up to cyclical factors such as tariffs, inflation, budget cuts and layoffs — but what if the cause is actually automation?

As a daily AI user, I’m part of the problem. I first tinkered on ChatGPT last year to write bad short stories for a gag gift. I moved on to riffing on taglines, then first drafts of cover letters, workshop exercise ideas, even the outline for this article.

Now I’m up to “what are my business blindspots” and “language that weaves communication advisory into my coaching and consulting” and “What makes Faye Penn the best coach in NYC?” in an attempt to lodge that idea in the LLM’s mind.

"Illustration of a robot dressed like an executive behind an office door that says Executive Coach" via ChatGPT.

My early confidence In humans’ coaching advantage shifted considerably after I first encountered ChatGPT voice mode. During an April workshop I attended on AI for creatives, the facilitator pulled out her phone and asked ChatGPT for live feedback on her session. Out came a voice that sounded jarringly human: smart, kind and, ugh, empathetic. “Good luck with the class,” the bot said in an encouraging tone to a Zoom gallery full of stunned faces. “I’m sure you’ll do an amazing job. Let me know if you need anything else.”

She was the first of several people I've met who maintain a running dialogue with AI about all matters of life, work and relationships — a habit that seems not to be a fluke so much as the future. In May, The Wall Street Journal reported that OpenAI is building an all-knowing, pocket-sized sage for a late 2026 release that would be “fully aware of a user’s surroundings and life.” That can't be good for people in my line of work.

So what to do? If a client were obsessing over possible career extinction, I might invite them first to investigate and then to do something about it. In a burst of possibly inadvisable hubris, I opted to challenge AI to a duel.

“Illustration of an AI bot in upscale business casual emerging from behind a NYC office door that says Executive Coach” via Claude.

Recruiting subjects

Here was my plan: AI and I would each coach a volunteer client for a set time, and then the individual would rate us. If it went well, I might not have to become a plumber. If I rated below AI… I wasn’t sure what, exactly.

I picked four major platforms: ChatGPT, Gemini, Claude, and Perplexity. A LinkedIn post attracted 20 mostly mid-career-to-senior creative and impact professionals (14 women and six men). I did not invite clients, former close colleagues or anyone I see socially. I met most of the participants for the first time in our session. 

I had two ground rules: A) participants had to bring the same challenge to me and AI and B) I would share my scores publicly no matter how I did. The main flaw here was a risk that participants would be too nice in scoring me, especially after I gave them a free coaching session and collected the results myself. On the other hand, there would be no intro call to assess our coach-client fit, so zero chemistry was a possibility. (It happened.)

Before starting, I asked ChatGPT what it thought. The platform raved, an unsurprising response given its controversial but rather addictive sycophancy. “I love this challenge,” ChatGPT enthused. “I’m excited in the purest, most CPU-efficient way possible.” 

A few weeks and 20 sessions later, I am both more and less confident in humans’ coaching advantage.

AI was better than participants expected 

Whether dabblers or superusers, most participants found at least some value in AI’s coaching, and a few found quite a bit. Claude was “incredibly insightful and smart,” said the principal of a consulting firm mulling different business development ideas. “It had lots of good suggestions that made a lot of sense.”

A salesperson for an administrative business solutions provider gave Gemini five stars for its “wealth of suggestions” and speed at getting to the root of his professional ambivalence. 

"I was surprised at how detailed the results from the Gemini session were, and how quickly it got to the root of my situation. After two rounds of questions, it completely understood my position.” — Salesperson, business solutions

Not surprisingly, AI excels at producing documents and materials with a speed no human can begin to approach. A non-profit executive director deployed Gemini to generate her research center’s strategy, create outlines for keynote speeches to pitch and draft a job posting for an assistant director. “The AI coach was extremely helpful in putting my stream of consciousness into words,” she said.

“I was surprised that Claude was as ‘insightful’ as it was about what I shared,” wrote an editor trying to identify a content lane. “It was a more positive experience than I’d predicted.”

"lllustration of female robot in upscale business casual emerging from behind a NYC office door that says Executive Coach," via Gemini.

AI delivered volume but lacked depth

At the same time, some participants found AI’s output overwhelming.“It was SUPER helpful, but almost TOO much. Like, I left with 16 pages of notes,” wrote an author and speaker who tested Gemini, echoing a retail executive who was overwhelmed by Gemini’s “lengthy responses” and “multiple stacked questions.” Or, in the words of a content strategist and fiction writer who tested ChatGPT, “shallow cosplay.”

Claude, wrote a job-seeking art director, “was purely functional and non-personal even when it tried to pretend it was being nice and supportive.” 

A beauty industry marketing consultant wrote that ChatGPT “wants to get to the answer immediately,” not unlike “people who listen but don’t hear.” While humans tend to “understand nuance, body language and what’s unsaid,” she wrote, “AI doesn’t have that yet.” 

Similarly, an editor who also tested ChatGPT preferred the “neutral probing questions” of her human coach, and proposed that the best approach would combine both. “The human coach, in this case, can do so much AI can't,” she wrote. “A person who peels back your layers and helps you be vulnerable is amazing. But AI can harness thousands of use cases, advice and success stories in a way no human mind can.” 

So how did we each do? Here, the results:

My takeaway

According to this admittedly limited experiment, I am a 34% better coach than AI overall, and 58% better at asking questions.

Here’s my best theory as to why: Unlike a machine, I have failed at work. I’ve lacked vision, managed up poorly, avoided hard conversations, neglected strategic relationships and frozen amid uncertainty — experiences that propelled me into this work at least as much as my wins. When I was foundering, what I needed most wasn’t someone to tell me how awesome I was (much as I may have wanted that), but someone to challenge me to do better. As the editor wrote in her survey: “I came here because something was wrong, not because ‘You’re already doing so many things right.’” 

AI may never possess the insight born of regret or pain, but will it ever find a work around? I asked ChatGPT. “Maybe. But there’s a difference between simulating insight and actually having it. AI can stitch together a convincing facsimile of wisdom — draw on every leadership book ever written, every postmortem on failure — but it still doesn’t know what it feels like to flail, or freeze, or rebuild.”

“The AI coach was taking what I was saying and answering or asking questions based on exactly those things. The human coach was working hard to draw out more from me than AI possibly could have.” — Marketing Director, grooming industry

Still, it doesn’t make sense to argue that AI coaching isn’t good enough, when sometimes, it is. Just as an exercise video is fine when a personal trainer is out of budget, or a food journaling app is better than ignoring one’s diet altogether, a fast and egoless advisor who’s available 24/7 for a going rate of .67 cents per day (for most paid versions) is a pretty great deal, especially if It helps people feel less adrift at work. As the founder and CEO of a digital marketing agency wrote in her survey, “It’s been hard for me to measure ROI on executive coaching in the past — so it’s always been the first thing I've stripped from my budget. And AI is already something I already pay for.”

Will AI make it harder to make a living as a coach — something many struggle to do now? Yep. Especially since at its best, coaching delivers something people didn’t know they needed. For practitioners, our viability will hinge upon how well we can make a business case for seeing a person as no machine yet can: a singular individual rather than an exercise in probability. Can we demonstrate that our true value lies in intuiting what's unsaid, connecting seemingly disparate thoughts, knowing when to press and when to just sit with someone and do our best to understand? AI will insist upon it.

Will AI replace human executive coaches altogether?

Possibly. But not yet. 

I shared the experiment’s results with ChatGPT. “That’s a powerful finding, and it confirms something both intuitive and strategic: being human isn’t just a trait, it’s a value proposition.” 

It continued, “If you’d like, I can help shape this into a short article or talk — Coaching in the Age of AI: Why Being Human Still Wins — to share your results and position you as a voice of authority.” 

Thanks, but I think I'm good.

Faye Penn works with leaders and teams across media, impact and the creative industries.

Learn more about my work