Enterprise Adoption2 min read

Sometimes the Best Move for a Sales AI Is to Send Nothing

By , Senior AI ConsultantPublished

An AI agent that follows up with every renewing customer can lose sales, and a phone company's experiment showed contacted customers left more often than ones left alone.

A professor of innovation has pointed out something easy to miss about corporate AI: companies judge agents by how much they do, and the harder skill is knowing when to do nothing. His example is a sales agent that reviews an account, writes an email, offers a discount and books a follow-up, for a customer who would have renewed anyway.

Contact has a cost that an activity log hides

When a person has to write every follow-up, time rations the contact. An agent removes that limit, so the only brake left is judgment, and a dashboard that counts emails sent and tasks completed does not measure judgment at all.

Marketing has a name for the customers this hurts: sleeping dogs, the ones who are more likely to leave if you contact them. The best evidence comes from a field experiment at a mobile phone company, where one random group got a retention offer and another got nothing. The group with the offer left at 10% over two months, and the group without it at 6%. The offer did work for some customers, and the signs were in data the company already had, such as how their usage had been changing. On average it lost customers.

A renewal after an email proves very little

The same experiment shows why the agent's own results will mislead. Customers who accepted the offer left less often than the untouched group, which looks like success. But customers who got the offer and ignored it left far more often than the untouched group. The contact itself had changed what they did.

So a report that most contacted customers renewed says nothing about whether the emails helped. Only a comparison of everyone contacted against everyone left alone does, whether or not they replied.

The same logic applies to every assistant

A coding assistant has the same problem on a smaller scale. Microsoft researchers used records of how programmers worked with GitHub Copilot to show that a filter could hide a quarter of the suggestions, at least 95% of which the programmers would have rejected anyway. Each unwanted suggestion costs a few seconds of reading and dismissing. The same holds for a reminder, an alert to staff or a draft reply that nobody asked for: each one takes a little attention, and an assistant that never holds back spends a lot of it.

What a manager can do

Suppose a distributor has 500 contracts coming up for renewal each quarter. It lets the agent contact a random 400 and keeps 100 untouched. At the end of the quarter it compares the renewal rate of the 400 with the renewal rate of the 100. If the emails behaved like the phone company's offer, a 4-point gap on 400 customers is 16 contracts lost because somebody wrote to them. If the contacted group renews more, the company knows the agent is worth running, and splitting the results by customer type shows which customers to contact and which to leave alone.

That record also gives the agent something to learn from. Most business AI never gets smarter with use, and a quarterly untouched group is a cheap way to build the evidence that tells you where it should hold back. Keep it every quarter.

Share this

STAY INFORMED

Get AI intelligence like this delivered to your inbox.

Free forever · Unsubscribe anytime


You May Also Find Valuable