Channel partners and vendors who sell AI like to extol its virtues and business value. But during a ChannelCon Visionaries Panel this week, speakers also shared stories of when AI use went bad.
“I know a case where a financial firm's earnings got dumped into a public AI tool and became exposed,” ConnectWise CEO Manny Rivelo said. “Mistakes like that will keep happening. You have to be careful what information goes into your prompts, because public models learn from them. There are solutions for this — prompting security, data-loss prevention — but that's where leakage begins inside an organization.”
Microsoft corporate VP Nina Harding added: “It used to be all about outside threats. Now it's inside your own walls too.”
Of course, we’ve all heard about AI models going rogue. Craig Fulton, M&A advisor for portfolio MSP Evergreen, shared an incident about online misinformation regarding his company.
“A few months ago there was suddenly an announcement online that Evergreen had made an acquisition,” said Fulton, whose company has made over 140 acquisitions. “We all looked at each other — ‘did we do this?’ No. The site looked AI-generated. Our best guess is that somewhere in searching companies, a model just concluded we'd made an acquisition and started reporting it. We had to tell all our MSPs that you can't just ask these tools questions. They form their own conclusions and share them back out.”
The moral of the story: be careful with the data you put into AI.
“Everyone's talking about AI, but not enough about data,” said Jennifer Anaya, Ingram Micro’s SVP of global marketing. “Data is the absolute core of making any of it work. That's why MSPs are in the pole position -- you sit on top of your clients' networks and data. Understanding your data DNA and how to apply it will be critical over the next couple of years.”
Another huge challenge is Shadow AI. Almost everyone inside most organizations uses AI in some way. Policing them all is difficult. For MSPs, they also have to deal with Shadow AI inside their clients’ organizations.
“You have to invest in policy and verify – not just assume – that clients are actually locked down,” said Jonathan Philipsen, MSP Thrive’s EVP of channel and alliances. “You have to reduce Shadow AI so these things don’t happen.”
Harding said Microsoft puts a strong focus on governance to reduce the damage of Shadow AI. “You can't stop everyone from swiping a credit card, but you can make the governed path the easy path,” she said. “Anyone can have an AI conversation today. Everyone's curious, everyone wants a strategy. But it quickly becomes a data conversation, then a security conversation.”
The bottom line: to gain the most value from AI, you must protect your data.
“Protecting IP is critical,” Rivelo said. “Your differentiation isn't the IP itself so much as what you do with it. But if you feed it into a public engine, it can be replicated, including by agents. Take advantage of AI, but protect it.”