Topics
Latest
AI
Amazon
Apps
Biotech & Health
Climate
Cloud Computing
Commerce
Crypto
Enterprise
EVs
Fintech
Fundraising
Gadgets
Gaming
Government & Policy
Hardware
Layoffs
Media & Entertainment
Meta
Microsoft
Privacy
Robotics
Security
Social
Space
Startups
TikTok
Transportation
Venture
More from TechCrunch
Staff
Events
Startup Battlefield
StrictlyVC
Newsletters
Podcasts
Videos
Partner Content
TechCrunch Brand Studio
Crunchboard
Contact Us
Satya Nadella has issued a shocking warning to companies using AI
Of all the debates raging about the potential downsides of AI, there is one worry causing the most hand-wringing among AI enthusiasts in Silicon Valley. Their fear is that the giant AI labs that sell proprietary models are somehow acting like Trojan horses.
The concern is that, as startups and enterprises use AI models from labs like OpenAI and Anthropic, the labs gain ever-increasing access to those companies’ most sensitive business information. The model makers can then use that knowledge for themselves, potentially becoming competitors to their own customers. Those issuing such warnings range from VCs like Jason Calacanis to Palantir CEO Alex Karp.
Now, in a surprising blog post published on Sunday, Microsoft CEO Satya Nadella has joined this crowd. Nadella warns that AI users (the “buyers” as he calls them) are paying twice. They knowingly spend for AI token usage but they also, obliviously, hand over valuable data in the process.
“You essentially pay for intelligence twice, once with money, and again with something even more valuable: the proprietary knowledge you must reveal to make that intelligence useful. The better you want the model to perform, the more of that knowledge you have to feed it!” he writes.
Most dangerously, enterprises are literally teaching the models about the nuances of their businesses, he argues.
“Models learn from ‘exhaust,’ the prompts people write, the tools agents use, and especially the corrections people make when the model is wrong. Every correction is distilled into institutional know-how,” he writes.
This is “the kind of knowledge a competitor could never buy,” and yet enterprises are handing it over.
Nadella argues that if AI companies get to freely scrape the internet to train their models, it’s only fair that enterprises get to study — or “distill” — those models in return. “Distillation” is the practice of using a model’s own outputs to learn how it works and to train a new, often cheaper, model based on those insights. In February, Anthropic accused Chinese open source models of sending millions of prompts to Claude as a way to improve their own models, and urged the U.S. government crack down on export controls.
sending millions of prompts to Claude
Nadella’s point is that model makers can’t have it both ways. It’s hypocritical for them to freely train on the world’s data while restricting others from doing the same to their models.
“While the great innovation that comes from model providers having fair use rights to train models on public data is needed, I find it ironic that the status quo is to then turn around and impose restrictive terms on distillation,” Nadella writes.
Nadella is particularly concerned when model makers “reserve the right to learn from customer usage and interaction data.”
Nadella’s solution is the kind of thing the CEO of a giant cloud provider would suggest. He wants companies to “retain ownership” of their data, including prompts, feedback, etc. So he’s urging them to build their own “proprietary learning environments” on the cloud (where their data is likely already stored anyway and, conveniently, could mean Microsoft’s cloud, Azure). He also wants companies to build in what he calls “orchestration layers” — essentially, a way to easily switch between AI models from different providers rather than being locked into one. Tools like AI “gateways” that let companies do exactly this have become increasingly popular.
While Nadella never uses the words “open source” as the method for retaining ownership, this is an obvious subtext. Yet, there’s another subtext.
Large companies, many of which still have some of their own data centers in addition to using the cloud, are already moving to open source models installed on their own premises (“on-prem,” in industry jargon). Idit Levine, founder and CEO of Solo.io — which makes networking and security software that helps enterprises manage AI systems — says she’s seeing exactly this shift play out with her own customers. After experimenting with proprietary model makers, they start asking themselves: “Can I take an open source model and run it on-prem? It will do almost 90% of what the big one’s doing. It will cost way less,” she tells TechCrunch. “They understand that, and they can control it.”
Solo.io’s technology was selected last year to be the tech powering the Linux Foundation’s Agentgateway project. Her company counts enterprises like T-Mobile, ADP, and SAP as customers. She sees companies increasingly installing on-premise open source models and sees it as the next big wave in enterprise AI use.
Linux Foundation’s Agentgateway project
She’s not alone. Vercel (best known as a platform for building and hosting websites, which has recently added AI model-switching tools) and OpenRouter (a company that helps developers route requests across different AI models) are both seeing a surge in traffic to open source models. In fact, open models accounted for 29% of all traffic routed through Vercel’s gateway last month.
With the CEO of Microsoft, a company that has invested in both OpenAI and Anthropic, now openly urging enterprises to be wary of using proprietary models, we’ll bet this trend continues to grow. “In consuming intelligence, you are creating intelligence. And what you create should belong to you,” Nadella writes.
Topics
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.
we may earn a small commission
Venture Editor
Last chance to save up to $190 on TechCrunch Founder Summit. Join 1,000+ founders and VCs at all stages for real-world scaling insights and connections that move the needle.Savings end June 26, 11:59 p.m. PT.
Most Popular
Satya Nadella has issued a shocking warning to companies using AI
Julie Bort
Satya Nadella has issued a shocking warning to companies using AI
Satya Nadella has issued a shocking warning to companies using AI
Satya Nadella has issued a shocking warning to companies using AI
Julie Bort
Reed Jobs would rather talk about curing cancer than his last name
Connie Loizos
Reed Jobs would rather talk about curing cancer than his last name
Reed Jobs would rather talk about curing cancer than his last name
Reed Jobs would rather talk about curing cancer than his last name
Connie Loizos
Phia accused of ‘cookie stuffing,’ taking affiliate credit on purchases it didn’t earn
Dominic-Madori Davis
Phia accused of ‘cookie stuffing,’ taking affiliate credit on purchases it didn’t earn
Phia accused of ‘cookie stuffing,’ taking affiliate credit on purchases it didn’t earn
Phia accused of ‘cookie stuffing,’ taking affiliate credit on purchases it didn’t earn
Dominic-Madori Davis
Meta removes controversial AI feature on Instagram after backlash
Lucas Ropek
Meta removes controversial AI feature on Instagram after backlash
Meta removes controversial AI feature on Instagram after backlash
Meta removes controversial AI feature on Instagram after backlash
Lucas Ropek
Apple sues OpenAI over alleged trade secret theft
Sarah Perez
Apple sues OpenAI over alleged trade secret theft
Apple sues OpenAI over alleged trade secret theft
Apple sues OpenAI over alleged trade secret theft
Sarah Perez
Elon Musk praises Mythos/Fable, promises not to ‘cut off’ Anthropic
Julie Bort
Elon Musk praises Mythos/Fable, promises not to ‘cut off’ Anthropic
Elon Musk praises Mythos/Fable, promises not to ‘cut off’ Anthropic
Elon Musk praises Mythos/Fable, promises not to ‘cut off’ Anthropic
Julie Bort
Instagram users: Here’s how to stop Meta’s AI from using your photos
Lauren Forristal
Instagram users: Here’s how to stop Meta’s AI from using your photos
Instagram users: Here’s how to stop Meta’s AI from using your photos
Instagram users: Here’s how to stop Meta’s AI from using your photos
Lauren Forristal