TechCrunch Desktop Logo

TechCrunch Mobile Logo

Topics

Latest

Latest

AI

AI

Amazon

Amazon

Apps

Apps

Biotech & Health

Biotech & Health

Climate

Climate

Cloud Computing

Cloud Computing

Commerce

Commerce

Crypto

Crypto

Enterprise

Enterprise

EVs

EVs

Fintech

Fintech

Fundraising

Fundraising

Gadgets

Gadgets

Gaming

Gaming

Google

Google

Government & Policy

Government & Policy

Hardware

Hardware

Instagram

Instagram

Layoffs

Layoffs

Media & Entertainment

Media & Entertainment

Meta

Meta

Microsoft

Microsoft

Privacy

Privacy

Robotics

Robotics

Security

Security

Social

Social

Space

Space

Startups

Startups

TikTok

TikTok

Transportation

Transportation

Venture

Venture

More from TechCrunch

Staff

Staff

Events

Events

Startup Battlefield

Startup Battlefield

StrictlyVC

StrictlyVC

Newsletters

Newsletters

Podcasts

Podcasts

Videos

Videos

Partner Content

Partner Content

TechCrunch Brand Studio

TechCrunch Brand Studio

Crunchboard

Crunchboard

Contact Us

Contact Us

OpenAI CEO Sam Altman speaks during the Microsoft Build conference at the Seattle Convention Center Summit Building in Seattle, Washington on May 21, 2024.

AI

OpenAI is scared of open-weight models. Should the US be?

Tim Fernholz

The impressive capabilities of Chinese lab Moonshot’s Kimi K3, the biggest open-weight large language model, have kicked off a debate that conflates two things: the economic possibilities of American AI giants and the future of LLMs as a technology.

OpenAI’s head of strategic futures, Dean W. Ball, went so far as to argue that the U.S. government should find a pretext to create regulatory fear, uncertainty, and distrust around the new models, since open-weight models must necessarily deter capital spending by the frontier labs.

argue

People freaked out, with tech luminaries like Yann LeCun and Martin Casado arguing that open software can accelerate innovation and coexist with proprietary projects. Ball soon retracted his claims that a regulatory crackdown was the White House’s “best strategy” and that open-weight models necessarily slow down advances in the technology.

freaked out

Yann LeCun

Martin Casado

retracted

However, Axios reports that the Trump administration is considering banning K3 and other advanced Chinese models at the behest of American frontier labs. Another report from Politico said that the Department of Commerce would not take that step anytime soon.

reports

report

The benefit for major AI companies is clear: Open-weight models, running on independent infrastructure or inside major enterprises, offer cheaper intelligence than Anthropic or OpenAI’s class-leading models. If users increasingly spend more outside the closed labs, that means smaller returns on their massive investments in model training.

That view extends far beyond OpenAI. “Strong, frontier-caliber open source models will place a squeeze on the margins and will bring down the prices of the frontier companies,” Braden Hancock, the co-founder of Snorkel AI and a research partner at the Laude Institute, told TechCrunch. “It will not necessarily mean that the amount of AI usage goes down a little bit. You know, obviously, quite the opposite.”

That’s not a problem for people without shares in Anthropic and OpenAI. AI will still proliferate. So what’s the justification for the government to block Americans from purchasing something in our ostensibly free markets?

Concerns over Chinese models come in several flavors. One is protecting U.S. data from the Chinese government; the U.S. banned the import of modern Chinese EVs over concerns about their data gathering. But experts tend to think that open-weight models run on U.S. servers are unlikely to leak data back to China, although it’s not impossible that such a thing could be done.

Another is that the models may have implicit bias toward the PRC — but it’s not clear what that might mean for, say, coding tasks.

A third common worry is that Chinese models lack the guardrails that the U.S. government has mandated (through an opaque process), which aim to prevent leading U.S. LLMs from being used to exploit closed computer systems or create weapons. However, those same guardrails may make U.S. companies more vulnerable: David Sacks, the venture capitalist and Trump adviser, has been sharing cases of U.S. companies turning to Chinese LLMs to close security gaps when U.S. frontier models refuse to do the tasks.

through an opaque process

sharing cases

But the most significant motivation for restricting the models is fear that China will be able to outpace the U.S. if the frontier labs slow down.

Sam Bresnick, a China-focused research fellow at Georgetown’s Center for Security and Emerging Technology, says the growing importance of AI to the U.S. military operations gives the U.S. a reason to support continued investment in AI at the frontier labs. But the whole question, he says, is fraught.

“Why should the weight of the U.S. government be aimed at protecting these these companies from competitors that are being locked out from the U.S. market based on their origins?” Bresnick asks.

Advocates for open AI say that the frontier companies are creating a false binary between innovation and closed models.

“The bigger impact of having these open source models come from China is less that they’re sneaking in back doors, and more that they are owning the innovation,” Hancock told TechCrunch. “You end up with, effectively, an expanded workforce on your model. PyTorch became the industry standard because it was open source, and so the whole community could contribute to it rather than just one company, and it grew and grew, and all the rest of the deep learning libraries kind of died in comparison.”

Hancock and other advocates fear that Chinese LLMs will become the locus of international research. Already, U.S. graduate programs mainly build on open-weight Chinese models, and Hancock says that half of the papers students study are coming from Chinese institutions, with American frontier labs increasingly reticent about sharing their work widely.

“Restricting open models wouldn’t make AI safer,” said Clem Delangue, the CEO of Hugging Face, a platform for open AI collaboration. “It would simply hide the risks, concentrate power in the hands of a few and make it harder for the next generation of builders, researchers, academia, nonprofits, governments to participate in making AI safer and more beneficial for all.”

Bresnick says that the real way to slow China would be to focus more on chip export controls. A better way to preserve U.S. AI leadership would be to stop selling Nvidia H200 processors to China. “That,” he says, “could potentially keep us out of this thorny debate about banning open source technologies that huge numbers of U.S. companies want to use.”

Part of the problem is that uncertainty around AI economics. “The open business model, the proprietary business model — neither one is figured out. AI companies are struggling to figure out how to make money on their tools, especially as training costs need to go up and up,” Bresnick points out.

The same challenges that play out in the U.S. are also playing out in China, where AI companies are also struggling to generate revenue and access compute power, and the government is seen as encouraging open releases for policy reasons despite the challenge in capitalizing on them.

Some U.S. companies, including Thinking Machines Lab and Nvidia, are trying to make a business around releasing open models. Hancock points out that Nvidia would do better “if there are dozens or hundreds of companies building AI rather than two or three that are well capitalized enough to make their own chips,” which is one reason behind its investment in Nemotron, a collection of open models.

Nemotron

“The main point is the U.S. would be very well served to have its own very capable, much less expensive open models,” Bresnick said. “It just clashes with the approach the frontier labs have taken.”

With additional reporting from Rebecca Bellan.

Topics

AI

kimi

OpenAI

When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.

we may earn a small commission

Tim Fernholz

Senior Reporter

View Bio

Event Logo

Scale faster. Grow your portfolio. Gain practical expertise. No matter your goal, Disrupt can empower you.Save up to $330 today!

REGISTER NOW

Most Popular

Judge pauses $110B Paramount-Warner Bros. merger

Aisha Malik

Judge pauses $110B Paramount-Warner Bros. merger

Judge pauses $110B Paramount-Warner Bros. merger

Judge pauses $110B Paramount-Warner Bros. merger

Aisha Malik

Aisha Malik

Apple and Google ordered to purge ‘nudify’ apps from App Stores

Lucas Ropek

Apple and Google ordered to purge ‘nudify’ apps from App Stores

Apple and Google ordered to purge ‘nudify’ apps from App Stores

Apple and Google ordered to purge ‘nudify’ apps from App Stores

Lucas Ropek

Lucas Ropek

Coca-Cola suspended production at its Fairlife dairy after a ransomware attack

Zack Whittaker

Coca-Cola suspended production at its Fairlife dairy after a ransomware attack

Coca-Cola suspended production at its Fairlife dairy after a ransomware attack

Coca-Cola suspended production at its Fairlife dairy after a ransomware attack

Zack Whittaker

Zack Whittaker

Tesla driver in fatal Texas crash pressed accelerator 100%, NTSB confirms

Sean O'Kane

Tesla driver in fatal Texas crash pressed accelerator 100%, NTSB confirms

Tesla driver in fatal Texas crash pressed accelerator 100%, NTSB confirms

Tesla driver in fatal Texas crash pressed accelerator 100%, NTSB confirms

Sean O'Kane

Sean O'Kane

Amid hardware legal battle, OpenAI releases a $230 keyboard for Codex

Lucas Ropek

Amid hardware legal battle, OpenAI releases a $230 keyboard for Codex

Amid hardware legal battle, OpenAI releases a $230 keyboard for Codex

Amid hardware legal battle, OpenAI releases a $230 keyboard for Codex

Lucas Ropek

Lucas Ropek

Microsoft patches bug in video game Age of Empires II

Lorenzo Franceschi-Bicchierai

Microsoft patches bug in video game Age of Empires II

Microsoft patches bug in video game Age of Empires II

Microsoft patches bug in video game Age of Empires II

Lorenzo Franceschi-Bicchierai

Lorenzo Franceschi-Bicchierai

Anthropic, Blackstone bet the next trillion-dollar AI business is implementation, not just models

Rebecca Bellan

Anthropic, Blackstone bet the next trillion-dollar AI business is implementation, not just models

Anthropic, Blackstone bet the next trillion-dollar AI business is implementation, not just models

Anthropic, Blackstone bet the next trillion-dollar AI business is implementation, not just models

Rebecca Bellan

Rebecca Bellan