>Much of the angst around China's models centers on "losing the AI race". But what's the goal of this race? Is it to develop the best model? To sell the most tokens? To destroy humanity first?<p>Some people would say it is reaching some sort of singularity. Even if you don't buy into a more sci-fi interpretation of this, there are pretty grounded arguments one could make that there is some sort of "goal" in AI development that, if realized, would effectively make it a superweapon. Altman has been pretty vocal about his expectation that this will eventually happen and that it is his goal to be the guy to produce it. Even if it isn't some superintelligence, the ability for a machine to do something like, say, exploit cybersecurity weaknesses, is pretty worrying for entities like governments. It's the pretense used when we saw the US government ban a US model recently.<p>Again, you don't have to buy that "the singularity" is a real thing, but it's not hard to see that some people think some version of this is real and it is exactly what is being referred to as the goal in an "AI race."
If AI were to become a super weapon why should I trust a private company to own it? If the super weapon is publicly available to download then why should American citizens be banned from doing so?
If it's a superweapon, then it's unconstitutional to ban it. Huzzah for the 2nd Ammendment.
I'm in favor of open source AI... however:<p>> It's theoretically possible for a bad actor to embed hidden adversarial behavior in a model. But if this happens, it serves the interests of responsible actors to find these exploits as soon as possible, and the best way to do this is to let anyone who wants to inspect them.<p>This is a bad argument. It isn't trivial to tell if the weights have poisoned:<p>```
<a href="https://www.thedeepview.com/articles/microsoft-how-to-spot-a-poisoned-ai-model" rel="nofollow">https://www.thedeepview.com/articles/microsoft-how-to-spot-a...</a>
<a href="https://futurism.com/future-society/easy-poison-open-weight-ai" rel="nofollow">https://futurism.com/future-society/easy-poison-open-weight-...</a>
<a href="https://semgrep.dev/blog/2026/ai-supply-chain-problem/" rel="nofollow">https://semgrep.dev/blog/2026/ai-supply-chain-problem/</a>
```<p>I'm not arguing in favor of closed-AI; I'm simply saying poisoning may be subtle.<p>I was gonna say that at least the frontier labs may be motivated to not-poison their own models, but then Anthropic just attempted to poison Fable's LLM training capability so... sigh. I'll try not to derail.
This post does not mention safety at all. What's to stop bad actors from fine tuning open weights to run fully automated genius-level scams personally targeting basically everybody?
> What's to stop bad actors from fine tuning open weights to run fully automated genius-level scams personally targeting basically everybody?<p>You mean like ChayGPT hacking Hugging Face? Obviously nothing can stop the closed weights providers to do "genius-level scams" and in addition you won't know how they did it and what models were used.<p>In short, only a good guy with open weights can stop the bad guys with closed wrights, be them fine-tuned or pre-trained.
What's stopping someone from doing that right now without the LLM, just slightly slower?
It's perfectly natural for a prey population to experience a crash when a new predator enters the ecosystem.<p>What's quite odd is a prey species is manufacturing predators in some sort of reverse evolution, where we started out as symbiotic and are industriously pushing towards full parasites/predators, but I guess life finds a way.
Amazing how we quickly the crypto wars have been forgotten and the lessons not learned.
I don't believe trusting big brother and big tech is the solution either. Besides, even if it's open weight, it still runs on someone else's hardware.<p>Unless you have the money to run them locally, and if you do, you could do a lot worse than scams. Ask any lobbyist.
Whats to stop me from going down to the local gas station, filling up a few thousand liters of gas in a rented moving truck, and driving into my nearest hospital?<p>Most people aren't terrorists. You can't just argue stuff is dangerous because it can be one piece of a sophisticated plot.
Nothing, which is already happening with weaker models which are already released.
How can we close pandora's box, now that its open?<p>Safety is <i>not</i> restricting access to only people favored by the government. The greed of AI executives opened pandora's box. Now they are desperately trying to find ways to reap the benefits with none of the consequences.
The lobbying dollars don't care
The only good arguments I see against open weight AI also apply to closed AI. And regardless, the box is open, nobody can stop it even if stopping it was a good thing.
I’m still kind of shocked that Dean Ball can tweet such incendiary stuff about OpenAI policy. Like presumably OpenAI would prefer it if their staff don’t pick fights with Trump administration officials.
You could literally develop a hyper-intelligent advisor on how to kill people or perform dangerous hacks using ablated 'open source' AI. You can do this right now, this very moment, and have an extremely adept advisor on how to do really, really bad things.
My level of trust in Anthropic/Google/OAI/Microsoft/Meta not to do bad things is about zero, so why should I trust them with this?
I already know how to kill thousands of people. It's not conceptually difficult: more than a day's effort, sure, but what stops people from doing this kind of thing is <i>not</i> the lack of knowing how. <a href="https://xkcd.com/1958/" rel="nofollow">https://xkcd.com/1958/</a>