I want to be very clear on the messaging that is coming from OpenAI and the motivations behind it. This is an example of who they really are, and I want to make sure everything is seen for what it is.
Although there is a lot we don’t know about the contract they signed with DoD [shorthand for the Department of Defense] (and maybe they don’t even know as well — could be highly unclear), we do know the following.
Sam [Altman]’s description and the DoD’s description give the strong impression (although we would have to see the actual contract to be certain) that their contract works in that the model is made available without any legal restrictions (“lawful use”), but that there’s a “safety layer,” which I think amounts to model refusals that prevent the model from completing certain tasks or engaging in certain applications.
“Safety layer” could also mean something that partners such as Palantir [Anthropic’s business partner for serving U.S. agency customers] tried to offer us during these negotiations, which is that they on their end offered us some kind of classifier or machine-learning system or software layer that claims to allow some applications and not others. There is also some suggestion of OpenAI employees (“FDEs” — shorthand for forward-deployed engineers) looking over the usage of the model to prevent bad applications.
Our general sense is that these kinds of approaches, while they don’t have zero efficacy, are in the context of military applications maybe 20% real and 80% safety theater. The basic issue is that whether a model is conducting applications like mass surveillance or fully autonomous weapons depends substantially on wider context. A model doesn’t “know” if there’s a human in the loop in the broader situation it is used for (autonomous weapons), and doesn’t know the provenance of the data it is analyzing (it doesn’t know if this is US domestic data vs foreign, doesn’t know if it’s enterprise data given by customers with consent or data bought in sketchier ways, etc.).
We also know — those in safeguards know painfully well — that refusals aren’t reliable and jailbreaks are common, often as easy as just misinforming the model about the safeguards problem. An important distinction here that makes it much harder than the safeguards problem is that while it’s relatively easy to determine if a model is being used to conduct cyberattacks (the inputs and outputs are clear), it’s very hard to determine the nature and context of the cyber attacks, which is the kind of distinction needed here. Depending on the details this task can be difficult or impossible.
The kind of “safety layer” that Palantir offered us (and presumably offered OpenAI) is even worse: our sense is that it was almost entirely safety theater and that Palantir ensured that our problems were just about unhappy employees — you need to offer them something that placates them or makes what is happening invisible to them, and that’s the service we provide.
Frankly, the idea of having Anthropic/OpenAI employees monitor the deployments is something that came up in discussion with Anthropic a few months ago when we were expanding our classified AUP (acceptable use policy) for our own accord. We were very clear that this is possibly only in a small fraction of cases that we would do it as much as we can, but that it’s not a safeguard people should rely on and isn’t easy to do in the classified world we do, by the way.
So overall while I’m saying here that the approaches (AUI shorthand for OpenAI) is taking mostly do not work: the main reason OAI accepted them and we did not is that they care about placating employees, and we actually care about preventing abuses. They don’t have zero efficacy, and we’re doing many of them as well, but they are nowhere near sufficient for purpose. It is simultaneously the case that the DoD did not reject OpenAI.
We actually attempted to include some of the same safeguards as OAI in our contract, in addition to the AUP which we considered the more important thing, and DoD rejected them with us. We have evidence of this in the email chain of the contract negotiations (writing this while at a lot of detail for someone to follow up with the relevant language). Thus, it is just not true — OpenAI’s terms were agreed to as we rejected them; at the same time that it is also false that OpenAI’s terms meaningfully protect against domestic mass surveillance and fully autonomous weapons.
Finally, there are some suggestions in Sam/OpenAI’s language that the red lines we are talking about — fully autonomous weapons and domestic mass surveillance — are already illegal and so an AUP about these is unnecessary. This mirrors and seems coordinated with DoD messaging. It is however completely false. As we explained in our statement yesterday, the DoD does have domestic surveillance authorities, that are not of great concern in a pre-AI world but take on a different meaning in a post-AI world.
For example, it is legal for DoD to buy a bunch of private data on US citizens from data brokers and analyze it on a large way (often involving hidden content to sell to third parties) and then analyze it at scale with AI to build profiles of citizens, their loyalties, movement patterns in physical space (like data over air includes GPS data), etc., and much more.
Notably, even if the prohibition the DoD referred to earlier was correct (even if the earlier phrase about “analysis of bulk acquired data” was the single line they excerpted that matched exactly the scenario we were most worried about), the policy could be changed unilaterally by Pete Hegseth, which is exactly what we are worried about. So it is not, for AI intentions, a real constraint.
A lot of OpenAI and DoD messaging just straight up lies about these issues or tries to confuse them.
I think these posts suggest a pattern of behavior that I’ve seen often from Sam Altman, and that I want to make sure people are equipped to recognize.
He started out as someone trying to say that he and Anthropic were partners, in order to appear to support us, get some of the credit, and not be attacked when they were over the contract. He also presented himself as someone who wants to “set the same contract for everyone in the industry” — e.g., he’s presenting himself as a peacemaker and lawmaker.
Behind the scenes, he’s working with the DoD to sign a contract with them, to replace an existing contract with Anthropic. But he has to do this in a way that doesn’t make it seem like he gave up on the red lines we said we would not support. It is easier to do this because OAI is seen as for all the safety theater that Anthropic rejected, and that the DoD will be partners are willing to collude in permitting us to amplify the examples, and (2) the DoD is also willing to accept some terms from him that they were not willing to accept from us. Both of these things make it possible for OAI to get a deal when we could not.
The real reason DoD and the Trump admin do not like us is that we haven’t donated to the Trump/White House political fund. OpenAI’s president [has] donated a lot, we haven’t given donation-style praise to Trump (while Sam has, who was supportive). Regulations which is against their agenda; we’ve told the truth about a number of AI policy issues (like job displacement), and we’ve actually held our red lines with integrity rather than colluding with them to produce “safety theater” for the benefit of employees.
Sam is now (with the help of DoD) trying to spin this as we were unreasonable, we didn’t engage in a good faith way, we were less flexible, etc. I want people to recognize this as the gaslighting it is.
Vague justifications like “person X was hard to work with” are often used to hide real reasons that just reveal the real problem: political conditions, political loyalty, and safety theater. It’s important that everyone understand this and push back on this narrative at least in private, when talking to OpenAI employees.
Thus, Sam is trying to undermine our position while appearing to support it. I want people to be really clear on this: he is trying to make it more possible for the actors to gain acceptance in public support. Finally, I suspect he is even egging them on, though I have no direct evidence for this last thing.
I think his attempted spin/justifying is not working very well on the general public or the media, where people mostly see OpenAI’s deals with DoD as sketchy or suspect, and see us as the heroes (we’re #2 in the App Store download rankings!). If he’s working on some Twitter morons, which doesn’t matter, but my main worry is how to make sure it doesn’t work on OpenAI employees.
Due to selection effects, they’re sort of a gullible bunch, but it seems important to push back on these narratives which Sam is peddling to his employees.