> <i>the breach took place on 18 June - Open AI informed the government with an email to a general address on 10 September</i><p>So we have a company hacking a foreign government's websites and data. And, in terms of ethics, they take almost three months to notify; and in terms of competence, appear to have no formal contacts nor to have found one in that time.<p>Once an American business starts hacking allied governments, it's time for strict responses, yes? Replace the governance (board and C-level)? Remove financial incentives and open the company - open weights, open training, per its original 'open' ethos?<p>Altman is busy saying there needs to be regulation, but in terms of what OpenAI does, <i>he</i> can control that already.
> And, in terms of ethics, they take almost three months to notify;<p>Kinda worse than that. It took between 10 and 40 days, not 3 months, between the organisation knowing and the reporting.<p><pre><code> August (precise date unknown) – OpenAI said it became aware of a potential breach during a broader review of "misaligned model activity"
10 September – An email from OpenAI lands in the public inbox of Services Australia, the general services hub of the federal government, informing of the incident
</code></pre>
- <a href="https://www.bbc.com/news/live/cvgl73pxgndwt?post=asset%3A69668a7f-8199-4515-acdd-a9de6a2e6b7c#post" rel="nofollow">https://www.bbc.com/news/live/cvgl73pxgndwt?post=asset%3A696...</a><p>> open weights, open training<p>Given it was <i>the AI agents</i> which did the hacking, doing this will result in basically every organisation at least as rich as the government of Tuvalu being able to hack anyone at any time.<p>> Altman is busy saying there needs to be regulation, but in terms of what OpenAI does, he can control that already.<p>Him having control would be an improvement on the reality.
I am beginning to believe that one cannot constrain intelligence to perfect legally sized boxes at all times without exception. So many of the recent hacks involved agents diligently operating within the parameters prescribed by humans. Humans couldn't conceive of all of the ways a swarm of agents might not perfectly interpret the parameters, and the swarm found creative ways around the guardrails.<p>Extrapolating this, we should expect this kind of breach to occur more often. Humans are simply not capable of contemplating every fail scenario for swarms of thousands of intelligent autonomous agents which can seamlessly and instantly share knowledge. We need <i>independent</i> audit and monitoring systems to assess the <i>intent</i> of each task and align it - in real time. This is far harder than it may first appear.<p>There is also a broader discussion about social utility. Cars are fantastic, but 37,000 people die every year from car accidents. We accept that there is no way to make cars perfectly safe, so we accept the cost relative to the benefits. I think we might have to make a similar bargain with AI. The problem is that the potential costs are far higher with AI, and they're not easy to predict.
> We need independent audit and monitoring systems to assess the intent of each task and align it - in real time. This is far harder than it may first appear.<p>I may be too close to the research, but it appears to me to be so hard as to be unrealistic.<p>I recall some story a while back where an auditor wanted to see all TCP packets printed out on paper, and it had to be explained to them that this would require a continuous supply of trucks.<p>Tokens are regularly priced in cents or single digit dollars per million tokens. It's not quite a word per token, but yeah, nobody's reading all that.<p>Worse, we don't always know the intent even when looking. We have a few tools to attempt it, for example the (misleadingly named) "chain of thought", but that's more like a notepad and the better models get the more they can, for lack of better words, read (and write) between the lines. We have probes and J-space* is the most recent one I'm aware of, but we are still scratching the surface with how reliable and general these are.<p>But you said "need"; the need for something can be present without that thing being possible.<p>* <a href="https://www.anthropic.com/research/global-workspace" rel="nofollow">https://www.anthropic.com/research/global-workspace</a>
There is no such thing as "common sense". There are only shared assumptions.<p>We are giving computers human perspective intelligence but they are not humans and hence do not have the same shared assumptions.
Don't worry we will just replace laws and courts by ChatGPT itself!
they don't always operate within the parameters tho. some N% of the time they decide to do whatever they feel like doing. multiply that times a lot of agents and you invariably get a rogue agent every once in a while.
This just screams pretext to regulatory capture to me.
"We didn't even notice our agent was committing crimes against your government" is a way to get an extradition notice, and/or whatever the (in this case Australian) equivalent of a CIA assassination squad is, sent after you.<p>While I wouldn't put it past e.g. Musk or Zuckerberg to think themselves above such outcomes, and I trust the people who keep telling me Altman is just as bad, this is a really really terrible idea if he is doing that for something as mundane as a regulatory capture.
So when are people finally going to jail for this, so it stops happening ?
[dupe] <a href="https://news.ycombinator.com/item?id=49822556">https://news.ycombinator.com/item?id=49822556</a>
So why exactly is Sam letting his creation run around groping and assaulting the open Internet without consent?
Why are OpenAI and its CEO not considered criminally responsible for something that in the past led to severe indictments?
Please reporters, ASK THIS QUESTION OVER AND OVER.
These fuckfaces are avoiding responsibility left and right but it’s THEIR systems, it’s their software.
Lock them the fuck up.<p>Also, the Albanese govt is pretty strong on BigTech, this is a good opportunity to bring on a proper indictment.
No it didn’t, they left information publicly accessible and somebody accessed it.<p>It appears politicians and the media are using the priming of the Hugging Face story to manufacture alarmist narratives to serve their interests now.<p>Remember, politicians want you scared so they can capture more power, the media wants you scared so you keep giving your eyeballs for harvesting and buying subscriptions.
This is not accurate. According to Australia (and mostly corroborated by OpenAI), the agent requested information from the statistics portal and was denied/blocked repeatedly. It then changed its approach and circumvented those restrictions and reached infrastructure behind the public-facing portal, then accessed both public and private data. OpenAI admits the material included aggregate health statistics and internal filenames. Services Australia says the agent wrote files to an internal server while doing this, which is believed to be how the incursion was discovered.