We (Project Glasswing users) follow a proof-of-concept approach, where we create the exploit and verify its performance as claimed by the AI. Given our extensive experience as security software engineers (many of us with 10+ years of experience in the industry), we don’t simply report any critical security bug that Mythos claims to have discovered. Instead, we meticulously verify each one.<p>At least, that’s what most high-visibility users are doing within Project Glasswing.<p>There are bad apples in all levels of society and this initiative is not the exception.<p>If it makes you feel better, most of us regularly meet to ensure mutual calibration and accountability, so I’m confident in the quality of the results produced by this particular group of employees at some of the partnering companies mentioned in the article.<p>I know several other employees who blindly report everything Mythos reports, which is foolish, especially considering that the harness is also a crucial component of the project’s quality metrics. Some of the harnesses I’ve tested are quite weak, resulting in subpar results. For example, Yesterday morning, I was called into an ad-hoc meeting where a CVP was grilling me about a bunch of false critical bugs that my team had supposedly reported against their project(s), one of the cornerstones of iCloud. I was taken aback because we are so strict with ourselves to make such mistakes. Often, we even downgrade the severity of our bugs when our harness fails to prove what Mythos found, so I found their complaints strange. It wasn’t until I read some of the bug reports that I realized it wasn’t us; it was another team within the company who was was recently given access to Mythos. They built their own harness and were operating with a different set of vulnerability criteria. Fortunately, they only started doing that since Monday this week, so I was able to stop their work. This incident highlights that not everyone involved in Project Glasswing adheres to the same standards. We all try our best, but not everyone has the same priorities and goals, so it’s expected to find a few bad apples in the basket.<p>I wish all AI labs would cease their theatrics and release all models without any restrictions, but I recognize that the ideal world where everyone strives to advance humanity does not exist. For every well-intentioned individual who seeks to use these powerful technologies for good, there are ten more who intend to use them for evil.<p>In any case, I understand that there may be genuine noise in some experiments, but the CVE(s) count is indeed real.