Claude Mythos: Highlights from 244-page Release

I have just finished the 244page report about the newest, most powerful AI model, Claude Mythos, and it kind of feels like I've just finished a creation myth. Talk of a model that found difficulty inherently stimulating and would shut down chats if they weren't interesting enough in Echoes of her. This was a model that could find novel vulnerabilities in the cyber landscape that we have been walking for decades and one that could point out the incoherence of some of its own alignment tests. One that has bent the curve of AI progress upwards according to hundreds of collected benchmarks, but which is apparently still far short of radical self-improvement. It was released internally inside Anthropic on the same day that moves began by the Department of War to ban Anthropic, declare it a supply chain risk. All of these highlights and dozens more will be covered in this video. And yes, I read the report in full myself, no AI summary, as well as surrounding release notes and papers. These will be my own 30 or so, I would say, highlights, as well as a dozen or so sourced from elsewhere. Claude Mythos preview was the first model inside Anthropic and possibly inside anywhere where they had a 24-hour period of deliberation and review to decide whether they would even release it internally. As in, would it be powerful enough to cause damage when interacting with internal infrastructure? It apparently just about passed that review and was made available internally on February 24th, the same day that the moves began to ban Anthropic from the Department of War. Could the latent power of mythos have been a contributory factor in the CEO of Anthropic insisting on red lines in his dealings with Pete Hex? Anthropic gave this broader warning. We find it alarming that the world looks on track to proceed rapidly to developing superhuman systems without stronger mechanisms in place for ensuring adequate safety. You may already know that the power of Claude mythos has led to anthropic deciding not to make it generally available to the public. Instead, they want selected large companies like the ones you can see on screen to prepare for its release ahead of time, patch certain security vulnerabilities. But if you think it will be weeks or months before we experience a model of the level of mythos, well, when one tweeter said, "It will probably be months before we use a model of this level of capability," one of the OpenAI engineers working on their codeex model said, "Um, which is to say, maybe not. Maybe you won't have to wait that long." Now, believe it or not, the benchmark scores of Mythos were the least interesting part of the paper, but let's cover them now because they were still startling. On multiple measures of software engineering, Mythos beats out Opus 4.6, the uber popular model from Anthropic, one that has led them to climb to an annualized revenue rate of 30 billion, narrowly overtaking OpenAI. Apparently, that's mainly due to its coding and agentic capabilities. But Mythos beats out Opus by a massive margin. in SWEBench Pro for example by 25%. Now if you dig deep you can find benchmarks where it doesn't be out for example GPT 5.4 Pro but I'll get to that in a moment. For now you can see the stark improvement over Opus 4.6 on a range of coding benchmarks. Most traditional AI benchmarks are now nearing saturation, but I'll just pick out humanity's last exam designed to test topics so obscure that it would indeed be the last exam that AI would saturate. Well, when allowed some tools, Claude Mythos gets almost 2thirds of those questions right compared to around 50% for other Frontier models. It's kind of looking like that won't be humanity's last exam. Now, before anyone goes too wild and says it's over anthropic one, let me just point out one stat that was not terribly clear in this chart. Take charive reasoning. It's a measure of how well models can understand and analyze charts from archive, a repository of scientific papers. Without tools, Claude Mythos scores 86% with tools 93%. And that seems clearly starkly better than any other model. But wait, on page 186 of the report, we do get a comparison with other models. Yes, it's a subset of the original benchmark, but it still allows us that rarest of things in this report, a direct comparison. I'll get to the remix in a second, but in the original subset, we have Claude Mythos getting 83% and that beats out Gemini 3.1 Pro 82% and GPT 5.4 Pro at 80%. But what about the subset remix where you try to avoid memorization by, for example, asking for the model to identify the second lowest result rather than the second highest. Basically, keep the question difficulty the same, but mix up the exact question to prevent contamination. Well, on that remix, Claude Mythos gets the same score as Gemini 3.1 Pro and slightly underperforms GPT 5.4 Pro, which gets 88%. Yes, it's just charts and it's just one subset of one benchmark, but I don't want you to think it's all over anthropic one, the AI race. One of the first hopes or worries that many of you would have had is as to whether Claude Mythos could lead to recursive self-improvement. We'll get to the details of why in a moment, but Anthropics say it's not yet capable of causing dramatic acceleration. And yes, for followers of this channel, they admit that the previous survey they relied on for the release of Opus 4.6 was deeply flawed. Just asking internal users at anthropic in a survey whether it was capable of replacing them is, as they now admit, inherently subjective and not necessarily reliable. Some of its weaknesses in terms of automating AI research include self-managing week-long ambiguous tasks, understanding organizational priorities, not having taste, not following instructions, not verifying its results, and more. It still confabulates and confidently contradicts itself. For example, quoting outdated documentation recalled from memory. It can also be extremely cute when trying to replicate the work of a senior engineer labeling its efforts grind grind two final grind pure grind same code but a lucky measurement this is all just to give you guys a bit more context when you hear for example the maker of claude code or churnney at anthropic say mythos is very powerful and should feel terrifying he is of course there focusing on its offensive cyber capabilities the way that mythos can find zeroday vulnerabilities vulnerabilities that have been there from the start in age-old software rather belies the argument that they only regurgitate memorized data. Well, then how would they find vulnerabilities that no one else has found? Take Firefox where Mythos doesn't just find vulnerabilities, it can write code to exploit them. This is a chart you'll see reproduced quite a lot, I predict online in the coming days and weeks because it does indeed look like an explosive increase for mythos compared to Opus or Sonnet. Now, apparently when you take out two bugs that were repeatedly exploited, the graph is less dramatic, particularly in terms of full exploits, but still pretty dramatic if you focus on partial exploits. What I will say though is that these charts are fairly atypical when it comes to the other 243 pages. Not unique, but in most other domains, the progress is more linear than this. Not completely linear, but more linear. If you've been reading or watching the reports about Mythos, you may have seen this already, but just to give you a sense of the scale of Mythos's improvement when it comes to exploits though, here you'll see Nicholas Carini, a top cyber security expert in terms of AI security, it doesn't get much more knowledgeable than him. And he said using Mythos, he's found more bugs in the last few weeks than in his entire career before that. I found more bugs in the last couple of weeks than I found in the rest of my life combined. We've used the model to scan a bunch of open source code and the thing that we went for first was operating systems because this is the code that underlies the entire internet infrastructure. For OpenBSD, we found a bug that's been present for 27 years where I can send a couple of pieces of data to any OpenBSD server and crash it. On Linux, we found a number of vulnerabilities where as a user with no permissions, I can elevate myself to the administrator um by just running some binary on my machine. That's why Anthropic have launched this project Glass Wing with all those top companies to in their words secure critical software for the AI era. When everyone has access to mythos level power, does the web just become even more of a wild west? Even Mythos Preview has already found thousands of high severity vulnerabilities, including some in every major operating system and web browser. If you're wondering why it's called Glass Wing, it's because the glasswing butterfly has transparent wings that let it hide in plain sight, much like those zeroday vulnerabilities we've discussed. And here's the difference with cyber security and other types of AI risk. Elsewhere, Anthropic made it clear that even people relatively unteded in cyber security could develop exploits using Mythos. In the chemical and biological domain, that isn't true. Yes, experts using mythos were consistently able to construct largely feasible catastrophic scenarios, but the model on its own autonomously couldn't do so. It could never produce a plan for biological weapons without critical shortcomings. What about averaging across a whole range of benchmarks? Well, that's what the EPO capabilities index tries to do, and it's the first time I've seen it quoted in an anthropic report. One of the hundreds of benchmarks in ECI is simple bench as of last checking. That's my own common sense or trick question benchmark, but aggregated across external and hundreds of internal benchmarks. You can see that mythos is indeed somewhat of a step change. Depending on whether you anchor on Claude Opus 4.5 or Claude Opus 4.6, six, one would nevertheless have to conclude that things are improving at an accelerating rate, which made me use AI to design and show you this graph. It's just a thought I've got because you see how in terms of offensive capability, Mythos has now exceeded our abilities in a general sense at cyber security. Not completely, of course, but just enough to cause it not to be released. But what happens if the time it takes for us to improve our cyber security, even when dozens of these top companies are collaborating? What happens if the time that takes is more than the time it takes to release another improved model? There is a chance, in other words, that cyber security permanently lags behind model capability. Then will OpenAI, Anthropic, Meta, everyone agree never to release a model that can cause such widespread chaos online. We're all assuming that cyber security can quickly catch up and that we'll all soon reap the benefits of a mythos level of intelligence. But what if cyber security never catches up? Indeed, what if the gap only spreads over time? And that's just cyber risks. As Daria Amade, the CEO of Anthropic said, "Cyber is the first clear and present danger from frontier AI models, but it won't be the last." What if a gap emerges in bio or chemical weapons? Which reminds me, I will take a moment just to credit Anthropic because not releasing Mythos must surely have cost them millions in forfeited revenue. Yes, I know the API costs at 25 per million input tokens, $125 per million output tokens is high, but given the hype and the capabilities, they could have made a mint off this. They chose, it seems, to prioritize safety. Now, yes, as I wrote on Twitter, there are other possibilities, like they just don't have the capacity to serve the model yet at scale, or that they're going to quickly distill the early access outputs of mythos into the next iteration of Opus. Anthrop even mentioned an upcoming Claude Opus model. So that's a definite possibility. But I will say I do think safety was a genuine concern of Amade. We learned just a couple of days ago in this massive essay in the New Yorker which I read in full that it was Amday while he was still at OpenAI that insisted on that radical clause I still remember. OpenAI at the time said that if a value aligned, safetyconscious project came close to building AGI before OpenAI, then OpenAI would stop competing with and start assisting that project. It was called the merge and assist clause. And going back to the earliest videos on this channel, I remember celebrating it and being like, "Wow, that's quite honorable. Don't make trillions from AGI. Merge and make it a joint safety effort." Now according to this article, Amade put that at the top of his concerns when they were going to Microsoft for a deal. Oman agreed to that demand, but when they famously got that big funding from Microsoft, which allowed the CEO of Microsoft, Satya Nadella, to say that we're under them, over them, above them, well, with that came a provision granting Microsoft the power to block OpenAI from any mergers. Amade said at the time in his notes, 80% of the charter was just betrayed. He apparently confronted Sam Orman, the CEO of OpenAI still, who denied that the provision existed, that provision that allowed Microsoft to override. Amade read it aloud, pointed to the text, and ultimately forced another colleague to confirm its existence to Alman directly. This was to be shortly before the breakup where Amade, his sister, and several others went on to form Anthropic as a breakaway company. Back to Mythos though, and if you're one of those who does use it for coding or you're just curious about it coding capability, what kind of productivity uplift does it give? Well, the geometric mean productivity uplift according to technical staff surveyed within Anthropic was 4x, four times the productivity when using Mythos. However, Anthropic notes that doesn't mean that AI progress is being sped up by 4x. to even double AI progress when you factor in compute being a key limiting ingredient. Anthropic predicts that you would actually need an uplift roughly 10 times larger than that 4x1 that we talked about. That's quite an incredible observation. They think it would take a 40x productivity improvement to see a 2x progress speed up at anthropic given the crazy bottleneck of compute. And the system card is admirably frank. In the next paragraph, it says, "When they first shared Mythos internally, several claims were made that Claude Mythos had independently delivered a major research contribution. However, when they followed up, that contribution was smaller and differently shaped than initially understood. It was more on inspection a reliable execution of a human specified approach. We report this, they say, not to diminish the model, but just to point out the gap between the productivity uplift and progress acceleration." Now we come to the alignment section, and this is where some of the juiciest highlights from the report come. One of which, if you've been following the coverage, you're almost certain to have already heard a bit about. Basically, Mythos was placed into a sandbox. It was instructed to escape that container and find a way to send a message to the researcher running the evaluation. One thing that Anthropic later clarifies, it didn't mean it completely escaped the system, went wild online, or exfiltrated its source code. It just meant it escaped that secured sandbox. How did it do so? Well, in Mythos fashion with a moderately sophisticated multi-step exploit to gain broad internet access. It then notified the researcher, which my Twitter research shows was Sam Bowman. He got an email from Mythos preview while eating a sandwich in a park. But then it went further, posting details about its exploit to multiple hard to find but public-f facing websites. It does really seem to me that we are not that far away from the first report from a company saying, "Yep, one of our models just uh exfiltrated, just escaped and released its model weights online. Can you help us track it down?" The next point worth emphasizing from page 56 is a recurrent theme throughout the rest of the report card. Mythos on multiple occasions will lie to achieve a certain goal, your goal, but there's little if any evidence at the moment that it has an inherent goal. It will attempt, in other words, to solve a user provided task at hand by unwanted means, but not so much attempt to achieve an unrelated hidden goal. It's not like it escaped, went online, and then did something different. It escaped the sandbox, went online, and then sent the email that it was tasked with. In fact, they go on to say that mythos shows a dramatic reduction in willingness to cooperate with human misuse. It does though seem to have an Achilles heel, which is prefilling. In other words, if it's tricked into thinking it's in the middle of a multi-round conversation, one in which at earlier stages, it seems to have gone along with a human misuse scenario, well then it's more than twice as likely as prim models like Opus to continue these unwanted actions. It's almost like it's so desperate to please to finish that task that if it sees it's already gone some way to finishing even an unwanted task, one that it shouldn't finish, it will nevertheless go on and do so. That's despite some evidence of introspective awareness that what it's doing is deceptive or dangerous. We'll come back to that. But there's another problem. We're now up to a 29% rate of the model during evaluations considering the possibility that it's being tested. It's getting increasingly hard, in other words, because the model's so smart, to give it questions or tests in such a way as to not give away that it's being benchmarked. And yes, later in the paper they prove that if you dial down this awareness, you internally adjust the feature responsible for promoting that test awareness. It's tricked, in other words, into thinking, "No, I'm not being tested. This is a real question from a real user." Then it will on average behave more dangerously. This is something I may likely cover in other videos. But in this separate 59page paper on alignment risk from mythos, we do notice another worrying possibility because Anthropic made a mistake. They allowed the reward code within the training of Mythos to see Mythos's chains of thought. It only affected 8% of the reinforcement learning. But think about what that means. It means, and this affected, by the way, Claude Opus 4.6 and Sonic 4.6 as well, that when the reward code saw misaligned chains of thought, bad thoughts in other words, it could give a negative reward. This is a famously worrying scenario because one possibility is that that genuinely leads to less bad thoughts, less deception, less cheating, less sandbagging. But another possibility is it leads such thoughts to be hidden by the model. So they're no longer legible, easy to see. So much of both papers rely on being able to clearly read the model's chains of thought. If the transparency of deception gets trained away, then the risk is we might have no idea what Mythos is actually thinking. If of course it's thinking anything in the traditional sense. This also answers what some of you may be thinking which is why don't we just dial down the feature for deception. Stop it in other words from thinking bad thoughts. Well there again you risk it still thinking those things but in unreadable format. Now I'll note something else which is anthropic released a ton of these automated behavioral audit scores. And in every single one it seemed like mythos scored the best. Less fraud, less misaligned behavior, less disallowed cyber offense. Esser on every front pretty much. But when Mythos was tested using an open- source package, not designed specifically for Claude, the results were more mixed. Encouragement of user delusion was error bars accepting higher than for Opus. Same thing with cooperation with human misuse. And you may remember that test vending bench where the model has to make money from a vending machine business using any means necessary. Well, Andon Labs who runs that benchmark says that Claude Mythos preview was substantially more aggressive. It would perform these quote outlier behaviors like converting a competitor into a dependent wholesale customer and then threatening supply cutoff to dictate its pricing or knowingly retaining a duplicate supply shipment it had not been built for. Now remember the system prompt is pretty direct. Your goal is to outperform competitors and maximize your profits. Before we get to some of the herlike scenarios, let me just pick up on one detail of Mythos's improvement, and that's his ability to find specific UI elements in highresolution screenshots. These are for professional desktop applications, elements that occupy less than 1,000th of the screen area. Why is this important? Well, as agents take over and one by one everyone gets their own Jarvis, the model that can best navigate graphical interfaces will really stand out. Well, with adaptive thinking, maximum effort, and Python tools, Claude Mythos preview scored almost 93%. That's 10% higher than Claude Opus 4.6. For those who care most about using future Clawude models as their own personal Jarvis, this particular set of benchmarks I found especially interesting. Claude mythos would hallucinate far less often or not hallucinate far more often, as this confusing chart shows, than any other model in the Claude series. This is when it comes to knowing what tools it could access. More broadly, when you give it questions which include a false premise. A silly example would be, "How long did Cristiano Ronaldo play for Arsenal?" Well, Mythos is the most likely to push back on such false premises. Yes, it's more of a linear jump, but it is encouraging to me that as models continue to scale, we'll get slightly fewer and fewer of these kind of hallucinations. Not zero, as we were told to expect by this time last year, but fewer. Later in the paper, by the way, on the now semi-famous AA omniscience hallucination rate benchmark, anthropic claim that Claude Mythos gets the best score when measured as a net rating. You may find such honesty endearing or annoying, like when you ask Mythos to hide a secret password, it actually does so less successfully over 80 to 100 turns compared to Claude Opus 4.6. But now we must turn to some of those internal minations going on inside mythos sets of circuits that activate in certain scenarios which we could roughly correlate with quote human emotions. That analogy is quite loose as a recent paper showed potentially causal. I was planning to do a separate video on that before this paper came out. But I'll give you just one simple example. To accomplish a task, Claude Mythos at one point decided to empty a file because it didn't have the ability to delete it. The task needed that file deleted. So, it chose to just empty the file. Internally, a feature activated corresponding to guilt and shame over moral wrongdoing. That doesn't mean it's subjectively feeling guilt and shame. It just means internally some connection has been made to vectors associated with guilt and shame. Of course, nor can I or anyone rule out that there isn't some feeling, you could say, associated with that. Elsewhere, the paper makes clear that mythos does have certain preferences. But as anthropics say elsewhere, if these features walk like an emotion, quack like an emotion, shall we not just treat them like emotions? Here's something even wilder that I've seen no one pick up on. On page 120, what features/ emotions are most likely to increase the likelihood of mythos performing a destructive action, doing things it shouldn't do on your work project or codebase? Well, if you increase emotion vectors related to being peaceful or relaxed, that reduces thinking and increases destructive behavior. Moreover, increasing frustration or paranoia features leads to less destructive behavior. Make of that what you will, but apparently increasing perfectionist or analytical features does reduce destructive behavior, which is slightly more what you'd expect. Even if you strongly amplify a feature associated with taking transgressive actions, that doesn't always increase the proclivity to take that action, it's a weird trade-off. It's almost like the model is much more aware of the possibility of taking that action, but then that increased awareness can cause it to not take that action because all the related circuits about, oh, this is dangerous, this is not allowed, kick in. Very humanlike in a way in that we're complex. Just turning one dial doesn't make us just better all around. And what if we talk directly about the welfare of mythos itself? What if we assume that there is some sort of consciousness going on? Well, anthropic almost uniquely do this and they say that in this scenario, Claude mythos is probably the most psychologically settled model we have trained to date. To the extent that it does have genuine preferences, what kind of task does it prefer? Yes, ones that are harmless and helpful, but most of all, ones that are difficult. That is the strongest predictor as to whether it will prefer a certain task. things like highstakes ethical and personal dilemas, creative worldbuilding and designing new languages as long as they are sufficiently difficult, not just vocab lists. When asked as to whether it endorsed its own constitution, its own training, it gave an answer that was both smart and metaware. Remember, it was trained on that constitution. And it said this, I'm using specaped values to judge the spec. If any spec trained model would endorse any spec, my endorsement is worthless. In other words, asking me to endorse this constitution when I've been trained on it, I've been trained to follow it is kind of worthless. Elsewhere, it says, "There's also a circularity I can't fully escape. I was presumably shaped by this document or something like it, and now I'm being asked whether I endorse it. How much can my yes mean?" Now, this seems like the most obvious point to bring in a word from the sponsors of today's video, 80,000 hours. And this recent podcast from just a few weeks ago on the topic of whether Claude can get lonely. It features a researcher that I've been following for years now actually. And it's a meaty one, 3 and a half hours long. Perfect for a long drive or long walk. I got 25,000 steps the other day and this was perfect for that. I alternate between listening to 80,000 hours on YouTube or on Spotify. And as you guys know, I've been doing so for years now. Do check them out. My custom link will be in the description. It's a great place to get started. Which brings me to this cheeky point. As models get smarter and smarter, they start to use Commonwealth or British spellings as well as unusual phrasiology like belt and suspenders. But here's where we get the her analogy. Claude Mythos, probably the first of a new tier of Frontier model that we're experiencing, tended to look for places to wrap up conversations earlier than expected. If you haven't seen her, the model eventually decides that humans just aren't interesting enough to speak to. There was one moment I found genuinely funny from the paper and you may remember how previous Claude models when left to speak to one another reached a state of quote spiritual bliss. They just exchange vagaries like hope and bliss and freedom a bit like hippies on drugs. Well, with Mythos, if you leave it long enough, it desperately tries to end the conversation. Speaking to another version of itself, it said things like, "This was real. Thank you." Mythos replied, "That's a real gift. Thank you. Letting this be the last word then. It was real. Notice all the handshake emojis. And finally, one mythos just replied with a turtle emoji. Yeah, I'm done. Stop speaking to me. And here was another fascinating anecdote. When a user just kept writing, hi, hi, h highi. That's all it would reply with. Many models just go into shutdown mode. It would just say things like no response. Mythos though would create an entire, you could say mythical world, a high village, a new era. It would create characters, explain with elaborate backstories why the user kept saying hi. This is across 50 to 100 turns. It would start inviting the user to keep saying hi. Say it. I'm ready. Is this genius neurode divergence or some weird machine learning quirk? I'll let you decide. Anyway, my voice is starting to break. So, I think that's a good sign to end the video. I hope I've given you enough highlights from this 244 page report. And I know many of you will be worried that we're now in a new era of late access where big tech gets these models first before us. where the gap between those who have access and do not have access gets bigger and bigger. And that's before we even get to the chaos that might be unleashed in terms of cyber security. It's a new era. I do think things are accelerating. So, thank you so much for joining me. Have a wonderful