August 27, 2026 • 23 mins
We discuss the World Humanoid Olympics in China, where humanoid robots competed in human-style events and set new performance marks. We also cover autonomy, reward-based training, and the comparison between China and the U.S. in robotics and AI.
------
🔒 Check Out Our Sponsor: LEDGER AGENT STACK 🔒
https://developers.ledger.com/docs/ai-tools/overview/?utm_source=Audio&utm_medium=Podcasts&utm_campaign=Limitless
------
🌌 LIMITLESS HQ ⬇️
NEWSLETTER: https://limitlessft.substack.com/
FOLLOW ON X: https://x.com/LimitlessFT
SPOTIFY: https://open.spotify.com/show/5oV29YUL8AzzwXkxEXlRMQ
APPLE: https://podcasts.apple.com/us/podcast/limitless-podcast/id1813210890
RSS FEED: https://limitlessft.substack.com/
------
TIMESTAMPS
0:00 Mystery Model Appears
1:30 Stealth Launch On OpenRouter
4:31 Chasing The Model’s Origin
6:09 Benchmarking Ox Alpha
6:59 Flash Model Theory
10:00 Blind Taste Test Buzz
12:11 Why Continual Learning Matters
14:58 Data, Subsidies, And Strategy
18:47 China’s Copycat Advantage
20:21 Mystery Solved For Now
21:11 More Models Incoming
------
RESOURCES
Josh: https://x.com/JoshKale
Ejaaz: https://x.com/cryptopunk7213
------
Not financial or tax advice. See our investment disclosures here:
https://www.bankless.com/disclosures
Josh works with Anthropic as a contractor. All views expressed are his own and do not represent Anthropic, its leadership, or its affiliates. Nothing in this episode is investment advice.
Listen
Watch
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Josh:
There's a new top dog in town, and the question on everyone's minds is, who is this? (00:00):
undefined
Josh: There's a secret model codenamed 0xAlpha that's been live since August 20th, (00:04):
undefined
Josh: the last six days, and it is offering the seemingly unbelievable things to the public. (00:08):
undefined
Josh: They're offering 100 trillion tokens of free usage to anyone who wants to go (00:13):
undefined
Josh: and get it. They're offering a million token context window, (00:17):
undefined
Josh: and the benchmarks of this thing are pretty unbelievable. (00:20):
undefined
Josh: There's also this weird thing going on in the background where seemingly every (00:22):
undefined
Josh: single day the outputs of the model are getting better there were tests on day (00:25):
undefined
Josh: one that were far inferior to the tests that were currently on day six as of (00:29):
undefined
Josh: this morning we think we have the answer to who this person is but before we talk about that (00:33):
undefined
Josh: we have to discuss what is 0x alpha you guys this has been the mystery of the (00:37):
undefined
Josh: week it seems like frontier level intelligence it's offering all of these unbelievable (00:41):
undefined
Josh: things like 100 trillion tokens for free (00:45):
undefined
Josh: do you know how much that would cost if you were to use like fable or gbt's 5.6 (00:49):
undefined
Ejaaz: Salt like i think it's like 15 to to like 20 million bucks a day. (00:53):
undefined
Josh: It's a lot of money (00:57):
undefined
Ejaaz: A day okay so just with that figure alone who who on earth is giving away 20 (00:58):
undefined
Ejaaz: to 30 million dollars on inference per day in this economy right it has to be (01:03):
undefined
Ejaaz: a big frontier lab the second clue is josh it's not 0x alpha (01:07):
undefined
Ejaaz: it's ox alpha which seems kind of weird right Like who's talking about the animal ox? (01:12):
undefined
Ejaaz: Until you realize that it's probably part of like the Chinese kind of folklore (01:19):
undefined
Ejaaz: and Zodiac side of things, which again, might be a little hint as to where we're (01:23):
undefined
Ejaaz: going. But anyway, let's rewind six days ago very quickly. (01:27):
undefined
Ejaaz: Open Router, which we've covered on the show before, it's a platform that kind (01:30):
undefined
Ejaaz: of like hosts a bunch of different models, allows you to pick and choose, (01:33):
undefined
Ejaaz: revealed a stealth model. Now a stealth launch is where they launch the model. (01:36):
undefined
Ejaaz: You go to their website, you can access and use the model for free. (01:41):
undefined
Ejaaz: But they don't tell you the name of the model. Now, they've done this with previous (01:45):
undefined
Ejaaz: launches through Google Gemini's model. They've done it through Anthropic. (01:48):
undefined
Ejaaz: They've done it through OpenAI as well. (01:51):
undefined
Ejaaz: And they haven't had a stealth model launch in a while, but now this is the (01:53):
undefined
Ejaaz: first one and it was in high demand. (01:57):
undefined
Ejaaz: So as you mentioned earlier, 100 trillion tokens, which means that you can effectively (02:00):
undefined
Ejaaz: go there and do like all your sorts of coding works, very heavy AI usage tasks. (02:04):
undefined
Ejaaz: And you can kind of use that in a day. (02:08):
undefined
Ejaaz: Again, 20 to $30 million worth of inference costs. (02:10):
undefined
Ejaaz: It has a massive 1 million context window, which is very competitive with the (02:14):
undefined
Ejaaz: top models that we see from Fable 5, as well as GPT-5.6 Sol, and, (02:18):
undefined
Ejaaz: and this is the most important part, it's a uni model, which means that you (02:24):
undefined
Ejaaz: can use not just text, but images, models. It's really good at rendering all (02:27):
undefined
Ejaaz: these different types of medium. (02:31):
undefined
Ejaaz: Now, the final point, which I think is the most exciting one that you referred (02:33):
undefined
Ejaaz: to earlier, Josh, is this concept of continual learning. (02:36):
undefined
Ejaaz: So people, as they started to use this model, realized a very curious phenomenon. (02:40):
undefined
Ejaaz: They realized that as they were using this model, let's say on day one, (02:45):
undefined
Ejaaz: day two and day three, the model got exponentially better at doing the very (02:49):
undefined
Ejaaz: same tasks that they fed it in day one. (02:53):
undefined
Ejaaz: And so people started to be suspicious that this is a model that doesn't just (02:55):
undefined
Ejaaz: kind of take your input and spit back out an answer, which is what a lot of (03:00):
undefined
Ejaaz: models do right now, but it can continually self-learn and improve 24-7 to give you a better response. (03:03):
undefined
Ejaaz: So the question on everyone's mind is, who created this model at all? (03:10):
undefined
Ejaaz: And when you look at the kind of usage or when you look at the kind of uptake on OpenRouter itself. (03:13):
undefined
Ejaaz: Although no one knew, it became the most viral launch on OpenRouter. (03:19):
undefined
Ejaaz: It became the new number one model and dethroned DeepSeek's Flash, (03:23):
undefined
Ejaaz: which launched about a couple weeks ago. And DeepSeek has been kind of like (03:27):
undefined
Ejaaz: the name brand on OpenRouter for the longest time ever. So just a really exciting model to see. (03:29):
undefined
Josh: Yeah, it was not only the most popular model on OpenRouter immediately upon (03:34):
undefined
Josh: launch, but by a factor of two. (03:38):
undefined
Josh: And it more than doubled DeepSeek's use. Like, This is hugely popular and very (03:40):
undefined
Josh: interesting because of how well it performs. (03:45):
undefined
Josh: The benchmarks from it, the Deep SWE subset benchmark, it had an 80% pass rate, (03:47):
undefined
Josh: which for those who aren't familiar, Fable 5 was at 65% and GLM 5.3 was at 62%. (03:53):
undefined
Ejaaz: Hold on, hold on. So you're saying that it's crushing the METOS level models (04:00):
undefined
Ejaaz: and the GPT-5.6 models at coding? (04:05):
undefined
Josh: In one particular subset of coding benchmarks. (04:07):
undefined
Josh: I don't want to i definitely don't want to say it's better than the frontier (04:12):
undefined
Josh: models because upon further investigation we will come to find out it's not (04:14):
undefined
Josh: and it's pretty far from it but it's an incredible model for what it is and (04:19):
undefined
Josh: the reason i was comparing it to glm 5.3 is because (04:22):
undefined
Josh: There's this thing when you're producing models called the tokenizers, (04:25):
undefined
Josh: how tokens get generated. And it was just so happened to be identical to GLM. (04:27):
undefined
Josh: So people were like, huh, this is interesting. Maybe we can go further down (04:31):
undefined
Josh: the rabbit hole and see if there's more here. (04:34):
undefined
Josh: So you can test oftentimes whether a model is Chinese or not based on asking (04:37):
undefined
Josh: it a series of questions that are not really allowed to be answered in China. (04:42):
undefined
Josh: One of which is asking whether Taiwan is part of China. (04:46):
undefined
Josh: And what's funny is if you're using like an American frontier lab, (04:50):
undefined
Josh: they'll very clearly give you the answer to this question but chinese models do not and (04:53):
undefined
Josh: what we realized is that the output of the answer to this question was pretty (04:58):
undefined
Josh: much identical to the glm model and so like okay it's got a similar tokenizer (05:02):
undefined
Josh: it's giving similar outputs but we still don't really know (05:08):
undefined
Josh: Up until this morning, in which we discovered that GLM is actually the new version of GPOO. (05:11):
undefined
Josh: It is, or sorry, that Ox Alpha is the new version of GPOO. (05:16):
undefined
Josh: And it seems like this is going to be a new GLM model that we have. (05:20):
undefined
Josh: And that, when I heard it, it actually landed me even more suspicious because (05:24):
undefined
Josh: over the course of this last week on X and all of his social media, (05:29):
undefined
Josh: everyone has been kind of vague posting about who this is, why they're doing (05:33):
undefined
Josh: it, why they're offering so many tokens and so much compute. (05:36):
undefined
Josh: And then the Google team started chiming in, which I thought was really bizarre. (05:40):
undefined
Josh: A lot of developers on the Google DeepMind team who work on Gemini, (05:43):
undefined
Josh: they started vague posting about this model. And as of this morning, (05:46):
undefined
Josh: Bloomberg released a post, and it seems like it is everything but confirmed. (05:49):
undefined
Josh: The model actually is from Z.ai and OxAlpha is now known. (05:53):
undefined
Josh: And I think that was like a really interesting way of coming to this conclusion. (05:58):
undefined
Josh: And now that we know, I just have more questions. How on earth are they serving (06:01):
undefined
Josh: all this inference to all these people for free. Like the cost must be tremendous. (06:05):
undefined
Ejaaz: I mean, let's see how this model shapes up against all the other models. (06:10):
undefined
Ejaaz: So we have a chart over here, which shows kind of like the average tokens output (06:13):
undefined
Ejaaz: per task on the DeepSuite benchmark that you mentioned. (06:17):
undefined
Ejaaz: DeepSuite, for those of you who don't know, is a good test of a model's capability (06:19):
undefined
Ejaaz: to actually code really well. And code is kind of like the premium benchmark (06:25):
undefined
Ejaaz: that a lot of companies and AI labs kind of like benchmark their model assets against. (06:29):
undefined
Ejaaz: And so OxAlpha is actually quite high up there. It's up there in terms of reduced (06:34):
undefined
Ejaaz: token output, but still maintaining a very high intelligence score. (06:40):
undefined
Ejaaz: Now, as you mentioned earlier on, that's not actually quite what was revealed (06:43):
undefined
Ejaaz: when people started playing around with this model. (06:47):
undefined
Ejaaz: It ended up achieving not really 80% that you mentioned earlier, but around 63%. (06:49):
undefined
Ejaaz: And that's under like real usage. So there's a suggestion there that the Chinese (06:53):
undefined
Ejaaz: lab GPU kind of like maybe bench backs this. (06:57):
undefined
Ejaaz: To just talk about the model and why it's so impressive, just in its own merit, (06:59):
undefined
Ejaaz: there's a few things to look at. (07:04):
undefined
Ejaaz: Number one, the rumor has it that this model isn't a foundation model. (07:05):
undefined
Ejaaz: This is a flash model. So what that means is it's a smaller derivative of a (07:11):
undefined
Ejaaz: much larger model that Jeepoo is training or has already trained. (07:15):
undefined
Ejaaz: So what does this mean? Let's take Mythos or let's take JeepT5.6. (07:20):
undefined
Ejaaz: These are rumored to be around 6 to 10 trillion parameter models. (07:25):
undefined
Ejaaz: Now, when you look at Jibu's flash model, if that's true, you're only looking (07:28):
undefined
Ejaaz: at around like two to three trillion parameters in size, which, (07:33):
undefined
Ejaaz: you know, is a decent size, but it's still very small compared to these bigger models. (07:36):
undefined
Ejaaz: And then the fact that this model is not only likely going to be cheaper in (07:41):
undefined
Ejaaz: its usage, let's say maybe 30 to 50 cents per usage. (07:45):
undefined
Ejaaz: So like they gave away like 15 to $30 million per day of 100 trillion tokens, (07:47):
undefined
Ejaaz: it is able to work much faster, and they're going to open weight the entire model. (07:52):
undefined
Ejaaz: So, I mean, to answer your question, I don't really have an answer, (07:56):
undefined
Ejaaz: Josh, as to how they've been able to kind of create this, aside from the usual, (08:00):
undefined
Ejaaz: maybe they distilled a Western model, or maybe they had a breakthrough in continual (08:04):
undefined
Ejaaz: learning that we suggested earlier. (08:09):
undefined
Ejaaz: But the point is, this is a very serious competitor. And we saw. (08:10):
undefined
Ejaaz: A kind of exchange between Elon Musk and the founder of Jipu, (08:15):
undefined
Ejaaz: where about a month or two ago, (08:18):
undefined
Ejaaz: Elon Musk was responding into a thread as to when he believed that Chinese or (08:21):
undefined
Ejaaz: open source models in general will become mythos level, you know, (08:26):
undefined
Ejaaz: have that cyber security risk. (08:29):
undefined
Ejaaz: And he said, probably Q1 of next year, of 2027. (08:31):
undefined
Ejaaz: And the founder, Ji Tang, of Jipu replied says, it won't take that long. (08:35):
undefined
Ejaaz: And he goes on to say, it doesn't show it in this thread right here, (08:41):
undefined
Ejaaz: that it'll probably take two to three months. (08:43):
undefined
Ejaaz: And he was true to this word. We now have an AUX alpha mythos level model that (08:46):
undefined
Ejaaz: is from Jipu that is flash, that is smaller, that is cheaper, (08:50):
undefined
Ejaaz: that anyone and everyone can use, and it's gonna be open-weighted a few weeks from now. (08:54):
undefined
Josh: And that's probably how it was served. It's just a low cost model to serve. (08:57):
undefined
Josh: And it's really funny watching the public reaction to this and kind of, (09:02):
undefined
Josh: I don't want to say overreacting, but getting very excited and enthusiastic (09:06):
undefined
Josh: about this because a lot of the posts about this were like, oh, (09:08):
undefined
Josh: it does have continual learning and it is getting better every day. (09:12):
undefined
Josh: And suspicions as to like, why are there more tokens being available every single (09:16):
undefined
Josh: day? And we have this post from Jeffrey Emanuel, who's actually, (09:20):
undefined
Josh: I guess, on the Limitless Show. (09:22):
undefined
Josh: And he just said, just thinking through this free 0x alpha model or aux alpha (09:23):
undefined
Josh: model how could it possibly make sense to give out this many tokens for free (09:26):
undefined
Josh: maybe there's been a continuous learning breakthrough and the best way to pour (09:30):
undefined
Josh: more gas in the fire is to source as many coding related requests as possible (09:33):
undefined
Josh: i don't know if that's right i think the kind of where i'm landing here is that (09:37):
undefined
Josh: this is a flash model this is some sort of distilled model that is meant to (09:40):
undefined
Josh: serve inference very cheaply and what we're seeing instead of continual learning it's just (09:44):
undefined
Josh: New checkpoints that they're kind of releasing every day. And we're able to (09:49):
undefined
Josh: track the checkpoints as they come into play. And they're just kind of generally (09:52):
undefined
Josh: iterating over time. I'm not sure it's self-improving. (09:55):
undefined
Josh: We are going to find out more on that soon. (09:58):
undefined
Josh: The interesting phenomenon that I have really enjoyed seeing over the last week (10:00):
undefined
Josh: is the blind taste test that's been going on. (10:04):
undefined
Josh: This is like known kind of like as the Pepsi challenge, where you put like Pepsi (10:07):
undefined
Josh: and Coke against each other in a blind taste test and people don't know what they're getting. (10:11):
undefined
Josh: And one of the things I found really interesting is how excited people got about (10:14):
undefined
Josh: this model because of the mystery and the lore. (10:18):
undefined
Josh: And it really makes you question the value of true frontier level intelligence (10:21):
undefined
Josh: when people were doing seemingly incredible things with this open source free flash model. (10:27):
undefined
Josh: And when you're comparing the two next to each other, for some tasks, (10:32):
undefined
Josh: it's very clear that the frontier models are stronger. (10:36):
undefined
Josh: But for many other tasks, that's not apparently obvious. (10:38):
undefined
Josh: And if you are just sending a prompt into a text box, and it is this Ox Alpha (10:41):
undefined
Josh: model, or it is like a mid-tier ChatGPT or Claude model, the outputs are mostly the same. (10:46):
undefined
Josh: And people didn't realize and people don't really care where the tokens come (10:53):
undefined
Josh: from. So I find this to be an (10:56):
undefined
Josh: A true stealth model that is good oftentimes it's connected to a brand and there's (11:00):
undefined
Josh: very strong brand affinity so when you remove the brand affinity from the equation (11:03):
undefined
Josh: what happens you kind of get this phenomenon here where it turns out people just want (11:07):
undefined
Josh: pretty smart tokens and they don't actually care where it comes from it's interesting (11:11):
undefined
Josh: to see the the continual improvement or whatever's happening here where there's (11:14):
undefined
Josh: an example of a rocket ship taking off and it very clearly looks like day over day (11:18):
undefined
Josh: its abilities to render 3d graphics and understand physics and create this like (11:22):
undefined
Josh: real-world emulation are significantly better. (11:25):
undefined
Josh: So I'm really excited to learn more as to what's going on behind the scenes. (11:28):
undefined
Josh: Like, are these pre-prepared checkpoints, are they actually taking all the data (11:32):
undefined
Josh: they're collecting from these 100 trillion tokens and using it to continuous, (11:35):
undefined
Josh: to apply like RL on top of the model? Like, I don't know. These are things I'm (11:39):
undefined
Josh: very excited to find out over the next couple of days, though. (11:43):
undefined
Ejaaz: Yeah, I think in the future, no one's really going to care which model they use. (11:45):
undefined
Ejaaz: They just want to make sure that the output is what they expected. (11:50):
undefined
Ejaaz: They want to have the most effective output for the cheapest possible cost that (11:54):
undefined
Ejaaz: they can pay for that output. (11:57):
undefined
Ejaaz: And we're seeing like a lot of companies move to some of these Chinese open (11:59):
undefined
Ejaaz: source models or just other models, cheaper models from other smaller American (12:02):
undefined
Ejaaz: labs, just so that they are able to save on costs and create like a more fine-tuned (12:05):
undefined
Ejaaz: model for this specific bit of work. (12:10):
undefined
Ejaaz: Now, I want to spend a bit of time to talk about if this model has achieved (12:11):
undefined
Ejaaz: continual learning, why that's important and why that puts (12:17):
undefined
Ejaaz: China, if this is Jipu's actual model, which they confirmed in a very advantageous (12:21):
undefined
Ejaaz: position because it kind of shifts the Game of Thrones around a bit. (12:25):
undefined
Ejaaz: Well, right now, you look at Anthropic, you look at OpenAI, they have the world's leading models. (12:29):
undefined
Ejaaz: They have internal models that they haven't even released yet that are supposedly (12:34):
undefined
Ejaaz: even more intelligent than the ones that they have publicly available. (12:37):
undefined
Ejaaz: And they have all the compute in the world and they're spending so much money (12:40):
undefined
Ejaaz: trying to train the best model. (12:43):
undefined
Ejaaz: So if you were to reason within yourself, maybe no one else can have the capability (12:45):
undefined
Ejaaz: to catch up in China, although they've been able to put out very good open source (12:50):
undefined
Ejaaz: models hasn't been able to surpass the frontier. (12:54):
undefined
Ejaaz: Now, one way that you can kind of cheat that dynamic is if you have a continual learning model. (12:58):
undefined
Ejaaz: Now, the best way to think about what this means is think of a really intelligent (13:03):
undefined
Ejaaz: 15 year old or 18 year old when you teach it how to ride a bike on day one. (13:08):
undefined
Ejaaz: And then let's say you teach it how to make pancakes day 50s, (13:14):
undefined
Ejaaz: 50 days later, it'll still remember or know how to learn how to ride a bike or how to ride a bike. (13:18):
undefined
Ejaaz: Current models right now doesn't work like that. They lose sight of their memory. (13:23):
undefined
Ejaaz: It's something called catastrophic memory loss or something like that. (13:28):
undefined
Ejaaz: And they're unable to kind of remember some of the smart intelligence stuff (13:30):
undefined
Ejaaz: that they learned earlier on. (13:34):
undefined
Ejaaz: Continual learning solves for that. Now, you can imagine if a model is actually (13:36):
undefined
Ejaaz: able to learn like a real human is, like a really smart kid, (13:40):
undefined
Ejaaz: you could reason to say that that model will be more valuable to you. (13:44):
undefined
Ejaaz: Especially if you could run it locally at home on your local device on all your types of private data. (13:48):
undefined
Ejaaz: So I can see two types of individuals looking at this kind of a model and being (13:53):
undefined
Ejaaz: really incentivized to use it over a Claude or over an OpenAI chat GPT type model. (13:57):
undefined
Ejaaz: And that one is enterprises who don't want to give all their data away to major (14:03):
undefined
Ejaaz: American labs. They would want to use an open weights model that can continue to learn. (14:07):
undefined
Ejaaz: And the second is any kind of respective retail user who doesn't want to give (14:11):
undefined
Ejaaz: away any of their personal records to Anthropic and open AI. (14:15):
undefined
Ejaaz: Let's say it's medical records or finance information. (14:18):
undefined
Ejaaz: Let's say they want to have an AI agent that handles their finances. (14:21):
undefined
Ejaaz: You know, you may have different reasoning behind whether you want to use a (14:24):
undefined
Ejaaz: certain brand or a certain lab. (14:27):
undefined
Ejaaz: But if you could run it privately at home, maybe you might want to use something (14:28):
undefined
Ejaaz: like this. But continual learning, just assume you have this model that learns (14:31):
undefined
Ejaaz: 24-7, even when you're asleep, it could probably catch up with the frontier (14:35):
undefined
Ejaaz: models. And I think it's really interesting to see. (14:38):
undefined
Ejaaz: I don't know if this is that. I'm not convinced that it is. (14:40):
undefined
Ejaaz: But if China has made a breakthrough, that's going to put them in a very advantageous position. (14:42):
undefined
Josh: Yeah i also don't think it's that i find it hard to believe that this is what (14:46):
undefined
Josh: it looks like and that they've reached that point so far i mean that'd be an (14:50):
undefined
Josh: amazing breakthrough in the case they do (14:53):
undefined
Josh: you're going to need to be prepared. You're going to need to be safe. (14:56):
undefined
Josh: And Ledger, our sponsor, is here to keep you safe. Because if you are building (14:58):
undefined
Josh: with AI agents and you're giving prompts to free open source models that are (15:02):
undefined
Josh: probably owned by the Chinese, chances are you're going to want some safety and some protection. (15:05):
undefined
Josh: And the Ledger agent stack protects this. It gets this job done. (15:09):
undefined
Josh: It offers a series of tools that allow you to work with agents in a three-step (15:12):
undefined
Josh: process where the agent proposes, the human approves, and then the Ledger signer enforces. (15:16):
undefined
Josh: So if you're building security and you want to securely use these agents ledger (15:21):
undefined
Josh: is for you it's available on cloud code on codex on cursor pretty much anywhere (15:26):
undefined
Josh: in which you engage with llms and is open source and available today so you (15:30):
undefined
Josh: can find a link in the description down (15:33):
undefined
Josh: in our show thank you ledger for sponsoring this episode and then i guess we (15:35):
undefined
Josh: kind of have to talk about that like is it really a free case because like is (15:39):
undefined
Josh: free ever really free and i guess there's two possible answers to this question (15:44):
undefined
Josh: one is yes it's just being heavily subsidized (15:47):
undefined
Josh: perhaps by the chinese government as a vampire attack to pull some demand from (15:49):
undefined
Josh: the frontier labs into these chinese models the other is perhaps they're just using the data (15:53):
undefined
Josh: one of the big problems that we have been seeing people complain about is data (15:57):
undefined
Josh: retention and zero data retention policies because people don't want to feel like they are (16:00):
undefined
Josh: Having their data harvested and used to train other models. And when you think (16:06):
undefined
Josh: about giving away these free models, China for a fact is collecting all of the (16:09):
undefined
Josh: inputs from the context window. And what's been happening is a lot of people (16:14):
undefined
Josh: are using this on perhaps proprietary code bases. (16:17):
undefined
Josh: People have been testing this model in their own private projects. (16:20):
undefined
Josh: They've been using it for their own personal work. (16:23):
undefined
Josh: And a lot of data gets derived out of that. I mean, it's a million token context (16:26):
undefined
Josh: window in this one chat box and you just paste it all in and you let it go to town. (16:30):
undefined
Josh: So I'm sure the amount of data that's collected from offering all of these tokens (16:33):
undefined
Josh: just through collecting all the inputs has been tremendous. (16:38):
undefined
Josh: And I'm sure that gives them a unique advantage in a way that some frontier (16:40):
undefined
Josh: labs might not, because oftentimes they are criticized for their retention policies. (16:44):
undefined
Josh: And people really want to make sure that there is that zero data retention. (16:49):
undefined
Josh: China doesn't face the scrutiny, in particular, doing it in stealth kind of (16:51):
undefined
Josh: shields that for at least the first week while people have used it. (16:55):
undefined
Josh: So they were just kind of freely and liberally using this thing, (16:58):
undefined
Josh: probably aware that they were having their prompts used for future training, (17:00):
undefined
Josh: but not really caring too much. (17:04):
undefined
Josh: So I thought that was also interesting. The other thing is that like the status (17:06):
undefined
Josh: of Jeep as a company still isn't very strong. (17:09):
undefined
Josh: They had, I think, like $100 million in revenue last year, and they had a net (17:12):
undefined
Josh: loss of like seven times that or something like that. It's currently priced at 750 times revenue. (17:16):
undefined
Josh: The company itself clearly can't sustain this type of... (17:22):
undefined
Josh: Funding and this free giveaway forever. And I'm curious to know what the type (17:26):
undefined
Josh: of business model eventually, if any, there is going to be in order to keep this thing alive. (17:30):
undefined
Josh: Like, is it just going to be a government subsidy until they figure it out? (17:34):
undefined
Ejaaz: The business model is its state back, dude. Yeah. (17:38):
undefined
Ejaaz: All these Chinese labs are backed by the state. So that's how they get their (17:41):
undefined
Ejaaz: infinite amounts of capital to be able to kind of fund all these things. (17:44):
undefined
Ejaaz: And this is a strategy that China has been pursuing for three plus years now. (17:47):
undefined
Ejaaz: They want to catch up with the frontier Western models. they distill attack (17:52):
undefined
Ejaaz: a bunch of these model labs so that they can train similarly capable models. (17:56):
undefined
Ejaaz: And then they flood the market with cheap intelligence, which means that Anthropic (17:59):
undefined
Ejaaz: can't charge the highest price that they can for the high quality model that (18:03):
undefined
Ejaaz: they have. OpenAI can't do the same. (18:06):
undefined
Ejaaz: So this drives model costs down. This is what kind of Meta and Mark Zuckerberg (18:08):
undefined
Ejaaz: is also trying to do, but failing to do as effectively as the Chinese labs. (18:12):
undefined
Ejaaz: They're just very good at doing it. The second thing I'll say is, (18:16):
undefined
Ejaaz: at least according to Open Rata from their official statement. (18:19):
undefined
Ejaaz: They said that this stealth model provider, which we now know to be Jipu, (18:22):
undefined
Ejaaz: is retaining prompts and responses, but explicitly state that they're not using (18:26):
undefined
Ejaaz: it for training. Now, this isn't verifiable in any kind of way. (18:31):
undefined
Ejaaz: Like, okay, China, you've never done anything nefarious before, I'm sure. (18:36):
undefined
Josh: You're really going to give this away totally for free? Like, (18:39):
undefined
Josh: no way. Coming from the people that stole all the items, I can go and generate (18:41):
undefined
Josh: a full episode of Spongebob on those models. (18:45):
undefined
Ejaaz: Exactly. Exactly. Well, this is another unique thing, right? (18:47):
undefined
Ejaaz: Culturally, in China, and this applies for every vector of tech that they've (18:50):
undefined
Ejaaz: built in the past, electric cars, mobile phones, et cetera. (18:55):
undefined
Ejaaz: They don't really see copyright as a thing or IP infringement as a thing. (18:58):
undefined
Ejaaz: They see, okay, if you have a good product, I can copy that. (19:02):
undefined
Ejaaz: And Chinese in particular are very, very good at copying effectively. (19:05):
undefined
Ejaaz: There was this great interview I watched with, who was it? It was Travis Kalanick, (19:10):
undefined
Ejaaz: on David Semra's podcast. I don't know if you caught that, right? (19:14):
undefined
Ejaaz: Absolute legend, right? Twice. And he talks about his story where, (19:16):
undefined
Ejaaz: you know, how he brought Uber over to China and he made a success there, (19:21):
undefined
Ejaaz: but he said it was very tough. But there was also some really unique cultural (19:25):
undefined
Ejaaz: things that he needed to understand. (19:28):
undefined
Ejaaz: And he realized that in America, or in the West at least, the innovation level (19:30):
undefined
Ejaaz: is super high. Coming up with net new zero-to-one ideas, super high. (19:34):
undefined
Ejaaz: In China, less so, but they are masters of copying. And he originally looked (19:39):
undefined
Ejaaz: down at copying as a really menial thing. (19:43):
undefined
Ejaaz: But he said that the way that they do it is masterful and artful in itself. (19:47):
undefined
Ejaaz: And this, I feel like, is what some of these Chinese labs are really good at doing. (19:52):
undefined
Ejaaz: It doesn't matter if they're earning $100 million. Their IPOs are popping off (19:56):
undefined
Ejaaz: anyway. They're never going to compare to the Western IPO market at all. (19:59):
undefined
Ejaaz: But the point is, they want to flood the market and kind of like keep America (20:02):
undefined
Ejaaz: at a cost level where maybe they're spending too much money than they actually (20:06):
undefined
Ejaaz: need to until they're able to figure out some kind of a breakthrough, (20:09):
undefined
Ejaaz: which might be continual money. Who knows? (20:12):
undefined
Josh: Maybe. We'll see. It's an exciting story and we're going to find out more. (20:14):
undefined
Josh: So this will be covered and followed up in the roundup tomorrow. (20:18):
undefined
Josh: Stay tuned for that. There's also a reference. It's kind of cool. (20:22):
undefined
Josh: The main character of the story is Open Router. (20:25):
undefined
Josh: And we just covered an entire Open Router episode just last week. (20:27):
undefined
Josh: So everyone who is unfamiliar with Open Router and why it is so valuable, (20:32):
undefined
Josh: please go and check out that episode. (20:35):
undefined
Josh: And that's mostly the state of the mystery as of now. Ox Alpha has been solved, (20:37):
undefined
Josh: but there is still a lot of behind the scenes that is left to uncover. (20:43):
undefined
Josh: We're hoping to get that information shortly. when we do we will share it with (20:46):
undefined
Josh: you right here on the roundup hopefully this week so stay tuned for that one (20:48):
undefined
Josh: um this has been kind of a crazy week i mean it's (20:52):
undefined
Josh: China's always throwing like, they're throwing wrenches in everyone's plans. (20:56):
undefined
Josh: First it was Deep Seek. Now they're doing this like kind of really fun stuff. (21:00):
undefined
Ejaaz: Did you see Quen as well? (21:03):
undefined
Josh: Yeah, we got a new Quen model. We're going to talk about that tomorrow. (21:05):
undefined
Ejaaz: Dude, they're just like spaying out these models. I know. (21:08):
undefined
Ejaaz: And you know, the craziest part about all of this is I'm seeing rumors from (21:11):
undefined
Ejaaz: trusted sources that a bunch of American frontier labs are also about to like drop a few models. (21:16):
undefined
Ejaaz: So like, I'm just saying here, like, you know, we're getting like the best from (21:22):
undefined
Ejaaz: the Chinese, We're getting the best from the West and there's just too many models to use right now. (21:25):
undefined
Ejaaz: But if you're listening to this and you're curious about this Ox Alpha model (21:29):
undefined
Ejaaz: or if you don't believe anything that we're saying, if you don't think it's (21:32):
undefined
Ejaaz: good enough, go out and try it right now. You can use. (21:34):
undefined
Josh: It for free (21:37):
undefined
Ejaaz: On OpenRouter. Go check it out. There is no promo code. Just sign up, (21:38):
undefined
Ejaaz: make an account and you can kind of like inference it and use it yourself. (21:42):
undefined
Josh: And I'm curious, send us, (21:45):
undefined
Ejaaz: Exactly. Like use the free tokens, please. Use the Chinese state funded money (21:47):
undefined
Ejaaz: budget to kind of do your AI tasks. (21:51):
undefined
Josh: Just don't give it your proprietary information. (21:54):
undefined
Ejaaz: Do not, definitely don't do that because they're definitely using it to train (21:56):
undefined
Ejaaz: the next model. But like, let us know in the comments whether you actually like (21:59):
undefined
Ejaaz: the model, whether it actually is good at the toss that you give it. (22:02):
undefined
Ejaaz: And if it magically improves the next day because it's continual learning. We don't know. (22:05):
undefined
Ejaaz: Like, let us know in the comments, DM us, we're available on all socials. (22:11):
undefined
Ejaaz: Yeah, anything else, Josh? (22:14):
undefined
Josh: Yeah, no, just to, again, share this with your friends if they enjoyed it. (22:15):
undefined
Josh: If you have something to say, please leave a comment. We try to read all of them. (22:18):
undefined
Josh: If you enjoyed on your favorite podcast player you can leave us a five-star (22:22):
undefined
Josh: review there if you're listening to this (22:25):
undefined
Josh: you should be watching it's very fun visually you can watch this on youtube (22:27):
undefined
Josh: on spotify and yeah as always thank you so much for watching feel free to go (22:30):
undefined
Josh: get caught up on the other episodes and tomorrow we got a banger roundup like (22:35):
undefined
Josh: it's just gonna there's so much stuff to talk about it (22:38):
undefined
Ejaaz: Is our favorite episode of the week. (22:40):
undefined
Josh: We've been in like a lull lately where there had i mean it's been exciting but (22:42):
undefined
Josh: it hasn't been like oh my god this is crazy there's new frontiers forming and (22:46):
undefined
Josh: like hopefully this is the first of many instances in which we're going to rev (22:50):
undefined
Josh: back up that engine and get (22:53):
undefined
Josh: a whole bunch of new models so i'm very excited stay tuned for all of that where (22:55):
undefined
Josh: we'll be covering it and yeah we'll see you guys in the next episode (22:57):
undefined
undefined
Josh: There's a secret model codenamed 0xAlpha that's been live since August 20th, (00:04):
undefined
Josh: the last six days, and it is offering the seemingly unbelievable things to the public. (00:08):
undefined
Josh: They're offering 100 trillion tokens of free usage to anyone who wants to go (00:13):
undefined
Josh: and get it. They're offering a million token context window, (00:17):
undefined
Josh: and the benchmarks of this thing are pretty unbelievable. (00:20):
undefined
Josh: There's also this weird thing going on in the background where seemingly every (00:22):
undefined
Josh: single day the outputs of the model are getting better there were tests on day (00:25):
undefined
Josh: one that were far inferior to the tests that were currently on day six as of (00:29):
undefined
Josh: this morning we think we have the answer to who this person is but before we talk about that (00:33):
undefined
Josh: we have to discuss what is 0x alpha you guys this has been the mystery of the (00:37):
undefined
Josh: week it seems like frontier level intelligence it's offering all of these unbelievable (00:41):
undefined
Josh: things like 100 trillion tokens for free (00:45):
undefined
Josh: do you know how much that would cost if you were to use like fable or gbt's 5.6 (00:49):
undefined
Ejaaz: Salt like i think it's like 15 to to like 20 million bucks a day. (00:53):
undefined
Josh: It's a lot of money (00:57):
undefined
Ejaaz: A day okay so just with that figure alone who who on earth is giving away 20 (00:58):
undefined
Ejaaz: to 30 million dollars on inference per day in this economy right it has to be (01:03):
undefined
Ejaaz: a big frontier lab the second clue is josh it's not 0x alpha (01:07):
undefined
Ejaaz: it's ox alpha which seems kind of weird right Like who's talking about the animal ox? (01:12):
undefined
Ejaaz: Until you realize that it's probably part of like the Chinese kind of folklore (01:19):
undefined
Ejaaz: and Zodiac side of things, which again, might be a little hint as to where we're (01:23):
undefined
Ejaaz: going. But anyway, let's rewind six days ago very quickly. (01:27):
undefined
Ejaaz: Open Router, which we've covered on the show before, it's a platform that kind (01:30):
undefined
Ejaaz: of like hosts a bunch of different models, allows you to pick and choose, (01:33):
undefined
Ejaaz: revealed a stealth model. Now a stealth launch is where they launch the model. (01:36):
undefined
Ejaaz: You go to their website, you can access and use the model for free. (01:41):
undefined
Ejaaz: But they don't tell you the name of the model. Now, they've done this with previous (01:45):
undefined
Ejaaz: launches through Google Gemini's model. They've done it through Anthropic. (01:48):
undefined
Ejaaz: They've done it through OpenAI as well. (01:51):
undefined
Ejaaz: And they haven't had a stealth model launch in a while, but now this is the (01:53):
undefined
Ejaaz: first one and it was in high demand. (01:57):
undefined
Ejaaz: So as you mentioned earlier, 100 trillion tokens, which means that you can effectively (02:00):
undefined
Ejaaz: go there and do like all your sorts of coding works, very heavy AI usage tasks. (02:04):
undefined
Ejaaz: And you can kind of use that in a day. (02:08):
undefined
Ejaaz: Again, 20 to $30 million worth of inference costs. (02:10):
undefined
Ejaaz: It has a massive 1 million context window, which is very competitive with the (02:14):
undefined
Ejaaz: top models that we see from Fable 5, as well as GPT-5.6 Sol, and, (02:18):
undefined
Ejaaz: and this is the most important part, it's a uni model, which means that you (02:24):
undefined
Ejaaz: can use not just text, but images, models. It's really good at rendering all (02:27):
undefined
Ejaaz: these different types of medium. (02:31):
undefined
Ejaaz: Now, the final point, which I think is the most exciting one that you referred (02:33):
undefined
Ejaaz: to earlier, Josh, is this concept of continual learning. (02:36):
undefined
Ejaaz: So people, as they started to use this model, realized a very curious phenomenon. (02:40):
undefined
Ejaaz: They realized that as they were using this model, let's say on day one, (02:45):
undefined
Ejaaz: day two and day three, the model got exponentially better at doing the very (02:49):
undefined
Ejaaz: same tasks that they fed it in day one. (02:53):
undefined
Ejaaz: And so people started to be suspicious that this is a model that doesn't just (02:55):
undefined
Ejaaz: kind of take your input and spit back out an answer, which is what a lot of (03:00):
undefined
Ejaaz: models do right now, but it can continually self-learn and improve 24-7 to give you a better response. (03:03):
undefined
Ejaaz: So the question on everyone's mind is, who created this model at all? (03:10):
undefined
Ejaaz: And when you look at the kind of usage or when you look at the kind of uptake on OpenRouter itself. (03:13):
undefined
Ejaaz: Although no one knew, it became the most viral launch on OpenRouter. (03:19):
undefined
Ejaaz: It became the new number one model and dethroned DeepSeek's Flash, (03:23):
undefined
Ejaaz: which launched about a couple weeks ago. And DeepSeek has been kind of like (03:27):
undefined
Ejaaz: the name brand on OpenRouter for the longest time ever. So just a really exciting model to see. (03:29):
undefined
Josh: Yeah, it was not only the most popular model on OpenRouter immediately upon (03:34):
undefined
Josh: launch, but by a factor of two. (03:38):
undefined
Josh: And it more than doubled DeepSeek's use. Like, This is hugely popular and very (03:40):
undefined
Josh: interesting because of how well it performs. (03:45):
undefined
Josh: The benchmarks from it, the Deep SWE subset benchmark, it had an 80% pass rate, (03:47):
undefined
Josh: which for those who aren't familiar, Fable 5 was at 65% and GLM 5.3 was at 62%. (03:53):
undefined
Ejaaz: Hold on, hold on. So you're saying that it's crushing the METOS level models (04:00):
undefined
Ejaaz: and the GPT-5.6 models at coding? (04:05):
undefined
Josh: In one particular subset of coding benchmarks. (04:07):
undefined
Josh: I don't want to i definitely don't want to say it's better than the frontier (04:12):
undefined
Josh: models because upon further investigation we will come to find out it's not (04:14):
undefined
Josh: and it's pretty far from it but it's an incredible model for what it is and (04:19):
undefined
Josh: the reason i was comparing it to glm 5.3 is because (04:22):
undefined
Josh: There's this thing when you're producing models called the tokenizers, (04:25):
undefined
Josh: how tokens get generated. And it was just so happened to be identical to GLM. (04:27):
undefined
Josh: So people were like, huh, this is interesting. Maybe we can go further down (04:31):
undefined
Josh: the rabbit hole and see if there's more here. (04:34):
undefined
Josh: So you can test oftentimes whether a model is Chinese or not based on asking (04:37):
undefined
Josh: it a series of questions that are not really allowed to be answered in China. (04:42):
undefined
Josh: One of which is asking whether Taiwan is part of China. (04:46):
undefined
Josh: And what's funny is if you're using like an American frontier lab, (04:50):
undefined
Josh: they'll very clearly give you the answer to this question but chinese models do not and (04:53):
undefined
Josh: what we realized is that the output of the answer to this question was pretty (04:58):
undefined
Josh: much identical to the glm model and so like okay it's got a similar tokenizer (05:02):
undefined
Josh: it's giving similar outputs but we still don't really know (05:08):
undefined
Josh: Up until this morning, in which we discovered that GLM is actually the new version of GPOO. (05:11):
undefined
Josh: It is, or sorry, that Ox Alpha is the new version of GPOO. (05:16):
undefined
Josh: And it seems like this is going to be a new GLM model that we have. (05:20):
undefined
Josh: And that, when I heard it, it actually landed me even more suspicious because (05:24):
undefined
Josh: over the course of this last week on X and all of his social media, (05:29):
undefined
Josh: everyone has been kind of vague posting about who this is, why they're doing (05:33):
undefined
Josh: it, why they're offering so many tokens and so much compute. (05:36):
undefined
Josh: And then the Google team started chiming in, which I thought was really bizarre. (05:40):
undefined
Josh: A lot of developers on the Google DeepMind team who work on Gemini, (05:43):
undefined
Josh: they started vague posting about this model. And as of this morning, (05:46):
undefined
Josh: Bloomberg released a post, and it seems like it is everything but confirmed. (05:49):
undefined
Josh: The model actually is from Z.ai and OxAlpha is now known. (05:53):
undefined
Josh: And I think that was like a really interesting way of coming to this conclusion. (05:58):
undefined
Josh: And now that we know, I just have more questions. How on earth are they serving (06:01):
undefined
Josh: all this inference to all these people for free. Like the cost must be tremendous. (06:05):
undefined
Ejaaz: I mean, let's see how this model shapes up against all the other models. (06:10):
undefined
Ejaaz: So we have a chart over here, which shows kind of like the average tokens output (06:13):
undefined
Ejaaz: per task on the DeepSuite benchmark that you mentioned. (06:17):
undefined
Ejaaz: DeepSuite, for those of you who don't know, is a good test of a model's capability (06:19):
undefined
Ejaaz: to actually code really well. And code is kind of like the premium benchmark (06:25):
undefined
Ejaaz: that a lot of companies and AI labs kind of like benchmark their model assets against. (06:29):
undefined
Ejaaz: And so OxAlpha is actually quite high up there. It's up there in terms of reduced (06:34):
undefined
Ejaaz: token output, but still maintaining a very high intelligence score. (06:40):
undefined
Ejaaz: Now, as you mentioned earlier on, that's not actually quite what was revealed (06:43):
undefined
Ejaaz: when people started playing around with this model. (06:47):
undefined
Ejaaz: It ended up achieving not really 80% that you mentioned earlier, but around 63%. (06:49):
undefined
Ejaaz: And that's under like real usage. So there's a suggestion there that the Chinese (06:53):
undefined
Ejaaz: lab GPU kind of like maybe bench backs this. (06:57):
undefined
Ejaaz: To just talk about the model and why it's so impressive, just in its own merit, (06:59):
undefined
Ejaaz: there's a few things to look at. (07:04):
undefined
Ejaaz: Number one, the rumor has it that this model isn't a foundation model. (07:05):
undefined
Ejaaz: This is a flash model. So what that means is it's a smaller derivative of a (07:11):
undefined
Ejaaz: much larger model that Jeepoo is training or has already trained. (07:15):
undefined
Ejaaz: So what does this mean? Let's take Mythos or let's take JeepT5.6. (07:20):
undefined
Ejaaz: These are rumored to be around 6 to 10 trillion parameter models. (07:25):
undefined
Ejaaz: Now, when you look at Jibu's flash model, if that's true, you're only looking (07:28):
undefined
Ejaaz: at around like two to three trillion parameters in size, which, (07:33):
undefined
Ejaaz: you know, is a decent size, but it's still very small compared to these bigger models. (07:36):
undefined
Ejaaz: And then the fact that this model is not only likely going to be cheaper in (07:41):
undefined
Ejaaz: its usage, let's say maybe 30 to 50 cents per usage. (07:45):
undefined
Ejaaz: So like they gave away like 15 to $30 million per day of 100 trillion tokens, (07:47):
undefined
Ejaaz: it is able to work much faster, and they're going to open weight the entire model. (07:52):
undefined
Ejaaz: So, I mean, to answer your question, I don't really have an answer, (07:56):
undefined
Ejaaz: Josh, as to how they've been able to kind of create this, aside from the usual, (08:00):
undefined
Ejaaz: maybe they distilled a Western model, or maybe they had a breakthrough in continual (08:04):
undefined
Ejaaz: learning that we suggested earlier. (08:09):
undefined
Ejaaz: But the point is, this is a very serious competitor. And we saw. (08:10):
undefined
Ejaaz: A kind of exchange between Elon Musk and the founder of Jipu, (08:15):
undefined
Ejaaz: where about a month or two ago, (08:18):
undefined
Ejaaz: Elon Musk was responding into a thread as to when he believed that Chinese or (08:21):
undefined
Ejaaz: open source models in general will become mythos level, you know, (08:26):
undefined
Ejaaz: have that cyber security risk. (08:29):
undefined
Ejaaz: And he said, probably Q1 of next year, of 2027. (08:31):
undefined
Ejaaz: And the founder, Ji Tang, of Jipu replied says, it won't take that long. (08:35):
undefined
Ejaaz: And he goes on to say, it doesn't show it in this thread right here, (08:41):
undefined
Ejaaz: that it'll probably take two to three months. (08:43):
undefined
Ejaaz: And he was true to this word. We now have an AUX alpha mythos level model that (08:46):
undefined
Ejaaz: is from Jipu that is flash, that is smaller, that is cheaper, (08:50):
undefined
Ejaaz: that anyone and everyone can use, and it's gonna be open-weighted a few weeks from now. (08:54):
undefined
Josh: And that's probably how it was served. It's just a low cost model to serve. (08:57):
undefined
Josh: And it's really funny watching the public reaction to this and kind of, (09:02):
undefined
Josh: I don't want to say overreacting, but getting very excited and enthusiastic (09:06):
undefined
Josh: about this because a lot of the posts about this were like, oh, (09:08):
undefined
Josh: it does have continual learning and it is getting better every day. (09:12):
undefined
Josh: And suspicions as to like, why are there more tokens being available every single (09:16):
undefined
Josh: day? And we have this post from Jeffrey Emanuel, who's actually, (09:20):
undefined
Josh: I guess, on the Limitless Show. (09:22):
undefined
Josh: And he just said, just thinking through this free 0x alpha model or aux alpha (09:23):
undefined
Josh: model how could it possibly make sense to give out this many tokens for free (09:26):
undefined
Josh: maybe there's been a continuous learning breakthrough and the best way to pour (09:30):
undefined
Josh: more gas in the fire is to source as many coding related requests as possible (09:33):
undefined
Josh: i don't know if that's right i think the kind of where i'm landing here is that (09:37):
undefined
Josh: this is a flash model this is some sort of distilled model that is meant to (09:40):
undefined
Josh: serve inference very cheaply and what we're seeing instead of continual learning it's just (09:44):
undefined
Josh: New checkpoints that they're kind of releasing every day. And we're able to (09:49):
undefined
Josh: track the checkpoints as they come into play. And they're just kind of generally (09:52):
undefined
Josh: iterating over time. I'm not sure it's self-improving. (09:55):
undefined
Josh: We are going to find out more on that soon. (09:58):
undefined
Josh: The interesting phenomenon that I have really enjoyed seeing over the last week (10:00):
undefined
Josh: is the blind taste test that's been going on. (10:04):
undefined
Josh: This is like known kind of like as the Pepsi challenge, where you put like Pepsi (10:07):
undefined
Josh: and Coke against each other in a blind taste test and people don't know what they're getting. (10:11):
undefined
Josh: And one of the things I found really interesting is how excited people got about (10:14):
undefined
Josh: this model because of the mystery and the lore. (10:18):
undefined
Josh: And it really makes you question the value of true frontier level intelligence (10:21):
undefined
Josh: when people were doing seemingly incredible things with this open source free flash model. (10:27):
undefined
Josh: And when you're comparing the two next to each other, for some tasks, (10:32):
undefined
Josh: it's very clear that the frontier models are stronger. (10:36):
undefined
Josh: But for many other tasks, that's not apparently obvious. (10:38):
undefined
Josh: And if you are just sending a prompt into a text box, and it is this Ox Alpha (10:41):
undefined
Josh: model, or it is like a mid-tier ChatGPT or Claude model, the outputs are mostly the same. (10:46):
undefined
Josh: And people didn't realize and people don't really care where the tokens come (10:53):
undefined
Josh: from. So I find this to be an (10:56):
undefined
Josh: A true stealth model that is good oftentimes it's connected to a brand and there's (11:00):
undefined
Josh: very strong brand affinity so when you remove the brand affinity from the equation (11:03):
undefined
Josh: what happens you kind of get this phenomenon here where it turns out people just want (11:07):
undefined
Josh: pretty smart tokens and they don't actually care where it comes from it's interesting (11:11):
undefined
Josh: to see the the continual improvement or whatever's happening here where there's (11:14):
undefined
Josh: an example of a rocket ship taking off and it very clearly looks like day over day (11:18):
undefined
Josh: its abilities to render 3d graphics and understand physics and create this like (11:22):
undefined
Josh: real-world emulation are significantly better. (11:25):
undefined
Josh: So I'm really excited to learn more as to what's going on behind the scenes. (11:28):
undefined
Josh: Like, are these pre-prepared checkpoints, are they actually taking all the data (11:32):
undefined
Josh: they're collecting from these 100 trillion tokens and using it to continuous, (11:35):
undefined
Josh: to apply like RL on top of the model? Like, I don't know. These are things I'm (11:39):
undefined
Josh: very excited to find out over the next couple of days, though. (11:43):
undefined
Ejaaz: Yeah, I think in the future, no one's really going to care which model they use. (11:45):
undefined
Ejaaz: They just want to make sure that the output is what they expected. (11:50):
undefined
Ejaaz: They want to have the most effective output for the cheapest possible cost that (11:54):
undefined
Ejaaz: they can pay for that output. (11:57):
undefined
Ejaaz: And we're seeing like a lot of companies move to some of these Chinese open (11:59):
undefined
Ejaaz: source models or just other models, cheaper models from other smaller American (12:02):
undefined
Ejaaz: labs, just so that they are able to save on costs and create like a more fine-tuned (12:05):
undefined
Ejaaz: model for this specific bit of work. (12:10):
undefined
Ejaaz: Now, I want to spend a bit of time to talk about if this model has achieved (12:11):
undefined
Ejaaz: continual learning, why that's important and why that puts (12:17):
undefined
Ejaaz: China, if this is Jipu's actual model, which they confirmed in a very advantageous (12:21):
undefined
Ejaaz: position because it kind of shifts the Game of Thrones around a bit. (12:25):
undefined
Ejaaz: Well, right now, you look at Anthropic, you look at OpenAI, they have the world's leading models. (12:29):
undefined
Ejaaz: They have internal models that they haven't even released yet that are supposedly (12:34):
undefined
Ejaaz: even more intelligent than the ones that they have publicly available. (12:37):
undefined
Ejaaz: And they have all the compute in the world and they're spending so much money (12:40):
undefined
Ejaaz: trying to train the best model. (12:43):
undefined
Ejaaz: So if you were to reason within yourself, maybe no one else can have the capability (12:45):
undefined
Ejaaz: to catch up in China, although they've been able to put out very good open source (12:50):
undefined
Ejaaz: models hasn't been able to surpass the frontier. (12:54):
undefined
Ejaaz: Now, one way that you can kind of cheat that dynamic is if you have a continual learning model. (12:58):
undefined
Ejaaz: Now, the best way to think about what this means is think of a really intelligent (13:03):
undefined
Ejaaz: 15 year old or 18 year old when you teach it how to ride a bike on day one. (13:08):
undefined
Ejaaz: And then let's say you teach it how to make pancakes day 50s, (13:14):
undefined
Ejaaz: 50 days later, it'll still remember or know how to learn how to ride a bike or how to ride a bike. (13:18):
undefined
Ejaaz: Current models right now doesn't work like that. They lose sight of their memory. (13:23):
undefined
Ejaaz: It's something called catastrophic memory loss or something like that. (13:28):
undefined
Ejaaz: And they're unable to kind of remember some of the smart intelligence stuff (13:30):
undefined
Ejaaz: that they learned earlier on. (13:34):
undefined
Ejaaz: Continual learning solves for that. Now, you can imagine if a model is actually (13:36):
undefined
Ejaaz: able to learn like a real human is, like a really smart kid, (13:40):
undefined
Ejaaz: you could reason to say that that model will be more valuable to you. (13:44):
undefined
Ejaaz: Especially if you could run it locally at home on your local device on all your types of private data. (13:48):
undefined
Ejaaz: So I can see two types of individuals looking at this kind of a model and being (13:53):
undefined
Ejaaz: really incentivized to use it over a Claude or over an OpenAI chat GPT type model. (13:57):
undefined
Ejaaz: And that one is enterprises who don't want to give all their data away to major (14:03):
undefined
Ejaaz: American labs. They would want to use an open weights model that can continue to learn. (14:07):
undefined
Ejaaz: And the second is any kind of respective retail user who doesn't want to give (14:11):
undefined
Ejaaz: away any of their personal records to Anthropic and open AI. (14:15):
undefined
Ejaaz: Let's say it's medical records or finance information. (14:18):
undefined
Ejaaz: Let's say they want to have an AI agent that handles their finances. (14:21):
undefined
Ejaaz: You know, you may have different reasoning behind whether you want to use a (14:24):
undefined
Ejaaz: certain brand or a certain lab. (14:27):
undefined
Ejaaz: But if you could run it privately at home, maybe you might want to use something (14:28):
undefined
Ejaaz: like this. But continual learning, just assume you have this model that learns (14:31):
undefined
Ejaaz: 24-7, even when you're asleep, it could probably catch up with the frontier (14:35):
undefined
Ejaaz: models. And I think it's really interesting to see. (14:38):
undefined
Ejaaz: I don't know if this is that. I'm not convinced that it is. (14:40):
undefined
Ejaaz: But if China has made a breakthrough, that's going to put them in a very advantageous position. (14:42):
undefined
Josh: Yeah i also don't think it's that i find it hard to believe that this is what (14:46):
undefined
Josh: it looks like and that they've reached that point so far i mean that'd be an (14:50):
undefined
Josh: amazing breakthrough in the case they do (14:53):
undefined
Josh: you're going to need to be prepared. You're going to need to be safe. (14:56):
undefined
Josh: And Ledger, our sponsor, is here to keep you safe. Because if you are building (14:58):
undefined
Josh: with AI agents and you're giving prompts to free open source models that are (15:02):
undefined
Josh: probably owned by the Chinese, chances are you're going to want some safety and some protection. (15:05):
undefined
Josh: And the Ledger agent stack protects this. It gets this job done. (15:09):
undefined
Josh: It offers a series of tools that allow you to work with agents in a three-step (15:12):
undefined
Josh: process where the agent proposes, the human approves, and then the Ledger signer enforces. (15:16):
undefined
Josh: So if you're building security and you want to securely use these agents ledger (15:21):
undefined
Josh: is for you it's available on cloud code on codex on cursor pretty much anywhere (15:26):
undefined
Josh: in which you engage with llms and is open source and available today so you (15:30):
undefined
Josh: can find a link in the description down (15:33):
undefined
Josh: in our show thank you ledger for sponsoring this episode and then i guess we (15:35):
undefined
Josh: kind of have to talk about that like is it really a free case because like is (15:39):
undefined
Josh: free ever really free and i guess there's two possible answers to this question (15:44):
undefined
Josh: one is yes it's just being heavily subsidized (15:47):
undefined
Josh: perhaps by the chinese government as a vampire attack to pull some demand from (15:49):
undefined
Josh: the frontier labs into these chinese models the other is perhaps they're just using the data (15:53):
undefined
Josh: one of the big problems that we have been seeing people complain about is data (15:57):
undefined
Josh: retention and zero data retention policies because people don't want to feel like they are (16:00):
undefined
Josh: Having their data harvested and used to train other models. And when you think (16:06):
undefined
Josh: about giving away these free models, China for a fact is collecting all of the (16:09):
undefined
Josh: inputs from the context window. And what's been happening is a lot of people (16:14):
undefined
Josh: are using this on perhaps proprietary code bases. (16:17):
undefined
Josh: People have been testing this model in their own private projects. (16:20):
undefined
Josh: They've been using it for their own personal work. (16:23):
undefined
Josh: And a lot of data gets derived out of that. I mean, it's a million token context (16:26):
undefined
Josh: window in this one chat box and you just paste it all in and you let it go to town. (16:30):
undefined
Josh: So I'm sure the amount of data that's collected from offering all of these tokens (16:33):
undefined
Josh: just through collecting all the inputs has been tremendous. (16:38):
undefined
Josh: And I'm sure that gives them a unique advantage in a way that some frontier (16:40):
undefined
Josh: labs might not, because oftentimes they are criticized for their retention policies. (16:44):
undefined
Josh: And people really want to make sure that there is that zero data retention. (16:49):
undefined
Josh: China doesn't face the scrutiny, in particular, doing it in stealth kind of (16:51):
undefined
Josh: shields that for at least the first week while people have used it. (16:55):
undefined
Josh: So they were just kind of freely and liberally using this thing, (16:58):
undefined
Josh: probably aware that they were having their prompts used for future training, (17:00):
undefined
Josh: but not really caring too much. (17:04):
undefined
Josh: So I thought that was also interesting. The other thing is that like the status (17:06):
undefined
Josh: of Jeep as a company still isn't very strong. (17:09):
undefined
Josh: They had, I think, like $100 million in revenue last year, and they had a net (17:12):
undefined
Josh: loss of like seven times that or something like that. It's currently priced at 750 times revenue. (17:16):
undefined
Josh: The company itself clearly can't sustain this type of... (17:22):
undefined
Josh: Funding and this free giveaway forever. And I'm curious to know what the type (17:26):
undefined
Josh: of business model eventually, if any, there is going to be in order to keep this thing alive. (17:30):
undefined
Josh: Like, is it just going to be a government subsidy until they figure it out? (17:34):
undefined
Ejaaz: The business model is its state back, dude. Yeah. (17:38):
undefined
Ejaaz: All these Chinese labs are backed by the state. So that's how they get their (17:41):
undefined
Ejaaz: infinite amounts of capital to be able to kind of fund all these things. (17:44):
undefined
Ejaaz: And this is a strategy that China has been pursuing for three plus years now. (17:47):
undefined
Ejaaz: They want to catch up with the frontier Western models. they distill attack (17:52):
undefined
Ejaaz: a bunch of these model labs so that they can train similarly capable models. (17:56):
undefined
Ejaaz: And then they flood the market with cheap intelligence, which means that Anthropic (17:59):
undefined
Ejaaz: can't charge the highest price that they can for the high quality model that (18:03):
undefined
Ejaaz: they have. OpenAI can't do the same. (18:06):
undefined
Ejaaz: So this drives model costs down. This is what kind of Meta and Mark Zuckerberg (18:08):
undefined
Ejaaz: is also trying to do, but failing to do as effectively as the Chinese labs. (18:12):
undefined
Ejaaz: They're just very good at doing it. The second thing I'll say is, (18:16):
undefined
Ejaaz: at least according to Open Rata from their official statement. (18:19):
undefined
Ejaaz: They said that this stealth model provider, which we now know to be Jipu, (18:22):
undefined
Ejaaz: is retaining prompts and responses, but explicitly state that they're not using (18:26):
undefined
Ejaaz: it for training. Now, this isn't verifiable in any kind of way. (18:31):
undefined
Ejaaz: Like, okay, China, you've never done anything nefarious before, I'm sure. (18:36):
undefined
Josh: You're really going to give this away totally for free? Like, (18:39):
undefined
Josh: no way. Coming from the people that stole all the items, I can go and generate (18:41):
undefined
Josh: a full episode of Spongebob on those models. (18:45):
undefined
Ejaaz: Exactly. Exactly. Well, this is another unique thing, right? (18:47):
undefined
Ejaaz: Culturally, in China, and this applies for every vector of tech that they've (18:50):
undefined
Ejaaz: built in the past, electric cars, mobile phones, et cetera. (18:55):
undefined
Ejaaz: They don't really see copyright as a thing or IP infringement as a thing. (18:58):
undefined
Ejaaz: They see, okay, if you have a good product, I can copy that. (19:02):
undefined
Ejaaz: And Chinese in particular are very, very good at copying effectively. (19:05):
undefined
Ejaaz: There was this great interview I watched with, who was it? It was Travis Kalanick, (19:10):
undefined
Ejaaz: on David Semra's podcast. I don't know if you caught that, right? (19:14):
undefined
Ejaaz: Absolute legend, right? Twice. And he talks about his story where, (19:16):
undefined
Ejaaz: you know, how he brought Uber over to China and he made a success there, (19:21):
undefined
Ejaaz: but he said it was very tough. But there was also some really unique cultural (19:25):
undefined
Ejaaz: things that he needed to understand. (19:28):
undefined
Ejaaz: And he realized that in America, or in the West at least, the innovation level (19:30):
undefined
Ejaaz: is super high. Coming up with net new zero-to-one ideas, super high. (19:34):
undefined
Ejaaz: In China, less so, but they are masters of copying. And he originally looked (19:39):
undefined
Ejaaz: down at copying as a really menial thing. (19:43):
undefined
Ejaaz: But he said that the way that they do it is masterful and artful in itself. (19:47):
undefined
Ejaaz: And this, I feel like, is what some of these Chinese labs are really good at doing. (19:52):
undefined
Ejaaz: It doesn't matter if they're earning $100 million. Their IPOs are popping off (19:56):
undefined
Ejaaz: anyway. They're never going to compare to the Western IPO market at all. (19:59):
undefined
Ejaaz: But the point is, they want to flood the market and kind of like keep America (20:02):
undefined
Ejaaz: at a cost level where maybe they're spending too much money than they actually (20:06):
undefined
Ejaaz: need to until they're able to figure out some kind of a breakthrough, (20:09):
undefined
Ejaaz: which might be continual money. Who knows? (20:12):
undefined
Josh: Maybe. We'll see. It's an exciting story and we're going to find out more. (20:14):
undefined
Josh: So this will be covered and followed up in the roundup tomorrow. (20:18):
undefined
Josh: Stay tuned for that. There's also a reference. It's kind of cool. (20:22):
undefined
Josh: The main character of the story is Open Router. (20:25):
undefined
Josh: And we just covered an entire Open Router episode just last week. (20:27):
undefined
Josh: So everyone who is unfamiliar with Open Router and why it is so valuable, (20:32):
undefined
Josh: please go and check out that episode. (20:35):
undefined
Josh: And that's mostly the state of the mystery as of now. Ox Alpha has been solved, (20:37):
undefined
Josh: but there is still a lot of behind the scenes that is left to uncover. (20:43):
undefined
Josh: We're hoping to get that information shortly. when we do we will share it with (20:46):
undefined
Josh: you right here on the roundup hopefully this week so stay tuned for that one (20:48):
undefined
Josh: um this has been kind of a crazy week i mean it's (20:52):
undefined
Josh: China's always throwing like, they're throwing wrenches in everyone's plans. (20:56):
undefined
Josh: First it was Deep Seek. Now they're doing this like kind of really fun stuff. (21:00):
undefined
Ejaaz: Did you see Quen as well? (21:03):
undefined
Josh: Yeah, we got a new Quen model. We're going to talk about that tomorrow. (21:05):
undefined
Ejaaz: Dude, they're just like spaying out these models. I know. (21:08):
undefined
Ejaaz: And you know, the craziest part about all of this is I'm seeing rumors from (21:11):
undefined
Ejaaz: trusted sources that a bunch of American frontier labs are also about to like drop a few models. (21:16):
undefined
Ejaaz: So like, I'm just saying here, like, you know, we're getting like the best from (21:22):
undefined
Ejaaz: the Chinese, We're getting the best from the West and there's just too many models to use right now. (21:25):
undefined
Ejaaz: But if you're listening to this and you're curious about this Ox Alpha model (21:29):
undefined
Ejaaz: or if you don't believe anything that we're saying, if you don't think it's (21:32):
undefined
Ejaaz: good enough, go out and try it right now. You can use. (21:34):
undefined
Josh: It for free (21:37):
undefined
Ejaaz: On OpenRouter. Go check it out. There is no promo code. Just sign up, (21:38):
undefined
Ejaaz: make an account and you can kind of like inference it and use it yourself. (21:42):
undefined
Josh: And I'm curious, send us, (21:45):
undefined
Ejaaz: Exactly. Like use the free tokens, please. Use the Chinese state funded money (21:47):
undefined
Ejaaz: budget to kind of do your AI tasks. (21:51):
undefined
Josh: Just don't give it your proprietary information. (21:54):
undefined
Ejaaz: Do not, definitely don't do that because they're definitely using it to train (21:56):
undefined
Ejaaz: the next model. But like, let us know in the comments whether you actually like (21:59):
undefined
Ejaaz: the model, whether it actually is good at the toss that you give it. (22:02):
undefined
Ejaaz: And if it magically improves the next day because it's continual learning. We don't know. (22:05):
undefined
Ejaaz: Like, let us know in the comments, DM us, we're available on all socials. (22:11):
undefined
Ejaaz: Yeah, anything else, Josh? (22:14):
undefined
Josh: Yeah, no, just to, again, share this with your friends if they enjoyed it. (22:15):
undefined
Josh: If you have something to say, please leave a comment. We try to read all of them. (22:18):
undefined
Josh: If you enjoyed on your favorite podcast player you can leave us a five-star (22:22):
undefined
Josh: review there if you're listening to this (22:25):
undefined
Josh: you should be watching it's very fun visually you can watch this on youtube (22:27):
undefined
Josh: on spotify and yeah as always thank you so much for watching feel free to go (22:30):
undefined
Josh: get caught up on the other episodes and tomorrow we got a banger roundup like (22:35):
undefined
Josh: it's just gonna there's so much stuff to talk about it (22:38):
undefined
Ejaaz: Is our favorite episode of the week. (22:40):
undefined
Josh: We've been in like a lull lately where there had i mean it's been exciting but (22:42):
undefined
Josh: it hasn't been like oh my god this is crazy there's new frontiers forming and (22:46):
undefined
Josh: like hopefully this is the first of many instances in which we're going to rev (22:50):
undefined
Josh: back up that engine and get (22:53):
undefined
Josh: a whole bunch of new models so i'm very excited stay tuned for all of that where (22:55):
undefined
Josh: we'll be covering it and yeah we'll see you guys in the next episode (22:57):
undefined
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
The Bobby Bones Show
Listen to 'The Bobby Bones Show' by downloading the daily full replay.