codethief
> Notably, Hy4 preview also contributed to its own development process, participating for the first time in the automated optimization of training methods, data strategies, evaluation frameworks, and low-level operators. The model proposed approaches, ran experiments, and iterated based on the results, with the resulting code, logs, and feedback feeding into subsequent rounds of exploration. This established an early-stage recursive self-improvement loop.
This reminds me of one of the predictions from https://ai-2027.com/ . Only that there it's "OpenBrain" doing this, not the Chinese. And the authors of that paper were also slightly wrong about "Mid 2026: China Wakes Up": China woke up already a while ago. And:
> But China is falling behind on AI algorithms due to their weaker models. The Chinese intelligence agencies—among the best in the world—double down on their plans to steal OpenBrain’s weights.
No need to steal anything, they have already caught up.
And then there's this prediction for February 2027:
> Officials are most interested in its cyberwarfare capabilities: Agent-2 is “only” a little worse than the best human hackers
I think we're past that point now, too…
bredrenreply
If the distillation "attacks" created useful inputs to open weight models, ai-2027 was directionally correct that the Chinese would find ways to extract IP from western firms. (Scaled account creation and grinding outputs etc is not a dramatic story element as spies, though!)
Whether the distillation has constituted "attacks" or has or will meet the bar of "stealing" IP is not super interesting to me, though.
vlyanreply
the chutzpah of calling it an `attack` or `stealing` is super interesting tho.
haazreply
The idea that distillation is a significant contributor to the capabilities of the Chinese models is not true. Kimi K3 came out 2 weeks after Fable and uses a number of novel NN architecture innovations.
0xbadcafebeereply
The AI 2027 paper/website is exactly the same as random guesses from tech bros after a couple of beers telling you what they think the future will be. It has nothing to do with political theory, economic theory, game theory, or any other quasi-scientific or rigorous evaluation of real world events and predictable outcomes. It's just vibes. If they're wrong nobody will notice, if they're right people will call them geniuses.
try-workingreply
Just like how Windows 95 contributed to its own development process.
minimaxir
Hy4 apparently has ludicrous traction on OpenRouter already (https://openrouter.ai/tencent/hy4-preview), with trillions of tokens processed in a couple days: more than GLM 5.3 in a week. That said, it's relatively cheap with a 5% cache cost when everyone is still doing 10%/20% cache costs, so Hy4 may be more compelling.
cyanydeezreply
i'd be curious if openrouter is just being gamed by these publishers by paying for the exposure.
wouldn't trust they dont do Capitalism like the rest of the AI field.
drob518reply
Of course they are. Of course they do. Nobody should be surprised by this.
tokaireply
>dont do Capitalism like the rest of the AI field
Like lobbying the US president to harm their competitors?
realoreply
I would suggest "lobbying" is not the correct word to describe all the corruption going on in the current USA administration cesspool.
martinaldreply
I wrote about this a couple of weeks ago. It's actually often the biggest cost and it tends to be hidden away on most platforms!
https://martinalderson.com/posts/watch-out-for-cache-read-co...
Btw I still haven't came across any decent model that is <$0.01/MTok cache costs apart from deepseek thru their official API (even with the price increases).
Seems like a bit of an opportunity for someone to take - drop cache read costs significantly.
dakollireply
That's because Deepseek invented the paradigm of prompt caching, they are the SOTA when it comes these techniques. Despite them open sourcing all their research, nobody beats them.
edit: I do wish openrouter would let you sort providers by Cache Hit % and Cache cost. These are the only things that matter to me at this point when choosing a provider.
minimaxirreply
You can click the table headers to sort Ascending/Descending.
Bolwinreply
Cache hit % on openrouter is not a good metric, it's mainly driven by openrouter's own provider juggling than the providers themselves
orbital-decayreply
>Deepseek invented the paradigm of prompt caching
Caching was always here, you don't need to do anything special to get it on a single user local backend running a base model or a chatbot in the first place. Among commercial providers, OpenAI adopted it in 4o first.
sievereply
For me, an average long session results in about 200-300M cached input, 4-800K input, 2-400K output. Mostly the lower bound. Output depends on how much the model thinks.
There are two problems here:
- cache hit pricing (both Muse Spark 1.2 Contributor and MiMo 2.5 are around the $0.002-3/M mark)
- cache persistence time
Muse Spark drops the cache in less than 5m. MiMo keeps it around for at least an hour based on my experience with whoever is serving it for OpenCode. This difference itself will inflate bills massively.
A 500K token input repeatedly read by MS 1.2 for full input price 12 times an hour = $0.60. You would be expecting $0.012. So a 50x difference. Same thing on MiMo 2.5 is $0.018 because of longer cache times.
sourcecodeplzreply
even with the 5m cache, Muse Spark Contribs is still best bang for your buck for the intelligence you get.
it is basically the old dsv4-flash prices, but even more smart.
Dinuxreply
Which explains why almost none of my request go though
fastball
I wish model providers would stop committing chart crimes in their releases.
- if you're gonna order the rest of the bar chart by rank, order your model accordingly.
- if you're gonna highlight a winner in a table of benchmarks, don't highlight your entire model row in the table.
Etc etc
jimbob45reply
I wonder if this is being reinforced via LLM because they see every other modeler doing the same thing.
nullbioreply
I'd bet there's a correlation between benchmaxxing and chart crimes. Companies who try to deceive perceptions via the charts are more likely to cheat at the benchmarks too, I'm sure. That's assuming ill intent, of course - which is often the case for charts related to model releases, but not necessarily always the case.
petcat
> Tencent has released and open-sourced Tencent Hy4 preview, a next-generation large language model with 770B total parameters and 49B active parameters, and a context window exceeding 1M tokens.
There are no open source models, at least not useful ones (yet) [0]. Open weight is not the same as open source. The current "open weight" models are just opaque binary blobs you can run on your own computer instead of through a web API.
[0] https://allenai.org/
Imagine thinking that running a Photoshop binary on your own computer instead of through a SaaS web app means that it's "open source". Of course you think that's ridiculous.
mirekrusinreply
You can open source dataset without all the details how it was assembled.
Models are lossy compressed datasets you can pick up and amend (fine tune / continue training / alter) according to license they were released under.
Hy4 is released under OSI approved Apache License 2.0.
kennywinkerreply
Parent poster is technically right - open “source” implies the source used to make something is open. The model source is training data and code, not just weights.
But the reality is, the weights are a useful artifact that you can use to create derivative works. So, dismissing it as a photoshop binary is as technically wrong as calling it open source.
Alpha3031reply
IIRC Nvidia claims to release enough data that it should be possible to fully reproduce Nemotron, so even if it's not as good as the current best models, GPT 5.1 or Opus 4.1.was still useful right? I guess it depends on what you wanted to do with them.
LtWorfreply
So windows is open source because the binaries are a lossy compression of the original source?
NitpickLawyerreply
Weights are not binary. A model is created at init time, with random values. After that, it is being modified using data. The key point is that the labs modify the models "as weights". That means that weights are the intended / preferred way of modifying a model. Which, coincidentally, matches the definition of source in Apache 2.0. There is no "higher level" place where editing takes place. It all happens in weight space. Through the license you get the same rights as the lab that created it: view, inspect, run, modify, re-release. That's it. That's the only thing a license can grant you.
The rest is semantics, misunderstandings, and FUD. A model released under an open source license is open source. Training data is lab knowhow / IP. Which, historically, has never been required for any open source release.
petureply
Before we worry about source code, Microsoft doesn't grant me rights to modify/redistribute/sell copy of Windows I have.
villishreply
Countries that aren’t competitive need access to training datasets so that they may train their own similarly capable models and be sure of the inputs. Governments cannot blindly trust open weight models from China and the US.