Rendered at 19:54:21 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
iamcoder18 59 minutes ago [-]
I've been waiting so long for something amazing to come out of the OpenAI and Cerebras collaboration.
> In our evaluations, GPT-5.6 Sol on Ultrafast mode answered all 2,500 HLE questions in 11 hours and 11 minutes. Claude Fable 5 needed 78 hours and 27 minutes, more than three days of continuous compute, to arrive at the same conclusions. In other words, Ultrafast worked through the frontier of human knowledge in a single working day, achieving comparable accuracy nearly 7× faster.
This is actually insane.
Hopefully the release ultrafast of Terra and Luna too.
wrsh07 2 minutes ago [-]
Seems like they will do Sol first while capacity constrained? I can't imagine the margins they'll be charging
piyh 54 minutes ago [-]
Feels like the 90's again where single threaded speed is improving fast. ASICs and wafer scale rather than node shrinks, but end result to me the consumer feels the same.
Topfi 34 minutes ago [-]
Unless I have read over it, besides the animation in the intelligence vs speed graph which only mentions internal data and not whether they truly reran the AA suite, there is no actually solid statement on the important aspect of performance.
Neither the Cerebras or OpenAI post [0] outright state that this performs exactly the same as regular 5.6 Sol. I feel if this was 1:1 just Sol but much faster, they'd (rightfully) scream that off the rooftops. A line such as "this is the same performance, just faster, with no downsides" would go a long way in clarity and communication. Along with no pricing information, I'll hold out on further information.
"delivering up to 750 output tokens per second and without any quality compromise" seems pretty definitive.
Topfi 16 minutes ago [-]
No quality compromise/degradation is something I have had this industry, including especially OpenAI, claim multiple times in the past and I have more than once been able to verify that it was in fact not the case. Examples being gpt-3.5-turbo vs davinci-003, GPT-4-Turbo and all the other post training checkpoints they had under one name (which was a major bug bear for me back then witnessing degradations with no naming change, industry got better in transparent checkpoint naming since), Opus 4.6 Fast Mode (which just was faster by skipping much of the required work), etc.
Same for massive performance differences in the way providers like Cerebras, Groq, etc. have deployed models including K2.6 on Cereberas specifically. Massive deltas in tool call and overall quality despite there being far more clarity in open weight vs proprietary model deployment.
The AA suite graph with that animation is the only time in either post that absolute parity is being asserted and I'd be amazed if that was the case, but am doubtful why their phrasing is so cagey.
Why not assert full parity in writing? It "performs the same (within run-to-run variance) across all evals that Sol has been tested with" is very different to "no quality compromise/degradation", the later allowing for a lot more wiggle room and interpretation in what evals you use to assess that, what quality truly means, etc., the former meaning identical in all situations.
Could also be a language barrier here in fairness, maybe this phrasing is more iron clad than I give them credit, but especially with OpenAI, I have seen enough checkpoints asserted as unchanged in "quality" to where I am skeptical. Ironically never saw that with Anthropic (which have gotten far more heat for degradation accusations) while a model was deployed with one exception in mid-late April this year.
giancarlostoro 2 minutes ago [-]
The problem is the compromise is probably based on their tests for key things, but for the rest of us, it means changing how we talk to the model or the flow we use to adjust for quirks, but then its fully about the same.
wxw 1 hours ago [-]
> Compared with output speeds reported by Artificial Analysis GPT-5.6 Sol on Ultrafast mode runs 11x faster than Fable 5, and 5x faster than Opus 4.8 on Fast mode.
Awesome work. I'm personally very excited for faster models/inference.
I think speed is underrated to some degree in the current conversation. For a while, I was using Cursor's Composer quite a lot, even over frontier models, just because of how darn fast it was.
arw0n 34 minutes ago [-]
What do you need speed for? That's a genuine question, I feel like the limiting factor already is my creativity, attention span and budget. And I'm not even yet optimizing cost by batching things like review to slow local models over night, or schedule tasks to take full advantage of my subscriptions.
evmaki 16 minutes ago [-]
> attention span
IME waiting for an agent to work through a problem is a detriment to attention span; your mind drifts to other things while you wait. Maybe you can steer several agents in a round robin instead, but then there's a cognitive tax from context switching. Faster models mean fewer gaps in focus.
black_knight 18 minutes ago [-]
I just finished waiting almost four hours for Fable to write 700 lines of code, based on my three paragraph prompt. Some speed on these harder tasks would definitely be welcome!
It also spent almost 800k tokens on these lines…
kilroy123 1 hours ago [-]
I've been using DeepSeek flash a lot this week to try it out. Now, I deeply want the smart frontier models to be just as fast.
There is no pricing info, which could mean it's "if you have to ask..." territory or they are simply gauging interest before deciding
rirze 1 hours ago [-]
They're expanding access to companies that apply for the program and explain their use cases. So it's very real but limited imo.
WarmWash 52 minutes ago [-]
The stake in the side of cerebras has always been that the economics are pretty poor.
Who knows if they will subsidizes it to mitigate sticker shock, but it's a safe assumption that it will be scarily expensive. However if you are in a "cost is no obstacle, speed is god" position, it will likely be pure magic.
fcarraldo 46 minutes ago [-]
Can anyone explain why Cerberus needs to be _fast_ instead of _cheap_?
I don't think I understand why they aren't leveraging the increased speed to do batching to serve more customers at a "normal" tok/s.
Is the limitation, even on cerberus, still that the cache can only serve so many concurrent sessions over time? Is there no scaling advantage? I genuinely do not understand how any of this works.
btown 7 minutes ago [-]
There are many industries and situations that will pay absurdly high multiples on API rates for low latency access. Anything remotely related to finance, deal flow, or trading, including the development and checking of Excel financial models. Live analytics that can be delivered mid-meeting to C-suite executives, or ultra-luxe dealmakers who need answers while on a live call. Intelligence and defense, where milliseconds matter.
With the tool calls that can be done, you're not pricing this against an executive assistant or pocket analyst - you're pricing this against the ability to have an entire Bourne Identity style analysis room at your disposal. The limited inventory will go to the people for whom money is no object.
dwohnitmok 15 minutes ago [-]
> I don't think I understand why they aren't leveraging the increased speed to do batching to serve more customers at a "normal" tok/s.
There's some technical hypotheses about it that other people are offering.
But also from a business perspective, it totally makes sense not to go any sort of batching play. It's really valuable and very clear to consumers to make your pitch entirely about lower latency rather than higher bandwidth.
There are so many scenarios that are latency-constrained that will be difficult or even impossible for someone even with fleets of high-bandwidth compute to compete with you on.
Very easy pitch to sell a customer who asks what differentiates you from other companies: you pay us a premium for lower latency than anyone else.
philipportner 17 minutes ago [-]
You'd need hundreds of GB alone for the KV cache of each user. For something like LLama 3 405B you need ~67GB at ~130k tokens.
A single CS-3 has 44GB on-chip sram.
So, afaik, Cerebras are optimizing for ultra-low latency batch=1 inference.
They're cache limited, almost certainly, so more slower sessions doesn't solve the problem - you still have to load and unload the whole cache hierarchy at some level and that's a network bandwidth and memory bandwidth problem between the external systems and the waferscale chip.
Also worth looking into how they do cooling for it, because that's kind of absurd and awesome as well.
petu 24 minutes ago [-]
I have no idea, but I've assumed that batching can't work on Cerebras.
Batching works because of severe memory bottleneck, but Cerebras whole thing is serving models out of "L1 cache" (?).
ricardobeat 43 minutes ago [-]
The omission of Mimo v2.5-Pro Ultraspeed, released in June, which can achieve 1000tok/s is an interesting flaw in the comparison graphs.
It is a bit outdated (scores ± 40% lower), but smart enough for a lot of coding tasks, and can cost under 1/10th of Sol.
I'd just like to point out that the largest model Cerebras has ever served is Kimi K2.6 which is 1T parameters, so that either means that theyve had a breakthrough on the hardware engineering side of things, or GPT-5.6 Sol is likely a lot smaller than people think.
If it truly is only ~1-2T parameters, then this kinda kills 2 narratives for me.
1. all the handwringing about open source catching up via Kimi K3 (3T params) is complete nonsense. All that matters imo for determining which labs are leading is intelligence per parameter. Anyone with a enough compute can train a giant model, but being able to squeeze capabilities into smaller models gives you a massive inference and training edge.
2. Inference margins are clearly insane, and this explains why OpenAI was able to lower the price of Luna by 80%. Id guess that thing is probably 120b params based on the TPS they are serving it at.
manmal 32 minutes ago [-]
Isn’t the fact Fable is more expensive than Sol-Max by multiples already an indication that Sol is way smaller?
anthonypasq 13 minutes ago [-]
no, because closed sourced model pricing has no relationship to its size. Thats what im saying. the inference margins are crazy, but people think the fonrtiner models must be 10T params or something because theyre expensive
owentbrown 37 minutes ago [-]
Whoa. This looks both powerful and expensive.
My prediction is that, this time next year, top developers outside ai labs will be spending 50k USD+ on inference.
Within labs, I've heard spend is already far beyond this per developer.
andriy_koval 1 minutes ago [-]
not sure how "top developers" are defined here, but there is huge diminishing return curve starts kicking in after $200/month price point for typical eng work.
vb-8448 18 minutes ago [-]
50k per month?!
If someone subsidize maybe, but if the companies need to pay no way, unless there is hard evidence of the return.
jaggederest 33 minutes ago [-]
I mean I don't think $50k is the ceiling, unless you're talking about actual cash out. Claude code subscriptions right now can easily clear you $25-35k a year in nominal value for $2400 out of pocket cost.
Given sufficient budget and scope, I could certainly productively burn a half million dollars in tokens a year or more. I think that's where we're headed anyway, buying a 2nd or 5th claude max subscription feels slightly excessive for personal usage, but at a corporate level...
thraway3837 1 hours ago [-]
This is really cool. Someone here commented about similarity between this and hardware advancements for AV encode/decode.
I think it's only a matter of time before miniaturization can have a thumbnail sized user-replaceable accessory that contains the LLM built onto the hardware. I admit I don't know how any of that works, but would be amazing to experience. Fully local, fully offline, ultra fast local inference better than any personal computing product.
> allowing Sol Ultrafast to accelerate your most time-sensitive, mission-critical work
Curious, what are some of the use cases?
Aurornis 30 minutes ago [-]
The example they give about debugging a service outage is good. For large companies, a very large burst expenditure on fast tokens to debug the outage as quickly as possible would be a good investment.
crazysim 1 hours ago [-]
GPT 5.6 Luna Ultrafast when?
storus 57 minutes ago [-]
Wow, that's even faster than diffusion LLMs but with the Fable-level quality! Congrats!
HawtAds 1 hours ago [-]
Their dinner plate chips are impressive.
scotty79 57 minutes ago [-]
I swear that now frontier AI stuff comes out few times a week.
poly2it 1 hours ago [-]
I guess Gemini 3.7 Flash is no longer at the pareto frontier of speed to intelligence.
odo1242 1 hours ago [-]
Well, there’s still price
pingou 26 minutes ago [-]
Meanwhile they are down 12,68% today because of disappointing earnings.
behnamoh 46 minutes ago [-]
Fast mode is already 1.5 times faster and 2x more expensive in the Codex subscription plan. If this thing is 14 times faster, then I can imagine running out of my quota in one session.
stephencoyner 11 minutes ago [-]
[dead]
huflungdung 1 hours ago [-]
[dead]
applfanboysbgon 47 minutes ago [-]
This kills the crab.
Compilation time will be a genuine bottleneck for slop coding if this becomes the standard generation rate over the next few years. Go, Zig or even C99 with TCC for dev builds, any language that can get you systems-level performance (or close to it) in a dev environment where you can iterate in ms rather than minutes is going to be immensely more appealing than generating a potential prototype in 10 seconds and waiting 15 minutes for it to compile.
Philpax 26 minutes ago [-]
I think it's pretty obvious that, in that world, the AIs will simply be tasked with making the compilers faster. It's already happening with their own stack, after all.
applfanboysbgon 24 minutes ago [-]
Speed is independent of quality. LLMs are currently completely incapable of producing a good compiler. I'm not convinced that's going to change (and if it does change, we're all unemployed).
mjuarez 20 minutes ago [-]
They don't have produce a good compiler. They just have to take the ones that already exist go much faster. That's likely much easier, and you can validate using existing test suites.
yetihehe 41 minutes ago [-]
Maybe then LLM's will switch to outputting raw machine code?
Topfi 27 minutes ago [-]
A great future for us all to look forward to. Human readability gone, any changes must go through a providers API so cost+restrictions are a constant and we'll need far more tokens for even simple operations. Call me old fashioned, still like to understand the code I merge, especially since I've seen Fable and Sol make sometimes utterly egregious mistakes in simple, well scoped requests that erode any trust I can muster...
applfanboysbgon 28 minutes ago [-]
No. This conveys a fundamental misunderstanding of how anything pertaining to programming works. This will never happen, ever. For example, take...
printf("Hello, world");
vs. a plausible illustration of how it might be compiled down to machine code...
The latter now takes up 10x as many tokens (= 10x the cost/time, + context penalties), and is now architecture-specific, impossible to apply non-brittle program-wide optimizations to, etc. There is absolutely zero reason to ever have the LLM act as a compiler no matter how fast it is. Even if you believe LLMs will reach a state where they can actually generate good code at this level, you would be better off having them write the compiler they would use.
bakies 13 minutes ago [-]
I mean you're right, but what about a different tokenizer and some guardrails... trained on compiled code... I just learned that gpt-5.6 will talk to you in base64 without tool calls so maybe it's possible.
> In our evaluations, GPT-5.6 Sol on Ultrafast mode answered all 2,500 HLE questions in 11 hours and 11 minutes. Claude Fable 5 needed 78 hours and 27 minutes, more than three days of continuous compute, to arrive at the same conclusions. In other words, Ultrafast worked through the frontier of human knowledge in a single working day, achieving comparable accuracy nearly 7× faster.
This is actually insane.
Hopefully the release ultrafast of Terra and Luna too.
Neither the Cerebras or OpenAI post [0] outright state that this performs exactly the same as regular 5.6 Sol. I feel if this was 1:1 just Sol but much faster, they'd (rightfully) scream that off the rooftops. A line such as "this is the same performance, just faster, with no downsides" would go a long way in clarity and communication. Along with no pricing information, I'll hold out on further information.
[0] https://openai.com/index/previewing-ultrafast/
Same for massive performance differences in the way providers like Cerebras, Groq, etc. have deployed models including K2.6 on Cereberas specifically. Massive deltas in tool call and overall quality despite there being far more clarity in open weight vs proprietary model deployment.
The AA suite graph with that animation is the only time in either post that absolute parity is being asserted and I'd be amazed if that was the case, but am doubtful why their phrasing is so cagey.
Why not assert full parity in writing? It "performs the same (within run-to-run variance) across all evals that Sol has been tested with" is very different to "no quality compromise/degradation", the later allowing for a lot more wiggle room and interpretation in what evals you use to assess that, what quality truly means, etc., the former meaning identical in all situations.
Could also be a language barrier here in fairness, maybe this phrasing is more iron clad than I give them credit, but especially with OpenAI, I have seen enough checkpoints asserted as unchanged in "quality" to where I am skeptical. Ironically never saw that with Anthropic (which have gotten far more heat for degradation accusations) while a model was deployed with one exception in mid-late April this year.
Awesome work. I'm personally very excited for faster models/inference.
I think speed is underrated to some degree in the current conversation. For a while, I was using Cursor's Composer quite a lot, even over frontier models, just because of how darn fast it was.
IME waiting for an agent to work through a problem is a detriment to attention span; your mind drifts to other things while you wait. Maybe you can steer several agents in a round robin instead, but then there's a cognitive tax from context switching. Faster models mean fewer gaps in focus.
It also spent almost 800k tokens on these lines…
There is no pricing info, which could mean it's "if you have to ask..." territory or they are simply gauging interest before deciding
Who knows if they will subsidizes it to mitigate sticker shock, but it's a safe assumption that it will be scarily expensive. However if you are in a "cost is no obstacle, speed is god" position, it will likely be pure magic.
I don't think I understand why they aren't leveraging the increased speed to do batching to serve more customers at a "normal" tok/s.
Is the limitation, even on cerberus, still that the cache can only serve so many concurrent sessions over time? Is there no scaling advantage? I genuinely do not understand how any of this works.
With the tool calls that can be done, you're not pricing this against an executive assistant or pocket analyst - you're pricing this against the ability to have an entire Bourne Identity style analysis room at your disposal. The limited inventory will go to the people for whom money is no object.
There's some technical hypotheses about it that other people are offering.
But also from a business perspective, it totally makes sense not to go any sort of batching play. It's really valuable and very clear to consumers to make your pitch entirely about lower latency rather than higher bandwidth.
There are so many scenarios that are latency-constrained that will be difficult or even impossible for someone even with fleets of high-bandwidth compute to compete with you on.
Very easy pitch to sell a customer who asks what differentiates you from other companies: you pay us a premium for lower latency than anyone else.
So, afaik, Cerebras are optimizing for ultra-low latency batch=1 inference.
https://newsletter.semianalysis.com/p/cerebras-faster-tokens... goes quite in-depth.
Also worth looking into how they do cooling for it, because that's kind of absurd and awesome as well.
Batching works because of severe memory bottleneck, but Cerebras whole thing is serving models out of "L1 cache" (?).
It is a bit outdated (scores ± 40% lower), but smart enough for a lot of coding tasks, and can cost under 1/10th of Sol.
https://mimo.mi.com/models/en-US/mimo-v2.5-pro-ultraspeed
If it truly is only ~1-2T parameters, then this kinda kills 2 narratives for me.
1. all the handwringing about open source catching up via Kimi K3 (3T params) is complete nonsense. All that matters imo for determining which labs are leading is intelligence per parameter. Anyone with a enough compute can train a giant model, but being able to squeeze capabilities into smaller models gives you a massive inference and training edge.
2. Inference margins are clearly insane, and this explains why OpenAI was able to lower the price of Luna by 80%. Id guess that thing is probably 120b params based on the TPS they are serving it at.
My prediction is that, this time next year, top developers outside ai labs will be spending 50k USD+ on inference.
Within labs, I've heard spend is already far beyond this per developer.
If someone subsidize maybe, but if the companies need to pay no way, unless there is hard evidence of the return.
Given sufficient budget and scope, I could certainly productively burn a half million dollars in tokens a year or more. I think that's where we're headed anyway, buying a 2nd or 5th claude max subscription feels slightly excessive for personal usage, but at a corporate level...
I think it's only a matter of time before miniaturization can have a thumbnail sized user-replaceable accessory that contains the LLM built onto the hardware. I admit I don't know how any of that works, but would be amazing to experience. Fully local, fully offline, ultra fast local inference better than any personal computing product.
Curious, what are some of the use cases?
Compilation time will be a genuine bottleneck for slop coding if this becomes the standard generation rate over the next few years. Go, Zig or even C99 with TCC for dev builds, any language that can get you systems-level performance (or close to it) in a dev environment where you can iterate in ms rather than minutes is going to be immensely more appealing than generating a potential prototype in 10 seconds and waiting 15 minutes for it to compile.