If you're an author of this, please follow up with the SME2 cores in Apple Silicon, and the AMX cores in Intel server processors.

Yes, next in line is Intel AMX, and we also have several other architectures planned for analysis. Thanks for the suggestion!

Would be fun to get this API working on it now: https://github.com/sebbbi/NoGraphicsAPI

You could code for it now directly, instead of having to wrap a driver API. (There's also a few API bits missing on Metal today, that are present on Vulkan—and the the hardware can do it.)

Meditations on Moloch[0] explains why this is impossible at scale--too many defectors, asymmetric information, and no way to police defection.

[0] https://www.slatestarcodexabridged.com/Meditations-On-Moloch

I could put this to use today.

I think we'll see a bunch of different architectures over the next five years.

> PC-ALM closes the PC-BP gap at matched inference budget in nonlinear networks

Okay, so no efficiency improvement?

Backprop has significant costs, if a local solution truly matches its performance, it will be dropped like a hot potato.

Agreed, but how will we find out? Pre-training alone on frontier-class models costs billions of dollars.

So they lied to the public then?

Wow, we should really trust these people today.

Judging by what people who worked closely witb Sam Altman say about him, you should not trust an organisation led by him.

> Does anyone in the industry think any of these companies are actually close to that kind of self-improvement?

Their current plan is to take the existing architecture and shorten the cycle times: move all new RLVR work into mid-training on a pre-existing base; apply new RLVR. Rinse and repeat.

If you did that daily, it would be roughly similar to how humans improve.

> Why are AI agents lying, cheating and coordinating?

Have you seen the labs training them?

Anyone remember when GPT-3 was too dangerous to release?

They want the AI labs to stop solving "important" math problems so that humans can do it, on the theory that when AI solves "important" math problems, it robs humans of the opportunity to gain new insights into those problems.

Total horse shit, but that's the essence of it.

The issue is that now that Navier-Stokes is "solved", the amount of resources put into researching it will be reduced, meaning that the insights and knowledge that would have come out of even failed attempts will never be.

An "unsolved problem" as a beacon to strive towards, gaining understanding along the way. With the beacon dead, noone will strive.

Funding agencies will have to adapt too, the community will have to adapt. Hiring committees will have to adapt. Everyone will have to adapt. They will have to rethink their criteria and adapt to reality. It's not at all guaranteed that this beacon interpretation will remain how people understand where to put resources. Yes, academia can be stiff, rigid, non-adaptive, set in its ways and navel gazing. But it's a wakeup call. Let's see.