mccoyb
Here's this boiled down:
> A stochastic search process with an executable optimization objective over space of programs S can only maintain or improve the objective
This is superoptimization. We've known this since the 80s (Massalin, STOKE is more recent: https://github.com/StanfordPL/stoke) The only novelty is that the proposer is now way better with LMs.
Further, there's a large number of reasons for software written by agents to be slow:
- LMs still don't do data or hardware-oriented design well out of the box, and therefore if you're engaging in any sort of serious novel work, beyond porting an extremely well-understood program with extremely well-understood workloads, you're going to be spending hours tracking down bad allocation decisions (c.f. why TigerBeetle doesn't use agents), which are often the root of evil (before you'd reach for anything further)
- The knobs you'd need to get serious performance are nearly unreachable in languages which LMs are good at (even Rust requires a discipline that the default language doesn't enforce). When you drop into the lower realms, you're trading consumption context for access to these levers. The levers are also "soft": you find yourself writing a bunch of skills, and tools to try and enforce the discipline.
The reality is to get performant code (quickly) out of an agent, you need to know how to write performant code (and you need to know how to surface the information that you'd use to create a verifier for such a thing to the agent), which 99% of developers do not know in 2026.
Sure, agents can teach you how to do this -- but it's one of these things where iykyk.
Experience: I've poured 10s of billions of tokens into Zig with the best agents and I have the time and space to try these things.
If you want to start learning the discipline, I'd recommend matklad's + TigerBeetle blog -- as well as hardware-oriented design.
Aurornisreply
> - The knobs you'd need to get serious performance are nearly unreachable in languages which LMs are good at (even Rust requires a discipline that the default language doesn't enforce). When you drop into the lower realms, you're trading consumption context for access to these levers. The levers are also "soft": you find yourself writing a bunch of skills, and tools to try and enforce the discipline.
Hasn’t been my experience at all. The latest LLMs can knock out assembly optimized subroutines and benchmark 100 different variations faster than I ever could dream of.
mccoybreply
That's fair for a well-scoped subroutine: what I meant is that if you ask an agent to write a compiler and let it rip for a few days, you are going to be spending a few more days correcting the default behaviors in the distribution, which often do not tend towards hardware-oriented design.
To correct those behaviors, you're going to write tools and skills, and that's going to help, but it is still clear that you are fighting the distribution (today).
embedding-shapereply
Yes, say "Build a compiler" will require you to clean up stuff if you leave the agent for days, but not because of the LLM or the quality of the tool, but because you hardly specified anything, so of course it's gonna make assumptions you need to correct.
If you instead spend a day writing a proper specification, then ask the agent to spend a week implementing that, you'll need zero tools and skills afterwards to clean it up, because there won't be any misunderstandings, assumptions or other things, just code fulfilling what the specification says.
Granted, this does require you to not use obviously dumb models, like anything you can run locally today, and at least within reasonable range of SOTA models. But they been able to do this for 6 months or more at this point.
senderistareply
Agreed, IME Fable can churn out decent SIMD kernels optimized for whatever tradeoffs you give it.
josephgreply
> iykyk
A story.
I have a friend, who is - like me - interested in the CRDT / collaborative editing space. He asked ChatGPT to write him a CRDT. Then he grabbed every good CRDT implementation, and asked chatgpt to benchmark and optimise his CRDT, using tricks and techniques from existing hand-optimised CRDTs. He got massive performance gains by doing this - which is really interesting! I think it helped that he had a clear objective function, and chatgpt could look at other projects for ideas on how to optimise.
He proudly boasted that his resulting code outperformed my diamond-types library. I asked him if he was comparing against the native implementation, or the -Oz webassembly build, running in a wasm vm. It was the latter. When he tested it properly, his CRDT was - and is - significantly slower than diamond types. As far as I know, chatgpt still hasn't been able to catch up. I tried myself using fable. Even with reference to my source code, Fable still doesn't understand what I did in diamond types and why. (... Maybe I should document what I did!)
I think his technique itself is solid though. I tried it myself. I asked fable to write a custom binary serialization format & parser. Then optimise. Then optimise, with explicit reference to existing libraries. Optimising with reference to other code made a huge additional difference. It is now nearly as fast as those libraries. (But still not faster than them.)
My takeaway is this: I think LLMs are exceptionally good at reading and understanding code. If you guide them to do so, they're good at profiling and benchmarking. But it seems like they're not very good at coming up with novel optimisations. If you have an obviously slow program (for example, some slop claude wrote), you can often get big speedups by asking it to benchmark and optimise. But if you have a complex, already well optimised codebase, like the zig compiler, claude doesn't seem very good at figuring out novel ways to improve things on its own.
This is good news for the 95% of slow software out there. But bad news for the 5% of us who write fast code already, but want our code to go even faster.
fc417fc802reply
The refutation of your takeaway is autoresearch and similar. They can brute force novel optimizations (and generally achieve superhuman performance) when provided with an appropriate environment.
Of course that doesn't mean they have a human level mental model and associated novel ideas. Brute force can be effective but remains entirely unsatisfying from an academic perspective.
josephgreply
True! How do you set up auto research loops? Are there any special tricks to it?
zemreply
yep, i think the real LLM superpower is knowing that something has been done before and having access to the code that did it. so much of even novel software includes bits and pieces that have well-optimised existing solutions, and the bot knows those solutions a lot better than i do, and can even pattern match them from the general shape of the problem.
josephgreply
Yes. It’s also excellent at reading large codebases and putting together a picture of what’s going on. I’ve been using it a lot lately to brief me on projects and design decisions. “Look at these two projects. They both solve task X. Write a report about their similarities and differences, and the tradeoffs as a result.” And then I ask followup questions. Saves a ton of time.
gritzkoreply
Hi Joseph,
A story for a story. I had my CRDT implementation in libdog, which does per-token CRDT weave/diff/merge over a DAG of git blobs. It was written by Claude 4.8 I believe, in several iterations. It was, as you may guess, a piece of neuro-slop that passed the tests by some miracle. Once I had some time to look into it, I used a trick: I supplied it with my article on Chronofolds and some helpful kicks in the butt. It implemented everything correctly on its k-th attempt, k<5. Then I used it with full intensity for three months without thinking twice. Now I have started mass-using it to resolve permalinks in the code. Like, tens of files to chronofold-ize per one commit. It is now showing up in the profile, so I may look into it once more.
Conclusion: it sort of expands its context and prompt by association. It has no sense of direction of its own maybe. Once you know what you are doing, you can ride it fine. If not, it makes a misstep so later things go haywire, and you are left guessing why. (And how can you guess if you have not had the experience. We all grew with Commodores and suchlike. I recall Spectrums, Robotrons, and Poisks. My friend sits on exam committees, says the youth arrives flatlined after 3 years of GPT. His words. Whatever.)
josephgreply
> It has no sense of direction of its own maybe. Once you know what you are doing, you can ride it fine. If not, it makes a misstep so later things go haywire, and you are left guessing why.
This is a perfect description. Last week I asked claude opus to get AAC audio working in davinci resolve on linux. It managed to add aac in mov and mp4 containers very quickly. But mkv was another matter. For mkv files, resolve doesn't use ffmpeg. Instead, it has its own parser. Claude got totally lost down a weird rabbit hole trying to add aac support to resolve's mkv code. It was really struggling. Claude even knew it was lost - it kept telling me we should cut our losses and I should just release aac support without mkv.
Eventually I gave it the executable for davinci resolve on mac, which has aac support. Claude found the corresponding part of the code for the mac version and used it as a reference. Turns out, claude had made some much earlier mistake. Just like you said, it was going down a wrong path. Then it couldn't stop itself, and it kept making it worse.
Using the mac version of the binary as a reference, claude figured out how to get everything working very quickly. But - I'm left wondering. Maybe my real mistake was using Opus and not Fable. I wonder if fable would have been smart enough to figure out the mistake and course-correct.
hiddencostreply
> A stochastic search process with an executable optimization objective over space of programs S can only maintain or improve the objective
Reasons this doesn't follow:
(1) Benchmarks never match real world use, and many optimizations the improve benchmarks degrade cases that aren't measured (think about how CPU cache behavior can be surprising)
(2) In software performance optimization, frequently there is significant noise, from many sources. This makes it difficult to guarantee that a measured change is actually an improvement.
fc417fc802reply
Both of those things are indicators of deficiencies in the testing process.
rbehrends
I think this oversimplifies the problem of dealing with performance issues. In my experience, there are three aspects that contribute to the software engineering cost of performance optimizations:
1. Identifying the cause of poor performance.
2. Implementation.
3. Architectural impact (performance is a classic case of a cross-cutting concern)
(I am specifically leaving out the case of realtime systems, hard or soft, where additional factors come into play.)
But the article seems to focus entirely on the second aspect, while largely ignoring the other two.
Most performance bugs are not difficult to fix [1, 2], but can sometimes be hard to identify. Implementation effort is not the driving limitation in those cases.
Conversely, other performance improvements may affect the overall design, e.g. 27% of all bugs identified in [3].
Having an obvious, self-contained optimization target with a benchmark where algorithmic optimization within a module is also the key problem seems to be the exception, not the rule.
Also, not all performance issues are created equal. Many have trivial cost.
In contrast, some of the most challenging performance issues are the ones that affect the design and architecture of the system. After all, the hard part of software engineering is not writing a small, self-contained application. It's managing system complexity, while maintaining (in the words of Fred Brooks) conceptual integrity. Fixing performance issues is at least in this regard not fundamentally different from fixing other software defects.
Unfortunately, this is an area that is also full of trade-offs, such as performance vs. architectural simplicity, or performance in one part of the system vs. performance in another part, all of which requires judgement.
For example, you may need to bypass an abstraction boundary or reorganize abstraction boundaries to improve performance. Or you may have to special-case something while keeping duplicated code at a minimum and easy to maintain.
This is not to say that agents cannot help here, too. In fact, agents can be very helpful at e.g. identifying bottlenecks that are not directly visible in a profiler or can be used quickly do c
hunterpayne
"LLMs are causing slow, bloated, code are going to eat crow once they re-write everything in super-optimized assembly."
This person doesn't understand how to make efficient code. I can write code in almost any language (with a couple of exceptions) that outperforms "super-optimized assembly". Writing efficient code isn't about the language, and often isn't about the best algorithms either (but sometimes it is). Its about optimizing memory and cache use. And that's orthogonal to anything the author is writing about. Also, LLMs are terrible at optimizing memory utilization. There is just too little training code that does it well and far too much that doesn't.
As proof, I'm can literally feel the web getting slower and I bet many others feel this as well.
malisperreply
> This person doesn't understand how to make efficient code
The author is one of the most knowledgeable people about performance there is
geraneumreply
Seems like author’s main focus recently is AI and agents unsurprisingly, hence the suspicion. But it seems like he has a backgrounded in relevant fields in the past.
whatisthisevenreply
My contradictory proof: I have been working on an old service with tons of performance issues, from server memory bloat, client graph rendering, excessive network requests, excessive repeat rendering, memory leaks, resource leaks, etc.
The app and service are measurably and subjectively faster. Because I chose to have the LLM focus on solving those problems. It obviously can. It described the issues in big-O.
It is a priority problem, as it always has been, not a knowledge or skill problem, like it always has been.
userbinatorreply
Also, LLMs are terrible at optimizing memory utilization.
I've already posted this elsewhere, but here it is again: https://news.ycombinator.com/item?id=49226923
A vibe-coded OS that runs on an 8088 with 256KB of RAM.
As proof, I'm can literally feel the web getting slower and I bet many others feel this as well.
"It's not the tool, it's how you use it..."
ehnto
One of the biggest causes of slowness is just waiting for web requests. The fact that so much software is either online or built using the same stack even if it isn't, puts all that software in this blocked/waiting state constantly while using it.
Anyone not in the US feels this even more since so much online is US hosted, 300ms for every little interaction adds up quick.
If your software has the affordance of a waiting dialogue or loading wheel for many of its UI controls, you are building with this default blocked assumption. Even if you are building something web based, ask yourself if that's actually necessary for your software or if you could build it differently to avoid constant UI blocking.
hsn915reply
It's not very hard to engineer software with these two constraints at the same time:
* Must feel very responsive
* Network requests can take up to 500ms end to end
alightsoulreply
That's why phones and windows use animations. You can also use intersitials related to the product you sell. Users are usually fine seeing many changes on the screen quickly because it gives the impression that stuff is happening on the background. For example in the interstitial, use an animation that takes up a small portion of the screen and not just a simple spinner or loading icon. Something more complicated with 2 or more things moving or changing at once.
ddejohnreply
> Users are usually fine seeing many changes on the screen quickly because it gives the impression that stuff is happening on the background
This gave me a chuckle because I personally hate things like watching the browser jump through 50+ redirects when logging into a website.
jbstackreply
If your app absolutely must rely on the cloud for every one of its interactions, then fine. If not, you're just applying band-aids to a problem of your own making. Many apps could easily be local only, or local first. If you're not constantly accessing the network for information which could be stored locally, then you don't need to hide your app's slowness behind animations.
duskdozerreply
I don't doubt that was an original justification, but most of what I see are not for this purpose. Most of the time they're just adding unnecessary delay and CPU cycles.
otterleyreply
It isn’t, but at the same time, smart hackers were working with highly constrained PC hardware in the 1980s and early 1990s and were cranking surprisingly good performance out of it. Folklore.org has plenty of stories about it, and John Carmack’s early career history is very impressive. We mustn’t forget the demo scene hackers either.
energy123reply
Can't they use ML to predict where I'm going to click, and pre-cache the predicted page whenever the predicted button doesn't mutate important state? Or skip the difficult ML and have some basic rule of thumb that pre-caches frequent button clicks, using a markov chain, and conditioned on those pages being low bandwidth to pre-load.
__MatrixMan__reply
If we stored the edges (links) and nodes (pages) separately, rather than requiring you to blindly run a node's code just to discover what its edges might be, then you could skip the prediction and instead pre-cache the next hop for all edges just in case you follow one. You could even do this to two or three hops.
This might seem wasteful, but if the web were content addressed instead of server addressed you could then be serving that cache to your municipality even after it became disconnected from the rest of the internet. Which sort of recasts it not like wastefulness but instead like fault tolerance and preparedness.
We could maybe even dispense with the servers entirely.
There are so many different ways to build a web. Why does it feel like we've landed on the worst possible one?
Onavoreply
> If we stored the edges (links) and nodes (pages) separately, rather than requiring you to blindly run a node's code just to discover what its edges might be, then you could skip the prediction and instead pre-cache the next hop for all edges just in case you follow one. You could even do this to two or three hops.
Welcome to Next.js
charcircuitreply
>but if the web were content addressed instead of server addressed you could then be serving that cache to your municipality even after it became disconnected from the rest of the internet.
This is already possible without content addressing with CDNs. They can serve content from a local cache even when the host is disconnected from the internet.
supriyo-biswasreply
> pre-caches frequent button clicks, using a markov chain
I don't think the current crop of fullstack engineers would be hard pressed to know what a "markov chain" is, but in theory yes, you could emit a bunch of speculation rules[1] based on your predictions.
I should also say that markov chain based approaches have been used for fraud detection, e.g. identifying checkout anomalies by detecting the sequence of web pages that they clicked on, amongst other factors.
[1] https://developer.mozilla.org/en-US/docs/Web/API/Speculation...
CalRobertreply
Heh, this is how browser accelerators from the dial up era worked
jcelerierreply
I have a new laptop with a rtx 5090. Opening any GL context takes more than half a second. There's tons of things that can be optimized and are pretty far from web.
userbinatorreply
We can only hope that the company building the hardware on which this new age of AI is based on soon starts "vibe-optimising" their own driver stack.
exe34reply
They don't want other companies to train on their IP and regurgitate it to random people.
genxyreply
You should profile that, it is probably hitting the registry, the disk and maybe the network.
Try turning off wifi and see if it improves.
fc417fc802reply
I assume you're running proprietary drivers? Because I've never experienced anything like that on mesa. Launching an app that opens a window with a gl or vk context is so fast on my almost 10 year old hardware that it's nearly imperceptible.
adrian_breply
There must be some quirk of whatever combination of software packages are installed there, but the proprietary drivers are not the culprit, at least not alone (i.e. there could be some interaction with other software packages with which I have little experience, like Gnome).
I have been using the proprietary NVIDIA drivers for more than 2 decades on various hardware, both desktops and laptops, mostly with Gentoo Linux.
Opening an OpenGL context or any other OpenGL operations have always been instant.
