For one codebase I work in, LLM written code is noticeably and measurably (we have literally measured it for bugs, performance, etc.) better than what existed before. So the LLM code is a welcome change.
Do you measure anything about complexity or brevity? I'm currently working on importing some LLM code and it's 5-10x bigger than it should be for the actual needs.
One of my open questions is how much that increases over time. It could be that is a constant. But it also could be that, say, coding agents will infer false needs from the excess code/docs and elaborate further.
I've only personally utilized it for code _review_ and always ignored the suggested result code instead to understand the source of the issue and try to arrive at a resolution I find satisfactory for my mental model of the issue and solutions I'd prefer.
My take is that the main difference is the approach to problems the coding agents have. They optimize towards presuming a fully working invocation environment, yet checking everything anyway, and then rolling back any changes and re-testing that such changes worked. Which is great for normal types of software but incredibly tedious for anything aimed at less than a fully hands free automated environment.
In other words, AI is very awesome at scope creep of assigned problems and targeting the validation prompts baked into the review system.
I find this shocking though not unbelievable. Could you share how you measure this accurately? I'm interested in integrating such measurements into our services as well.
We’ve been tracking performance and bugs for years. Including commits those bugs were introduced in. So when LLM-generated code started working its way into our codebases, we have the before and after. And even comparing human generated code today with LLM-generated code today.
I find it unbelievable. I've seen the code LLMs write and it sucks compared to what a typical human produces. The only way an LLM is doing better than human programmers is if your human programmers were producing really terrible work.
I personally find it unbelievable that you have access to all of the public GitHub projects available and still think the typical developer writes good code.
What exactly do you mean when you say llm generated code? Are people prompting llms for changes and features without reviewing the code or iterating on it and then comparing that to what human writes? Because if so it's not surprising that you're getting worse results. Humans also write code through iteration. You can definitely get llms to write good code by enforcing guardrails and constraints through tooling and agent.md, and iterative reviews to nudge towards what you want. The first pass will look nothing like the committed code. I don't expect the llm to one shot anything.
Believe it or not, most programmers by definition are average. Hence, producing code better than them is not a hard feat to achieve for today's models.
You need a better harness. LLMs are significantly better at this than humans. Add it as a step in your LLM process and your documentation will always be up to date.
It encodes context it needs into doc comments and regular comments that make them in insufferable to read and so verbose because the comments contain info on other parts of the code base (that might have changed since that comment was written) but were useful to the model for its implementation. I have so far found no way to stop claude from doing this. It will sometimes do it with hand holding but the moment a task takes a slightly large amount of context its back to the word barf.
This is certainly a problem with out-of-the-box usage. But easily overcome.
Spend time updating your harness. This is not a hard problem to solve and there are many helpful guides out there.
We basically took our documented (human) SDLC process and applied it to the relevant harness hooks. Since our SDLC docs talked about what good comments looked like and what to avoid, we basically had the direction for the tools. Opus 5 did throw us a loop and increase the verbosity and decrease usefulness of the text, but refining our documentation cleared that hurdle within about a day.
'ai slop' is hard for me to complain about when it's better than 95% of code prior. The issue is more people trusting it too much to go too far, and you end up with lots of extraneous bullcrap you don't need. Doing too much and too big of a change is still a problem in the agentic world.
1. It’s so much easier using RPis for that work. It can be done with no soldering at all and there are tons of easy to use libraries. Not having to solder is a huge plus IMO and I have a workstation with iron, hot plate, air gun, etc. ready and waiting.
2. Why hate on doing it Python? Even ESP32 embedded development ecosystem treats Python well with CircuitPython and MicroPython.
3. At least they are making something!
4. Even a simple ESP32 is overkill for what you described, so why are you hating on people using an RPi when your suggestion is just as laughable when taking your hardcore embedded stance?
As someone who has been working on embedded work since RadioShack was a good source of components - let’s not gatekeep makers.
1: Not sure what your point here, if you dont want soldering, just use jumper wire, it is not exclusive on rpi.
2: What i mean is the main reason they use Rpi because it can run Python(at least easy, this was before CP/MP mature). If they had spent some effort in C, they can get away with not using rpi.
3: I agree with that, but i really do wish they step out of their comfort zone a little more discovering what is possible.
4: I dont think esp32 is overkill, it's the cheapest and easy to acquire wifi MCU, if you have better suggestion i would like to learn, there is esp8266, but the cost different is not that much.
Dude, just be nice and let people have fun. Raspberry Pi’s have made it possible for people to do things they never otherwise would have done! It is pretty amazing!
Interesting how all of the replies so far to your comment assume we’d just release the existing animals into the wild instead of just not breeding more.
As one might guess, the "just" there is doing a lot of work.
Common pets reproduce just fine without our intervention - in practice it requires costly maintenance programs just to keep their population under control (not to mention the massive and ongoing eugenics program required to produce only the desirable breeds/attributes).
If your neighbourhood isn't overrun with stray cats and dogs, that's because someone is funding a spay/neuter program, and someone is funding animal control to round up (and ultimately euthanise) the rest.
All animal husbandry systems have to be maintained via regular culling, and "eliminating" pets would require orders of magnitude more killing
I always assumed the "enslaved animal-free" future would be something that came on gradually, likely motivated by economics (i.e. lab-grown meat becoming cheaper and higher quality than farmed, robotic pets being cheaper and more "trouble-free", etc).
I never felt like anybody seriously thought we'd release domestic animals into the wild to fend for themselves-- that's ludicrous.
The answer is so that they can charge different prices per credit. If you buy low amounts, they can charge one price. If you buy in bulk, they can offer a discount. The usage is the same, but they can differentiate price per usage to give people more a favorable price if they are better customers.
It's just a MEMS mic, a battery, and an ESP32, very simple but it works amazingly well. I wrote a companion Android app for it and it works extremely reliably!
I'm going to make it 40% smaller when the small battery arrives! I really have to write the article, but I've been working on my bot all day, which is becoming extremely amazing.
Don’t worry. This advice has been going around for much more than 2 months, including links posted here as well as official advice from the major companies (OpenAI and Anthropic) themselves. The tools literally have had plan mode as a first class feature.
So you probably wouldn’t have any clout anyways, like all of the other blog posts.
>what's the value of paying someone for a product like this vs just building it myself?
Same thing it’s always been. Convenience. Including maintenance costs.
AI tools have lowered the bar significantly on some things, so SaaS offerings will need to be better, but I don’t want to reinvent every wheel. I want to build the thing I want to build, not the things to build that.
Just like I go to restaurants instead of making every meal myself.
Right, but paying for food at a restaurant doesn't get me any closer to owning a restaurant. If promises are delivered on for these agentic coding platforms (which I do believe in), it seems the most reasonable path forward is to use those platforms to build your own platform.
I lived in such a place and never had power outages. Mostly because the power company came through on a regular basis (two years or so) and chopped down and trees that could cause problems. Some areas definitely looked terrible from a beauty standpoint, but it meant keeping power.
For one codebase I work in, LLM written code is noticeably and measurably (we have literally measured it for bugs, performance, etc.) better than what existed before. So the LLM code is a welcome change.
reply