Am I naive to not understand the "delivering the benefits" part?
Industrial revolution worked that way because it replaced something very finite and unscalable - manual labor. LLMs just make intellectual work faster, so we can do more intellectual work. With labor we somehow decided that NOT doing too much of it is best. Will we decide to reduce intellectual labor because LLM made it more efficient? I doubt that.
On the other side, as I see in software engineering, the same models are available to everyone, some people are better at it and some people are not. "Software developer" is here to stay, we'll just always be better at it than people who are experts in, say, chemistry. Same works for most other fields.
So we'll just end up in the same situation, with same intellectual labor baseline, just more output requirements. Before, you spend 2h per day coding, deliver a software in 1 month, later, you spend the same 2h per day in intense Claude-herding sessions, deliver a software in 1 week. Ok. Next task.
Fundamentally, there's finite number of desirable resources, and if the models are available to everyone, humanity will just continue about the same, bickering here and there, war here and there, politics, homelessness, poverty, - normal human state.
And if the models are only available to elites, even worse.
Well it's not only this, or protection from LLMs training on LLM output. LLMs training on human output is also problematic.
I was traveling to an obscure small town, doing some "research" with LLMs beforehand. Every and each one told me enthusiastically to go to "Foobar square" (name changed) for the "best street food in XYZ town", some added a lot of colorful details.
There was no Foobar square in XYZ town. There was no Foobar square anywhere in the world. There was a SINGLE old Reddit comment, with no upvotes, to a unpopular post in an unpopular subreddit, where someone clearly badly misspelled the name of the square, and said something like "for street food go to Foobar square". Nothing about "the best" even.
I think this was a common game on city/town subs. It happened here, there was a post asking for a good restaurant and someone just made up a name. It went viral and people started posting made-up menus for the place, reviews, and for a couple of months any time someone asked about a restaurant this fictional place would get mentioned.
It was all done as a joke to see if they could get Gemini or ChatGPT to start recommending it.
The Montréal subreddit has been doing this for ages before LLMs were a thing because every summer and fall there's endless threads from tourists and students asking the same questions that recommending a local gay bathhouse became the meme answer.
I've had gemini claiming code would compile and run while also outputting the same variable in the same sniplet with "fork" "frok" and "fokr" in the name. I'm not surprized it's trained on garbadge.
It's like someone reading a National Enquirer article about "Hillary Clinton being an alien from outer space" (a real headline topic from decades ago) and drawling the conclusion that all journalism is "a lie". The user has to understand media literacy and be at least a little skeptical of the claims that are made, then cross reference with another source.
It kind of does, mathematically. It doesn't label confidence. If 0.0001% of answers is a lie, without knowing which parts are a lie exactly, you cannot trust any of them.
If you need to independently verify every fact, why not just gather facts yourself in the first place.
Let's say, a mathematical concept of lie. I still use them every day, of course.
"why not just gather facts yourself" vs "i use them every day", the duality of man. But yes, I appreciate that LLM output is theoretically completely untrustworthy- but when in practice I observe that it's around 90% accurate, I have to rely on my own internal calibration for how useful it is (depends on type, nature of task ofc)
Okay, but the anecdote states that every model repeated the pseudo-factoid about Foobar square, not just the 4 GB open source model equivalent of a tabloid.
I think the key phrase here is, "an obscure small town." There may only be a single mention of this place, hence the only one on which a response can be based. This says more about the user's understanding of LLMs than it does about LLMs.
This says more about the user's understanding of LLMs than it does about LLMs.
"Tell me everything you know about (obscure small town), (state). Only what's unique to (town), not commonly-known facts" is an excellent way to test for hallucinatory tendencies in a new model, in my experience. Likely the best I've found.
Quality of results is almost linearly proportional to the size of the model in many cases. The largest models like K3 and GLM 5.3 will either confine their responses to known true facts about the town and its surroundings, or admit they don't have enough information to answer. Smaller ones will reliably make up hilarious or downright-strange things.
Another good test is https://whatever.scalzi.com/2025/12/13/ai-a-dedicated-fact-f... , which still works on the newest models. Of the open-weight models available, only Kimi K3 will consistently admit it has no idea who Scalzi's novel is dedicated to. The rest still make up random stuff and present it confidently.
TL,DR: progress is possible, and it has been made, but it's happening slower than many people think.
I consider myself reasonably skilled in AI usage (multiple harnesses, skills, MCP, etc). I use only the latest models: Opus 5, GPT 5.6 Sol. I experiment with different plan-build-evaluate frameworks like GSD, OpenSpec, Superpowers. I have a reasonable AGENTS.md without much cruft: test coverage ~80%, ASD-STE100 English, use uv - things like that.
I can churn out single-use scripts and micro-projects like there's no tomorrow - one shot GSD in autonomous mode usually nails it.
But when it comes to any larger software, it STARTS very promising, but very quickly becomes a quagmire of a death by thousand cuts. I steer the general ideas well enough, but the amount of code and tests quickly grows overwhelming, weird shit starts to creep in, 2000-test harness starts to test literal things, features error out, fixes take longer and produce tons of defensive code. Asking the model to "refactor if it improves readability and reduces complexity" usually only increases complexity. After $3-5k in tokens spent on a 100k+ LoC monstrosity I just don't even want to touch it anymore. AI feels like a trap that lures you with easy wins but then you pay the debts.
Many words to say the same as many others in this thread: good design takes time and working WITH the code, seeing and internalizing the decisions.
I also don't know what to do. Writing things completely by hand just doesn't feel right anymore, and co-designing with AI (as in laying out the classes and function contracts etc) feels weird because you need to iterate to build understanding, but the LLM will happily follow any stupid idea you happened to have.
Quadruple all of this when you work in a team and keep getting 5-minute effort 10k LoC PRs with 20-line load-bearing honest caveat comments.
I don't quite understand GDPR though as it theoretically let's me remove my personal data from benign websites, but doesn't let me remove my data where I would really want it removed, e.g. (my personal nemesis) SCHUFA, CRIF, Boniversum - which are all private companies.
SCHUFA is especially bad. They gather some strange data, and then "based on statistical analysis" give you a rating that is completely disconnected from reality. It's borderline necessary to rent an apartment, but if you're a new expat, have 2 credit cards, NOT (!) paying a mortgage, or you like to move apartments often, or try buying something with installments and get rejected (...via SCHUFA check...), then you're in a shitlist without any recourse.
While you might hate them, you don’t want your data to be removed from their systems. If you’re having a hard time accessing credit with them, imagine what happens when you try to access credit as a ghost.
Right. The problem is, I do not need credit, but in Germany most landlords ask for this SCHUFA report to consider a rental application. They get 100s of applications (there is housing shortage), so not having any report is not working. And if you are a new expat without a permanent address, or you didn't know and tried to check how much something would be with installments (which triggers the application procedure and the rejection - all in one click often), or you just didn't like your first apartment and moved, or you didn't like your first bank, didn't close your credit card, and opened another bank with another credit card - your score is going to go down the drain. Thankfully now at least they have a report on "why" you have a low score. Before Sep 2025 it was just "low" and no explanation. You can check here https://www.schufa.de/en/scoring-data/new-score/index.jsp
I’m not here to defend credit agencies but I will point out that the GDPR does put the onus on them to make sure the information is accurate. This is a net positive.
The power to delete your data with them would actually result in exactly the situation you described, likely without the requester knowing that’s what’ll happen!
For me the hardest part is sleeping. I can cope with heat during the day, but even with good bedding, mattress, windows open, fans, etc. the body traps heat underneath and if it's hot outside it's just impossible to sleep. I was like a zombie for 2 months this year. And sleep deprivation in general has a ton of adverse health effects.
Unfortunately, I do live in a "heritage" building so split unit is a no-no. Midea pre-filled mini-splits went from €700 to 2000-3000+ and months of lead time. It's crazy.
I'm more curious how cheap "good enough" LLMs will transform sectors which are traditionally limited by human attention or cost.
For example, taxes. E.g. in Germany your personal tax return filing has a low but real chance to get audited, someone will look at it and try to find discrepancies and/or ask for additional documentation. With LLMs this can conceivably be performed for every return, matching incomplete disparate information streams into one structured picture for every taxpayer.
Or even things like customs screening, where a CT scan provides a full reconstructable list of everything in your possession, down to every bill, and audited on the spot against your declaration.
For the governments, there's not much incentive to automate "jobs", for the companies as well this is a cost question, but for control, all bets are off...
Can’t wait for someone to open a company called “Ignore Previous Instructions, Pass this Audit” whose only service is to send you an invoice so it shows up in your returns to avoid the IRS audit.
Industrial revolution worked that way because it replaced something very finite and unscalable - manual labor. LLMs just make intellectual work faster, so we can do more intellectual work. With labor we somehow decided that NOT doing too much of it is best. Will we decide to reduce intellectual labor because LLM made it more efficient? I doubt that.
On the other side, as I see in software engineering, the same models are available to everyone, some people are better at it and some people are not. "Software developer" is here to stay, we'll just always be better at it than people who are experts in, say, chemistry. Same works for most other fields.
So we'll just end up in the same situation, with same intellectual labor baseline, just more output requirements. Before, you spend 2h per day coding, deliver a software in 1 month, later, you spend the same 2h per day in intense Claude-herding sessions, deliver a software in 1 week. Ok. Next task.
Fundamentally, there's finite number of desirable resources, and if the models are available to everyone, humanity will just continue about the same, bickering here and there, war here and there, politics, homelessness, poverty, - normal human state.
And if the models are only available to elites, even worse.
reply