The underlying assumption here is -- at least part of code is AI generated.
Also another unstated assumption is that both the "author" and "reviewer" are reading AI-generated stuff, at least partly
First of all -- the author should not be surprised by any of the things the AI is reporting. If the author is -- either the reviewer AI went wrong, or the generative component went wrong. Either way -- there is something worth looking at in greater depth.
For the reviewer -- or "traditional reviewer" as we may term them -- they are dealing with much greater volume per day now, again due to AI generation. This is supposed to help them get through things faster while not losing comprehension altogether.
There's a reason we still have the code diff right below -- they are encouraged to check things themselves if they feel "something is off" with the quiz.
The point is -- a few questions hopefully gets the wheels spinning in the reader rather than looking at the diff passively and saying "LGTM"
And yes -- all the review results are grounded strictly in the diff
Higher-level design often involves dissolving apparent tensions or opposites through layering or more sophisticated mechanisms, while preserving the benefits of both.
Example ->
Proposition: "Phew, we have to choose between convenience and security."
Naive design: "Let's find a watered-down compromise."
Good design: "Let's layer security on top of convenience by designing the system so users don't have to choose, or water either one down."
Great companies have a history of dissolving apparent contradictions under a single roof via good design.
Some might actually say that they found "a perfect compromise" where both use-cases are served well — this seems to be very much a linguistic preference today, regardless of the fact that "compromise" originally came from "making concessions".
Why is it bad? 90% thought is not productive or low value (distractions, routine work, etc, etc). You may think something else, something more useful, and with the full power of "thoughtware" backing you.
Just a question to people who may know better than me about this.
I thought the whole point of trying to write out TLA+ is so that you get a better idea of what you want and put it into formal language?
I get that an LLM can assist/help with expressing what we want in formal language a bit, but if one automates all this there is no human intent/design anymore.
If the LLM generates both the design (TLA+) and writes an arbitrary program that satisfies said design -- what exactly have we proved?
What assurance do humans get since human doesn't know or cannot specify what they want.
An LLM-generated TLA+ model can be verified for certain things in a way that LLM-generated code can't. It's infamously hard to exhaustively unit-test concurrency.
Whether or not you're modeling the right things or verifying the right things, of course... that's always left as an exercise for the user. ;)
(How to prove the implementation code is guaranteed to match the spec is a trick I haven't seen generalized yet, either, too.)
If you read the article carefully -- I've dealt with an alternative scenario as well -- where we may have smaller codebases with larger blast radius.
As to disposable software, it's harder to get traction/adaption when things constantly break or are slow or the experience is crappy in general.
To make it simpler - all else being equal - as a user would you prefer using highly reviewed/vetted/reliable software, or otherwise?
My bet is reliability is an invariant -- nobody wishes for software that crashes, leaks your private info, gives faulty output, is laggy to use and so on.
Specification languages need big investments essentially - both in technical and educational terms.
Consider something like TLA+. How can we make things such as that - be useful in an LLM orchestration framework, be human friendly - that'd be the question I ask.
So the developer will verify just the spec, and let the LLM match against it in a tougher way than it is possible to do now.
Also another unstated assumption is that both the "author" and "reviewer" are reading AI-generated stuff, at least partly
First of all -- the author should not be surprised by any of the things the AI is reporting. If the author is -- either the reviewer AI went wrong, or the generative component went wrong. Either way -- there is something worth looking at in greater depth.
For the reviewer -- or "traditional reviewer" as we may term them -- they are dealing with much greater volume per day now, again due to AI generation. This is supposed to help them get through things faster while not losing comprehension altogether.
There's a reason we still have the code diff right below -- they are encouraged to check things themselves if they feel "something is off" with the quiz.
The point is -- a few questions hopefully gets the wheels spinning in the reader rather than looking at the diff passively and saying "LGTM"
And yes -- all the review results are grounded strictly in the diff