I'm not familiar with the world of academic publishing, so I want to ask: how is the industry making sure that submissions aren't at least partially AI-generated?
Is it standard practice for authors to have to defend their submissions via interview like this? If not, why not?
Does the vetting process vary with the quality of the publisher?
As an outsider, it's extremely worrying that anyone would even attempt to submit an AI-generated paper for publication in an academic journal. At that level I would have assumed literally everybody should know better than to even try.
> Is it standard practice for authors to have to defend their submissions via interview like this? If not, why not?
It isn't, but maybe it should be. For post-grad qualifications oral defense is standard, and I didn't mind defending my central thesis then, and won't mind now.
Not in a direct interview style, but most us conferences can request additional information or feedback. If they conditionally accept or reject a paper, that conditional relies on feedback from the author(s).
Interviews like this are interesting, but in no way can scale to the infinite paper slop conferences are facing.
> Not in a direct interview style, but most us conferences can request additional information or feedback. If they conditionally accept or reject a paper, that conditional relies on feedback from the author(s).
Requesting feedback is useless, as the article points out - the "authors" could not answer basic questions during the interview, but after the interview were able to send full explanations to the interviewer.
If you have indirect feedback ("please answer these questions we have") the "author" will simply feed it into an LLM and send the results back. You need to get the author to do an oral defense to verify that they wrote the paper.
This is the main problem with AI generated output, whether it's a research paper, a blog, an email, a comment on a forum, similar: the value in knowing that a human wrote $X sends a signal - that the human understands what it is they wrote, even if they misunderstand the concepts.
When you get a message from someone who is a "I only used an LLM to clean it up, the thoughts are all mine"[1] person, you cannot engage with them, because they may not understand the message they transmitted, and so any human engaging with them is only burning their own time for no gain.
When you get a message from a real person, you get not only the message, you also get a signal about their understanding. That signal is missing in AI generated messages.
“Requesting feedback is useless, as the article points out”, I would agree post llm availability. Before that (~pre 2023 I suppose), this wouldn’t have been as large of a concern.
“You need to get the author to do an oral defense to verify that they wrote the paper.” While I agree with this point, it can’t scale. As with the indirect response, vishing and likely video mimicry are only going to be easier with the next llm release. This leaves requiring each author who submitted a paper to fly to a place just to defend a paper in person (assuming all volunteer reviewers are in the same place).
“Are we prepared to sacrifice everything on the altar of scalability?” If the performance of verification isn’t great, it won’t get adopted. Just general acceptance timelines, look at how long it took for Python 2->3, https 1.2, us real-ids. You can’t adopt a significant change immediately and expect things to work immediately.
“Universities already have the space to do this, for potentially hundreds of defenders per day.” Universities have space for their own students while they’re still funded for the time, but conferences aren’t tied to universities. Even so if they were, from the author “The fraction of desk rejected papers at TMLR used to be about 6% in 2023 but is now at about 53%.” universities would have to raise funds for this massive increase of slop review. Us conferences uses unpaid volunteers (most part) from those areas for reviews.
If an independent conference were to hold in-person interviews (and maybe fly people out) for 1000s of submissions, where would they hold it and how would they pay for it?
> I only used an LLM to clean it up, the thoughts are all mine"
I mean, it seems to me like that should be nearly everybody at this point, right? There are at least some elements of using LLMs that are the equivalent of a spell check, like asking to make sure that links work and that references are to the thing they're supposed to be, and so on. I feel like the only difference at this point is between people who do that and say that they do, and people who do that and don't say that they do.
> I feel like the only difference at this point is between people who do that and say that they do, and people who do that and don't say that they do.
No. Did you "clean this up" with an LLM before posting it?
I hate to go there, but how is that argument different from a thief who argues "Everyone steals. The only difference is some of them claim they do, and some claim they don't"?
> I mean, it seems to me like that should be nearly everybody at this point, right? There are at least some elements of using LLMs that are the equivalent of a spell check, like asking to make sure that links work and that references are to the thing they're supposed to be, and so on.
Right. Maybe. The problem is that when material has all the tells of LLM generation, do you expect the reader to figure out if the thoughts were the authors or introduced by the LLM?
The value in a human directly communicating with you is that you understand what they are saying; if they misunderstand, you spot their misunderstanding. If they are talking at cross-purposes, you know you are arguing past each other. If they agree with you, you know they agree with you.
That signal is missing when the communication is LLM generated; you message said "Not X, Not Y, just Z", but I can't tell if you even know what X, Y and Z are. f I ask you to give me a definition for them so I can ensure we aren't talking about different things, the LLM response will be "X is $SOMETHING", but I still can't tell if you think that X is $SOMETHING or if you still think that X is $SOMETHING_ELSE while your LLM and I agree that X is $SOMETHING.
Communication from a person tells us something about that person, outside of the message being communicated. If that communication is relayed via an non-invested third party, the missing signal prevents any communication.
You've nailed my thoughts on that aspect pretty well. You demonstrate why when we choose communicate with a person, we definitely want a person at the other end of the line.
But also: When we want to ask a question of a bot and then get an answer from that bot, then we'd have already made a deliberate and rational decision to ask a bot ourselves.
We not need nor want anyone's help on that front; this kind of low-effort help results in an insulting waste of time.
Abject silence would be an improvement in communication quality over having an unwanted third-party conversation with a bot.
I’ve only published a few papers, but this interview sounds extremely unusual to me (I mean, it is clearly a special thing that the editor is doing, which is fine). I wouldn’t do something unethical, but if I had and the editor asked me for an interview like this, I’d know I’d probably been caught.
Why is it unusual: it sounds extremely time-consuming.
As to how worrying AI-generated papers are… it sounds more like a headache for the editors really.
In general, journals don’t have to be perfect; mostly researchers read research papers. You already have to read critically (publish-or-perish has been a thing for a while, so there are plenty of not-so-great papers out there). Peer review is just the “entry” barrier, science is a social process and papers become more or less influential based on a fuzzy process of citation, conference talks, and peer-to-peer suggestions.
For whom? Surely the authors can find an hour after submitting the paper to a journal?
Beware that a reviewer easily spend a full week on reviewing a paper, and there are typically three of them. So if one hour of conversation can save three weeks work, it sounds worth it.
I was thinking for the reviewer. Also note that I was only answering as to why it wasn’t done in the past.
For the time comparison, I’m not sure, it doesn’t seem quite apples-to-apples:
1) It is scheduled time vs unscheduled.
2) There must be some base rate consideration… the papers being discussed here were planned to be desk-rejected.
Maybe it could be a good process for saving papers that were going to be desk-rejected, but I dunno, that seems like it’d just lower the quality standards.
Honestly, I was sloppily thinking of the general reviewer/editor system as one thing (and didn’t notice that I’d lumped them together until you brought it up here). Good point. Although the editor has a lot more papers to handle, so I think they are probably busy as well.
> "Endeladyze said she felt terrible that she had upset her customers, but she was also frustrated at the idea that she should have been paying a real artist. Making these signs was a job only she had done herself or with employees in the past, never hiring outsiders."
This seems like the wrong attitude to take after experiencing such a backlash. If she'd made the signs herself for years then why resort to using AI to produce what is so obviously a generic slop result?
The article says she paid $150 for printing and $200 for a stand. Why not throw another $200 towards an artist?
Basic economic theory states that only holds if the goods / services being provided are equivalent in the eyes of consumers. It seems self evident here that consumers do not consider them equivalent.
You said that the AI industry feels they're replaceable. I'm not saying that AI does equivalent work, I'm saying that AI companies don't choose the prices for art, they just play in the market.
It's multiple records per user over time. See something like posthog. One interaction with a form can generate 10-15 events. eg enter page, click link with text "settings", change input, clicked input, etc. So active users can generate 15-150 events per session, each ~1kb to 2kb in size.
The reason to store that data is to be able to build analytics and funnels you didn't anticipate and construct them retrospectively. You can of course save massive amounts of storage by building rollups and aggregating, but that prevents you from being able to eg change the funnels and see the funnel in the past.
10M users × 100 events/day × 1KB ≈ 1TB/day. A couple years gets you into PB range.
The other obvious thing is logs. Being able to have a vulnerability then go back in time and see if you were exploited is super valuable. See eg log4shell: if you kept records, you probably could go back and see if you'd been exploited.
I wholeheartedly agree on every point. I've felt this way for years. It's scandalous.
Many companies also rely on socials as their only effective means of customer service. If you have a problem or want to provide feedback but you don't use social media then you are SOL.
...because the overhead of even small claims is bigger than what you can possibly recover in most regular disputes. There needs to be something like a class action for small claims - government regulators where you can submit complaints with evidence to and if enough people complain about a company the government has to look into it.
Isn't small claims designed for, well, small claims and therefore low overhead? That's why you can't bring a lawyer. Which is super annoying for a company, because a company has no physical embodiment and has to send a lawyer or other representative just by virtue of being a company.
Most of the time when people try to define healthy food vs. junk food in a fair and unbiased way, the line gets drawn at "whole foods" vs "processed foods".
A typical burger meal is definitely processed food because the meat is processed mechanically and mixed from different sources (usually with added fat); the bun is made from refined flour and refined oils; and the fries are thinly-cut potatoes blanched and fried in more refined oils. (Not to mention a whole bunch of chemical additives which are the norm rather than the exception; and "hyperpalatability" tweaks which means the food is formulated to taste unnaturally good.)
You could perhaps argue that a home-made burger meal made using whole cuts of meat, a lean wholemeal bread dough, and potatoes roasted in animal fat could count as a healthy/whole food meal, but it would taste different to the typical hamburger meal we all associate with fast food.
Fast food also has an enormous marketing industry behind it which pushes it in unhealthy volume, with maximum convenience, and at (until recently, at least) affordable prices.
Why does grinding the meat at home make a huge difference vs grinding it in a factory? Depending on the cut you choose, the condiments you add, and the portion size, a hamburger at home could be Lot less healthy than a McDonald’s burger. McDonald’s patties don’t have any fillers afaict.
The point is that “junk food” is a pretty amorphous and vague category. If you want to enforce portion or calorie restrictions you can do that, but it would have to be way more encompassing of a regulation than targeting “junk food”
The additives to the meat. Has to retain its color, be preserved, and they also add pink slime to it. McDonald’s adds soy, I believe, and there are all kinds of other things that go into it. If you grind your meat at home you’re going to get some delicious burger.
As of the last 10 years none of this is true for McDonalds. Their published ingredients say 100% beef with no additives, and they stopped using pink slime 10 years ago.
But we’d still designate McDonald’s as junk food, so it can’t be the additives that make the category
I don't think there's a neat categorisation. If I cook at home, starting with just vegetables and cuts of meat, at some point the food I'm making stops being "whole foods" and starts being "processed foods". But when is that?
If I cook bread at home, is that a processed food? How about if I make a steak sandwich with that bread? How about if I first pass the steak through a mincing machine? Still whole? Or processed now?
What if I add an egg, some crumbs from the bread I made earlier, and some salt and pepper. Still whole? And then I formed it into a patty and cooked it on a pan. Is it now processed?
When does it become processed and how does that look different to my stomach when I chew it?
Apple will never, ever add anything even close to that kind of functionality. This is a company which took SEVENTEEN YEARS before it allowed iPhone users to have gaps between app icons on their home screens.
If you have ANY hope of Apple letting you have enough control over your device to visually block ads in apps then you are stratospherically shit outta luck.
Is it standard practice for authors to have to defend their submissions via interview like this? If not, why not?
Does the vetting process vary with the quality of the publisher?
As an outsider, it's extremely worrying that anyone would even attempt to submit an AI-generated paper for publication in an academic journal. At that level I would have assumed literally everybody should know better than to even try.
reply