The broken world of publishing scientific papers

Peer review is dead! Long live peer-review!

Centuries-old libraries, with ornate shelves, beautifully laden with exquisite books, of all sizes and shapes. Sacred, printed knowledge, met by curious and intelligent eyes, contemplating them in solemn silence. That used to be the world of old.

Between the 16th and the 20th century, scientific publications were published in either book or magazine format. The whole process was treated as a supreme intellectual exercise, as well as a careful editing one. A researcher would submit a paper for appreciation, which would be thoroughly reviewed by peers (subject matter experts), requesting changes and edits where necessary, and formulating an opinion of whether that specific work of science was worthy of publication. And then the editors, making final decisions, curating content to construct an issue that would be both appealing to the desired audience, and worthy of being published in that specific journal. At the end of the process, the idea was that a published paper would carry value, and that it could be trusted - presumably because it was subject to peer scrutiny.

By and large, with glamorous examples of fraud, this process has served the purpose of diffusing scientific knowledge to all corners of the world. But not all was created equally, and the game was never really fair. That, however, is a subject for another day (and I will get there, it’s one of my pet peeves).

With the advent of the internet, the world started spinning faster and faster. Knowledge dissemination became instant, for content published digitally. This resulted in the obsoletion of the old order, where the cadency of journal issues, and their specific focus, would dictate the waves and tides of publications. The digitisation of scientific journals also removed a lot of barriers to entry - it was no longer necessary to bear the burden of publishing physical assets that needed to be sold. With that, came a profusion of journals, of all shapes and sizes (and of various reputations). But it also brought a lot more potential readers.

The natural move for the industry was to place papers behind a paywall. This, ultimately, led to the creation of projects like sci-hub and others. I’m not getting into the debate of intellectual property and copyrights- not really an interesting subject.

Then came “open access”, where the authors essentially subsidised the fees for everyone to be able to read the paper for free, thus maximising reach. I am tempted to say this became a lucrative way of working, even without knowing the actual unit economics behind this. Suffice to say that, over the last decade, Sword has spent more than 200k dollars in publishing fees to ensure “open access” in all our papers. Oh, and did I mention that peer-reviewers do not get paid for reviewing papers? More on that in a minute.

While all of this was happening, there was also an explosion of research papers being submitted. This was due to a combination of several factors: a) more people with higher degrees of education doing research; b) more technology available meaning experiments could be run faster; c) more journals to publish the research. This, in turn, resulted in an explosion of the needs for reviewers. You know, the ones that work for free, out of the goodness of their hearts.

The end result? A complete mess, and one where the writing was clearly on the wall.

With the exception of the very top journals, there has been, by and large, a degradation of both journal prestige and of the very act of publishing a paper. Journals are taking impossibly long to get papers through review stage (there’s only so much you can squeeze people that are working for free, out of the goodness of their heart), and that is when they can get reviewers to accept (no shit sherlock). There has also been, by and large, a degradation of review quality. If you give peanuts, you get monkeys.

Enter AI. The mess became even messier. From AI used to generate or review manuscripts before submission, to AI being used extensively to review papers during the peer-review process (actually, with interesting datapoints emerging - see here), I wonder if we are not on the verge of getting to an alternate reality where AI helps researchers write papers appealing to the AI reviewers, so that publications ultimately become an exercise of token matching. Well, it would be somewhat fitting, right? The AI reviewer is indeed the appropriate peer-reviewer for another AI.

And then, the final straw. The sheer pace of AI evolution and research, which completely shatters any previous logic.

So, how do we solve this? Well, I would say the scientific community has already taken several steps in that regard.

First, people started publishing pre-prints of their papers on platforms like ArXiv, Research Square, etc. And here’s a very interesting finding. A recent paper (published in pre-print format in bioRxiv, looked at every bioRxiv preprint posted between 2018 and 2025 that could be matched by DOI to a published version — 72,644 pairs — and used an LLM to classify each abstract for content change and hedging shifts. They found that the primary claim was unchanged in about 40% of abstracts, minorly revised in 50%, and substantially revised in only about 10%. Makes you wonder, right?

Second, and especially in AI research, the place to publish was replaced by the palace to be. The important thing is now to have your research accepted in the largest and best conferences. This is an interesting case, because it serves both as a forcing function for researchers to submit within certain deadlines, and as a forcing function for the peer-review process to occur under strict timelines too, given the conference has a set date. Furthermore, there is so much money being poured into AI that the business of running a large AI conference is not really THE business. Will that change in the future? Who knows.

Finally, there have been experiments where (behold!) journals decided to pay reviewers a fair fee for their work. How kind of them, right? And guess what happened - review time drastically reduced, and review quality maintained (see here). Isn’t it wonderful when industries experiment what can happen when, instead of concentrating on extracting as much value as they can from others, actually find ways where it can be a win-win situation for everyone? I posit that this is the only way that journals will be able to survive (and even thrive). Or at least the good ones. The others I believe are at risk of oblivion.

Top journals can charge more for research to be published there. This results in more money that can be paid to secure the most reputed reviewers, who deserve to be paid by their time. The one thing researchers want is for their papers to be reviewed by someone who is knowledgeable and fair. We’ve all been there, and the worst thing is a bad reviewer. This, in turn, leads to better throughput, thus preserving trust and reputation, for both the researchers and the journals. Its not too late.

But if nothing changes, I believe it will not be long before platforms like ArXiv, Research Square and others figure out a way to ensure proper peer-review inside the platforms themselves, sidestepping the whole journal industry.