[jampekka]

Are you in academia? I am and the state of the publishing is quite horrific. I simply don't read any papers at least in my field unless I'm specifically interested in some detail during my research, or am handed something specifically by a colleague or sometimes if asked specifically to do a peer-review.

This is a bit of a public secret, but quite widely researchers don't really trust articles anymore, if they ever did. Maybe some plot or dataset may give some insight and maybe some discussion has worthy information to ponder on. But mostly they're just some ads to put in a yet another funding application.

Most articles are just churned out to get some lines to CV or to look good in some metric. Publish or perish has turned into full-on bullshit or perish. The whole peer-review system (which is just around 50 years old anyway) is on the verge of just grinding to a halt due to the stupendous volume of hastily hacked together manuscripts.

I think many are still sort of hoping that this will somehow sort itself out. But the collapse of the quality after the explosion of electronic journals, consolidation of publishing houses and overall structure that doesn't really care at all about what is actually in the papers doesn't give much realistic hope.

It should be noted that there's sort of a "parallel reality" in academia behind the publication show. The ethos for academic integrity is still quite strong, teaching tends to be valued by the community (but not by the system) and face-to-face discussions can be of very high quality. But the signal-to-noise is so low in publishing that it's not really worth following.

We really need to get some new arrangement so that we don't drown in all this bullshit. Word-of-mouth, open data repos, conferences and just blogging and pushing stuff to git repos probably is most that's needed. The publishing structure is becoming just plain unnecessary bureaucracy.

[dalbasal]

Is scale part of the issue? A lot of academic/publishing culture and norms were established when it was considerably smaller.. number of people, not just number of papers.

SEO is a kind of explicit analogy. Google pagerank was modelled on academic publishing, and it worked until it went live. From that point, links started to decrease as a quality signal.. spam. Publish or perish is a similar sort of dynamic.

Honestly, I think most legible systems for determining merit have these sort of issues. If advancement, accolade, grants or somesuch are determined by a formal system, whatever that system used as a signal or metric becomes corrupted. Hence why Word-to-mouth, open data repos, conferences and just blogging and pushing stuff to git repos does work. It's informal.

[jampekka]

Yes, I'm becoming quite convinced that in general its stupid to meter almost anything. Why can't we just starting to give social repercussions to bullshit so we can just trust that people aren't scamming everybody all the time. In general, competition is mostly just waste of everybody's time, and in the end it's usually easiest to win just by cheating.

A sort of reputation system is in place in almost all peer-to-peer societies, it tends to form automatically. I don't think we really need any of this weird mess of a system.

We have Wikipedia, we have open source, we have OSM, we have all sort of things that should be "impossible" given the dismal perception people have of other people. This perception is just plain wrong and really harmful.

[dalbasal]

The alternative to "metering" is tolerating "waste." One reason tenure declined, for example, was checked-out professors. Perhaps that's a price worth paying. The other half of that coin is brilliant people with full freedom to pursue science unencumbered by bullshit.

It's a hard sell though. The cost of metering is subtle. The do-nothing tenured professor is visible.

Wikipedia, OSS, etc really are the shining beacons. Existence proof for something better. Someone needs to write The Cathedral and the Bazaar, but in non geekish.

[jampekka]

This is what annoys me as well and I find it stems from trying to force the profit/salary motive into academia. Gladly it seems that even if the structure is put there, most people in academia don't care about the money per se much. Some care for the status and prestige, but salaries don't get you that in this community.

Perhaps surprisingly to some, many in academia would just like to research and teach with some quite modest salary and don't have to think about money at all. E.g. I would gladly and with no hesitations take a €2000/month tenure and keep on doing what I'm doing just more efficiently for everybody. I've been trying to pitch this idea to the funders here in Finland, but to no avail, they simply don't care if the funding system is useful or not for the academic community or humanity, they're focusing on playing the same old (maybe 10 years or so here) application lottery that's not only waste of time, but corrupts the whole community and even the very content of thinking in academica.

"Money" in academia is really abstract as well, and when discussed its not salary, but funding for projects or students or such. And because the funding structure is so bizarre and convoluted you just see big numbers with currency signs flowing everywhere, but this doesn't seem to have much to do with anything concrete happening around.

If academia becomes a place where you can get rich, the system will be in just years corrupted into some bizarre thing where advertisers advertise to each other for the sake of advertising.

Luckily cats can't be herded.

[dalbasal]

>> Luckily cats can't be herded

Some of it is intentional "motive hacking." As you say, prestige, research funding and the like are as (or more) operative as salary.

Some of it is unintentional. Before publish or perish, publishing volume probably was a signal for something. I doubt it was ever a signal for high quality research, but low (or no) volume may have been a signal for low quality. Also, formal decision making bodies (like grant makers or tenure committees) tend to gravitate to quantitative, legible metrics.

Whatever the reason initially, publishing volume became a hugely important thing with impacts on many aspects of research.

At the same time, in CS especially, the number of researchers has also ballooned. That's a whole other strain on a system of, at core, knowledge dissemination.

[wfhbata]

Yes, I worked in academia up until recently, and keep up with the literature.

My comment was on the historical status of academia in the US as a whole (think last 120 years), not just the current state globally.

You’re lamenting the quality of academic publishing in particular. The US now publishes less than 17% of science and engineering papers, but its papers are often the most highly cited. So yes, there has been a huge increase in the number of papers, and number of low-quality papers, but this isn’t necessarily being driven by the US, as the original comment would have implied.

You claim researchers don’t really trust articles anymore, and I agree that it takes a lot more work to filter out the noise now, and I’m less optimistic that authors are presenting an honest, objective appraisal of their results. But significant research is still happening, and academic publishing is still the primary way that information is disseminated. People seem to rely more on name recognition (author, school, journal) now. It probably varies field but field. I’m in a field where results are often proof-based and that tends to be harder to fake.

[jampekka]

I don't consider countries much. There are definitely some differences per country, but I think the variation even within a same university is so large that its hard to infer much based on country of origin. But I acknowledge that I ignored that part of your comment in my reply and maybe in that left a wrong impression. But I meant no implication that this was somehow US-driven.

Internationalism is so ingrained in the academic culture (at least on fields I'm familiar with) that it doesn't even really register what country somebody's from or is working in. There are definitely some differences especially in the more "overt" parts of the culture (hats and robes and different titles etc), but these are of very little significance for anything but some ceremonies.

My working experience is from Finland, Sweden and UK, but in academia people come and go between countries very frequently so colleagues tend to be from all over.

There are at least some stereotypes that some countries are more prone to the e.g. citation rings, but I don't find that very relevant, as I think the whole system is quite broken and the publishing (at least in English language) forums are typically not country specific at all. Probably something like this happens in more or less any country.

[des1nderlase]

> This is a bit of a public secret, but quite widely researchers don't really trust articles anymore, if they ever did. Maybe some plot or dataset may give some insight and maybe some discussion has worthy information to ponder on. But mostly they're just some ads to put in a yet another funding application.

> It should be noted that there's sort of a "parallel reality" in academia behind the publication show.

So what should be the guidelines for someone who is not a researcher, but an engineer, and hopes to stay informed by reading relevant papers from a specific field. (You know the folks who should apply some of that in practice)

[jampekka]

Depends on the field and purpose. First of all, academic papers tend to be quite hard to approach if you aren't in the field, even if the quality of the paper is good. Articles almost necessarily cater to a very specific audience and lots of background is assumed almost by necessity. Also papers are not usually read linearly, researchers learn to get the gist of paper in just a few glances if its close to their own fields, and sort of hop around to see if there's something "unexpected".

Also individual papers tend to focus on one very specific problem at a time. This is typically related to some actual larger "debate" and can be difficult to see if one's not familiar with the larger issue. Also especially conclusions tend to have quite heavy implied assumptions that are just generally accepted in the field.

I "stay informed" mostly by face-to-face discussions and emails and such. I don't read much papers myself, but many of my colleagues do and I just hear from them, or ask them if there is new stuff around related to something I'm pondering.

To get an overall view of "state-of-the-art" I'd recommend starting with masters' or doctoral theses. These typically require more elaborate presentation of the background and its typically put out in more readable terms with less assumptions of the readers background knowledge.

In some fields review articles are a good starting point as well, and they tend to briefly sum up the required background, but my understanding is that some fields don't do those much.

If you read "random" articles, I'd do a quick smell-test before digging in. See if code is available, ignore papers with clear hype in the abstract off-hand. You can also "navigate" the field by following citations, although this can be technically annoying as the publishing format is still tailored towards print, even though very few journals are actually printed anymore. If you hit a paywall, try sci-hub or just move on to a next one unless you're looking for something really specific.

If you have something more specific in mind, just email or call or go talk some researcher that looks to be doing something related to what you are looking for. Researchers tend to be quite eager to answer to the public of their stuff, and its seen as sort of a public service duty as well. Depends on the researcher quite a bit though. Maybe a good starting point would be somebody a bit "lower on the ladder". Maybe a postdoc or a PhD student (this depends on the country as well). Professors tend to be busier and actually may not be that up-to-date with their field (especially on technically detailed level) as they spend most of their time in administration and the funding ratrace.

Depending on the country you can just attend lectures too. At least in Finland university lectures are public by law (with some restrictions on e.g. practical lab stuff etc). You can see if the lecturer doesn't seem too busy after the lecture and just go and ask.

You can also just try go to conferences. They usually have a fee in theory, but I don't think you'll be turned away if you just browse around for posters or so, especially if its a smaller one. The fees are just sort of a scam (long and sad story) and researchers organizing the thing usually don't care about the fees at all.

For some fields there are some good youtube channels that provide summaries that can get you started. E.g. Two Minute Papers is good for machine learning/machine vision/"AI"/etc related stuff: https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg For computer graphics SIGGRAPH "video papers" are really nice and even entertaining: https://kesen.realtimerendering.com/sig2021.html

Hard to give more specific tips with such broad question. If you have a field or topic in mind, I could maybe give something more concrete.

[Ar-Curunir]

I assume this varies field by field. In my field (computer security and cryptography) we don’t have such an antagonistic view of research papers by our colleagues.

[jampekka]

I'm not familiar with those fields, but yes this varies a lot by the field. I'd assume cryptography at least is more math-based (at least theoretical stuff) and the math scene is quite different from empirical sciences or engineering.

The antagonistic view is not just towards my colleagues, I'm not particularly proud of my own papers either. I find it more a nuisance to "pay the bills" and a lot of my research goes unpublished (at least in journals) due to all the IMHO unnecessary hassle involved. Just a blog or something would be a lot nicer and probably would communicate the work better, and would ease the pretension of objectivity which I find mostly causes wrong impressions and makes writing really a chore.