[queuebert]

I do share code that way, but the traditional ivory tower standards by which I am judged require "refereed journal publications" in high impact factor traditional journals. I'm trying to fight back against that, largely unsuccessfully.

What would help me is to have the old geezers consider GitHub issues, PRs, and commits as a type of citation and to have a better way of tracking when my code gets used by others that is more detailed than forks.

I also think citations of your work that find errors or correct things should count as a negative citation. Because otherwise you are incentivized to publish something early and wrong. Thus the references at the end of the paper should be split into two sections: stuff that was right and stuff that was wrong.

[Mezzie]

> I also think citations of your work that find errors or correct things should count as a negative citation.

Strong disagree. Given how much influence colleagues can have over one another's career prospects, how petty academic disagreements can get, admin focus on metrics like citation count, and how it's easier to prove someone else wrong than to do your own original work (both have value, one is just easier), it would end up with people ONLY publishing 'negative citations' (or at least the proportion would skyrocket). I think that would be bad for science and also REALLY bad for the public's ability to value and understand science.

> Thus the references at the end of the paper should be split into two sections: stuff that was right and stuff that was wrong.

This, on the other hand, is brilliant and I love it and want to reform all the citation styles to accommodate it.

[fho]

Which is basically the reddit up- & down-vote versus the HN only upvotes story just on a broader scale, is it?

[Mezzie]

Superficially yes, but in actuality it would be very different due to the context surrounding academic papers vs. Reddit.

Organizationally speaking, Reddit is a dumpster fire; check out the 'search' function (I'm just speaking on a taxonomical/categorization perspective, I can't speak to their dev practices).

Academic papers aren't. (They're a dumpster fire in their own ways: The replication crisis and the lack of publishing negative results comes to mind, but damn if they aren't all organized!)

There's two key differences:

1.) Academic papers have other supporting metadata that could combine with the more in-depth citation information to offer clear improvements to the discovery process. Imagine being able to click on a MeSH term and then see, in order, what every paper published on that topic in the past year recommends you read. I also think improving citation information would do a lot to make research more accessible for students.

2.) Reddit's system lets anybody with an account upvote or downvote. Given you don't even need an email address to make a Reddit account, there's functionally zero quality control for expressing an opinion. For academic publications, there is a quality control process (albeit an imperfect one). If only 5 people in the world understand a given topic, it's really helpful to be able to see THEIR votes: If they all 'downvote' a paper that would suggest it's wrong.

[queuebert]

You make a good point, and maybe there is a mathematical solution, like netting out mutually negative citations.

[Mezzie]

Maybe! Probably not that solution since it would penalize academics who are having a long-standing but intellectually productive disagreement.

I think it'd be hard to math out right now since the skill doesn't exist in the departments who'd be doing the work, but in 20 years who knows?

[d110af5ccf]

> the references at the end of the paper should be split into two sections: stuff that was right and stuff that was wrong

I've seen stuff like this said before but I don't think it would work. Most citations are mixed in my experience. A few objections, a bunch of stuff you aren't commenting on, and some things you're building on. Or you agree with the raw data but completely disagree with the interpretation. Others are topical - see <work> for more information about <background>. Probably more patterns I'm not thinking of.

[Mezzie]

Yes. We should record all of this, and turn them into easily browsable graphs/hypertext to easily assemble sets of papers to read/look into. At the very least things like 'background reading', 'further reading', 'supporting evidence' and 'addressed arguments' would be useful.

'We' meaning the librarians and archivists. You guys actually researching have more than enough to do.

[d110af5ccf]

Actually I think that's an intriguing idea for how to improve citations. Instead of a single <work> citation, have multiple <work, subset> citations that include a region of text as well as basic categorization of how the citation is being used in that instance.

I'm not sure if it would prove feasible in practice. It seems like it would aid the writing process in some cases by helping the author keep track of details. But in other cases maintaining all that metadata would become too much of a burden while writing, so it would get put off, and then it would all fall apart.

Very interesting to think about!

[Mezzie]

I was imagining a post-writing process akin to assigning a paper its DOI[0] or a book its cataloguing info. Citations as they are can be done by researchers without impacting the process because they're binary: Something either is cited or it isn't. It either contributed to the creation of the research or it didn't. This probably couldn't be done by the researchers, but you identified why: The citations are data but this would be metadata.

Definitely don't want to encourage papers to take even LONGER.

[AshamedCaptain]

> What would help me is to have the old geezers consider GitHub issues, PRs, and commits as a type of citation

As a geezer myself I am imagining what this type of request would look like in less than 5 years from now.

"What would help me is to have the old geezers consider Tiktok videos and replies as a type of citation"

i.e. if you open this particular can of worms for a very restricted subset of users (not only programmers but specifically programmers who also happen to use github), you have to open it to everyone else. I am sure plenty of Youtube research qualifies as "citation" if you start counting Github commits.

[mattkrause]

So what?

Papers qua papers aren't the goal. The idea is to advance our collective understanding of a field. Papers are certainly a means to that end, but other things can too, like code, videos, and blogposts, even if they don't fit into the "6000 words and 6 figures" box.

I get that citations and citation metrics feel objective, but they emphatically aren't great measures of research/researcher "quality".

[detaro]

A good youtube video easily can represent more work than a bad (published) paper, so why not.

[fennecfoxen]

Tiktok videos are primarily used for entertainment, rather unlike Jupyter notebooks and source code repositories. Surely you have a more serious objection.

[queuebert]

You don't read git repos for entertainment?!

[fennecfoxen]

For entertainment, I tend to read about things outside my field — things like In the Pipeline where I learn about FOOF and chlorine triflouride and freaky molecules like jawsamycin (insert shark theme here). I also watch Chemical Safety and Hazard Investigation Board videos on YouTube, like that time a refrigerator accident at a poultry plant caused hydraulic shock and released a massive cloud of ammonia, and ~150 contractors hanging out across the river working on Deepwater Horizon cleanup measures got sent to the hospital.