[noosphr]

I've been at two startups that have done genuine world first fundamental research.

The first tried to publish novel results for 3 years in tier 1 journals before finally doing a preprint and telling the tier one publishers to jump in a fire.

The second, and ongoing, isn't publishing anything because of my experience with the first.

That and avoiding openAI and Anthropic copying our results and leaving us with nothing to show for six months of work. The papers only come with the pitch deck.

[YeGoblynQueenne]

Tier 1 journals have high standards (that's why they're tier 1) and they also have hundreds of submissions every cycle. It's common to have a paper rejected. On the upside you get your work seen by some of the most experienced researchers in AI, who are also specialists in your paper's subject. For example, here's the editorial board of the Journal of Machine Learning Research:

https://jmlr.org/editorial-board.html

Scroll down where it says "JMLR Editorial board of reviewers" and where you can find the names of reviewers and their areas of expetise.

The advice you get from such reviewers, even if your paper is rejected, is an invaluable tool to help you improve your work and its presentation. You eventually learn to not be sore about it and iterate until your work gets accepted.

Of course nobody's forcing you to publish, especially in a journal (in AI and CS conference proceedings tend to have much quicker turn around times and you naturally get more feedback if you get to present at a conference) but the alternative is to sit alone in your room hacking away at your code until you discover the perpetual motion machine.

[okaleniuk]

The alternative is to use your network directly. The very notion of "I'll publish my work and someone will notice me" is wrong. Most people in academia are too concentrated on their own work to care about yours. The only minority that cares is already in your contact list.

[YeGoblynQueenne]

Not always. AFAICT my papers have often been reviewed by people I have never met and who have expertise that overlaps with, but does not fully match, mine, and I couldn't possibly contact those people out of the blue to ask for feedback, particularly because I'd never get the chance to meet them outside of peer review (they wouldn't be in the same conferences as me etc). And that's a good thing because that way I get a sort of "outsider looking in" view.

I've sure given and taken informal feedback in the past from people I know and that's great but peer review also makes the whole process more formal and less sloppy; at least sometimes and depending on the venue and to my experience.

[tanker_war]

The publication model in academia is predatory and elitist, but have you considered that your paper just wasn't that good?

Almost every scientific paper published in every field is "world first" research, and the job of tier 1 journals is to decide what deserves to be signal boosted, and what doesn't.

This response to rejection to me confirms the fears implicit in the article linked above. Not only is private AI research not transparent and not documenting its work, it also has a deep rooted culture issue.

[jandrewrogers]

There is almost no benefit in publishing frontier research as a startup. People can try to argue it but it is not defensible. Publishing that kind of thing is a flex that companies risking nothing can do.

Cool for Google. Problematic if you are a startup.

[mkolodny]

Google has been publishing frontier research since it was an early-stage startup (https://research.google/pubs/the-anatomy-of-a-large-scale-hy...).

[silver_sun]

Looks like that was published in 1998, but that's also the year they submitted their patent application for PageRank. So you couldn't use it because they didn't want to risk losing their budding search business to, say, Microsoft.

In later years Google generally published the details of a system after they moved to the next one, to stay ahead of their competition, or if they didn't see it as important to their business (most infamously the Transformer language model). It was still generous of them to share their research at all though, and this allowed their scientists to talk publicly about their research which is a huge plus.

[otherme123]

Publishing is not fire and forget. You usually publish and then network your publication(s) in congresses, where you are approached by many people either working in closely related issues, or noticing your work is complementary to what they are doing. A lot of fruitful colabs start that way.

Also there are hybrid fairs with academic and market exhibitors in the same place, so you might find someone that sells something that fits your research. Some startups sprout from academia and keep being that hybrid for years (getting research grants but also selling something).

[thayne]

It is beneficial for society as a whole, even if it isn't beneficial to the founders and investors accrual of wealth.

[dgellow]

Could be a good way to attract talents. Could be a good way to establish your name for a potential acquisition

[make3]

I think talent would definitely take a startup that publishes more seriously, and it would be a smaller risk for top research talent to go somewhere where they can also publish than have to bear the full risk of their work at a no name startup not going anywhere

[ozgrakkurt]

You can define benefit in a lot of different ways. Not everything is measured in money.

[smcg]

Sure, but that means they should stop pretending that they are advancing scientific research, or pretending that their products are scientifically sound.

[patapong]

It depends - if you are looking to raise potentially billions of dollars, showing that you are contributing at SOTA level in GenAI through papers could be a good strategy.

[cortesoft]

> That and avoiding openAI and Anthropic copying our results and leaving us with nothing to show for six months of work.

Isn't the POINT of publishing research because you want others to copy it?

[willmadden]

Anthropic and OpenAI have proven themselves to be aspiring monopolists through their positions and actions. Why would you want to share your research with them?

[paytonjjones]

That's not really a common attitude in academia, unfortunately.

The most common attitude is something closer to "I want to be well known and respected by smart and influential people". In short, prestige.

People using your work is a side effect of success, but simultaneously threatens your ownership of your "brand". If they significantly improve upon the thing you did, your work could be made irrelevant, your prestige ruined.

[MichaelZuo]

Huh? Wouldn’t their prestige be improved if some other group spent a lot of effort improving on it?

That would clearly demonstrate it had real substance beyond hot air or puffery.

[zdragnar]

I would presume that if their results end up being more impactful or interesting, you end up being a footnote and lose the spotlight.

[perpetuallunch]

You could be proven wrong and wither in to obscurity.

Better to silo yourself and move in to the penthouse.

[r3trohack3r]

If you want people to copy it, but want a 20 year temporary monopoly, you can publish an open source step-by-step guide for someone to recreate what you’re doing with the patent office.

[thayne]

Unfortunately, there is an incentive to make the patent application contain just enough detail that a patent clerk with relatively little knowledge of the subject will grant a patent, but not enough detail that an expert would be able to follow to get the same result in practice.

Also 20 years is a crazy long time in some industries (like most software).

[jandrewrogers]

The patents are effectively unenforcible. People stopped patenting frontier computer science research 20 years ago. That is the current reality.

[sheeshkebab]

good.

[jandrewrogers]

The downside is frontier computer science in several domains is decades beyond the state-of-the-art from academic literature. This is the situation that patents were designed to disincentivize.

There is not an obvious path to making this research public again. That's the reality. For open source to be competitive it needs to be approximately as efficient as the best systems we design.

[elictronic]

Legislation??????

Listing on stock exchange of ai product requires open sourcing parts or all of your publicly trained stack.

Double or triple taxes on companies that do not provide for public good.

Will this happen, probably when the market crashes and people want companies and individuals to pay. Really matters if it’s a 2008 crash or a 1929 crash where the bankers and stock brokers were jumping out of upper stories.

[make3]

Do you want companies to have incentives to pay researchers even if NDAs and non competes are basically unenforceable? Do you want any communication of knowledge from corporations? Should companies not be allowed to benefit from sometimes huge investments in research?

[bigbadfeline]

> Do you want companies to have incentives to pay researchers even if NDAs and non competes are basically unenforceable?

I thought "recursive self improvement" would bring an effective abundance for everybody, any minute now, Musk said so himself. Seems the researchers in the field are lying to us about it while they clutch their pearls and dream about other people's money as if the future is a dog-eats-dog world of death-inducing scarcity.

They shouldn't be lying like that, or at least, their lying shouldn't be so obvious.

[notRobot]

And crucially, you should want to others to build upon it, like you build upon so many that came before you.

[sigmoid10]

If your perception of a moat is worth more than the GDP of most countries, publishing anything must seem like a bad idea. Much less helping others build upon your work.

[lenerdenator]

For researchers, yes.

For startups, no, because the point of that is to make money.

[geysersam]

So why is op trying to publish?

[perpetuallunch]

Arguably, the point of a startup isn’t to make money, it’s to attract attention and then investors.

Once the business is making money it’s not really a startup any more, it’s a going concern.

OP is an undergrad student. Attracting attention is what will make them stand out. And sharing research at this stage will probably expose them to research others don’t want to publish but do want to share with other researchers.

[lenerdenator]

> Arguably, the point of a startup isn’t to make money, it’s to attract attention and then investors.

Investors with money.

[perpetuallunch]

What do you mean by this?

What are investors who don’t have money?

Perhaps I should have said investors and speculators, but this doesn’t help you here.

[TeMPOraL]

Obviously.

To rephrase it more bluntly: the difference between normal business and a modern startup is that between a cow bred for milking vs. a pig bred for slaughter.

[xienze]

There seems to be this mentality among a large number of people who publish open data/research/software that everything is fine as long as no one else makes money using it. The second that happens, they're furious and act as though someone unfairly profited off of their "unpaid labor" (read: that thing they did purely for the love of the game). I think it has to be some sort of feeling of resentment and/or stupidity that _they_ were the person with a million dollar idea that let it slip through their fingers.

[fluoridation]

It's not that difficult. You invest effort into research in the hope it'll give you a competitive edge. If someone else can come along and reproduce your results practically at a much lesser cost, then you lose that competitive edge. You're back at square one, except you've lost the cost of the research.

[olalonde]

Hence why most companies don't publish research and most software is proprietary. Publishing and open source only make sense in certain situations.

[1718627440]

Open source doesn't mean, it's available to the general public.

[fluoridation]

What's your point?

[alt227]

Is this not why copyright exists?

[m000]

May I ask how did you try to publish, and if you have done this before?

Tier 1 is HARD. Universiy labs (where publishing is their bread and butter) and have multi-year streaks of Tier 1 papers, still seek out collaborations when publishing. Because you still need this extra angle on your work that will make it stand against the proverbial Reviewer 2, or that extra evaluation paragraph that will make it stand out. And don't forget that each Tier 1 publication is probably the life of at least 1 PhD student for several months.

So if you don't put the time and effort in it, forget Tier 1. This is like believing that because you have excellent voice and singing skills, all you have to do to get a top 10 hit is to send your demo tape to a recording company.

[eikenberry]

Why wouldn't they just skip the journals and publish the papers themselves? Sharing the research is the important part unless these are academics who are trying to get tenure, grants or such.

[londons_explore]

For the individual employees, having your name on a peer reviewed paper in a prestigious journal makes you far more employable elsewhere.

Moreso if your current employer is a stealth AI startup which hasn't yet produced much notable.

It also can encourage employees to work for fewer wages... 'If you do great work you can publish and then get a $$$$$$ job offer at OpenAI, or you can stay here and get equity in a fast growing startup. If however your work doesn't do well, you walk away from a bankrupt startup with worthless shares and minimum wage'

[noosphr]

The article is complaining that research isn't peer reviewed and research is turning into blogs. I'm explaining why that's the best outcome possible and why in a field as hot as large dl models you won't even get that.

[eikenberry]

I thought parent post was saying that they didn't publish them at all after they had so much trouble with the journals. If they did publish the papers themselves then I don't see the problem. The article itself read to me as your standard journal rent seeking and I wasn't really responding to it.

[fluoridation]

>Why wouldn't they just skip the journals and publish the papers themselves? Sharing the research is the important part

To answer your question, sharing the research isn't the important part. These are researchers in the private sector, in a highly competitive field. Arguably, doing the research isn't the important part either. The important part is making money, and the research is a means to that end. Publishing the research doesn't further that goal, and, according to GP, it's so difficult that it may not be worthwhile trying.

[Forgeties79]

Same reason movies released on YouTube aren’t treated with the same respect as those released in theaters.

[marcus_holmes]

Is this the "AI startups blog about discoveries rather than submitting to journals" phenomenon that TFA noted?

Are journals just too slow, hidebound, and predatory to be useful to this industry any more?

[accurrent]

This. The circus that is academic publishing is just not worth the effort. Ultimately, companies do have to make money.

[NetOpWibby]

Can you share a link to the preprint?

[noosphr]

https://www.biorxiv.org/content/10.1101/2021.12.02.471005v2....

[NetOpWibby]

Sweet, thanks! The intro reminds me of Tron: Ares, when Ares was put through endless battle training cycles until he was a solider. Fascinating stuff.

[fooker]

Publishing CS papers at the top venues requires using in-group language and formalism that is pretty much inaccessible to someone who has not done a PhD in that specific narrow field.

LLMs are pretty good at this, hilariously. This means a genuinely good paper by an outsider has significantly less chance of getting good reviews than AI slop.

[DonHopkins]

If an LLM can present good research in the field's required dialect better than a human outsider can, calling the result "AI slop" tells us nothing about its quality -- only about your prejudice against how it was produced.

And when that prejudice degrades the quality of your own thinking and writing, you're producing "human slop", which is considerably more tragic: you ought to be capable of doing better than the AI.

You're holding it wrong.

[phoghed]

> genuinely good content, dismissed without deep engagement because it didn’t use the correct in group language, would have otherwise passed muster

> genuinely bad content, dismissed after deep engagement

You have to be misreading the GP on purpose. Maybe feed it to your LLM of choice next time before you rant at someone.

The crux of their point is that outsiders have no mastery of this supposed specialized dialect, and LLMs do.

[fooker]

Exactly right.

Not only is it specialized, it also has its own field specific memes* and trends that evolve over time.

The purpose is mostly to signal "I'm one of you".

*memes as in the actual meaning of memes, not internet memes.

[fooker]

I think, maybe you misunderstood my comment?

I'm all for LLMs producing good research. That's obviously happening right now.

What I said was people with good research being penalized for not knowing in-group language.

It's not a zero sum game.

[ACCount37]

"Drop a preprint like it's a mic" is ML's new "publish".