[loumf]

Since the point of a title and abstract of a paper is to be a useful summary of the whole paper, the existence of this tool is an indictment of researchers to do this effectively.

One thing I could imagine being useful is to summarize it for a lay audience (rather than the intended audience of the paper).

[godelski]

> Since the point of a title and abstract of a paper is to be a useful summary of the whole paper,

I'd like to nit pick this a little. The title and abstract of the paper is optimized towards reviewers. There is the assumption that reviewers are aligned with researchers seeking to read new papers (after all, papers are simply the act of communicating from researcher to researcher), but I don't think this assumption is actually valid.

I complain a lot about reviews, but I'll give an example that is simple and I'm sure is true in almost every domain: paper length. There are far too many papers that could be a page or two but are 10 because that's what the journal/conference requires. If you don't fill the pages you're more likely to get rejected as the reviewer has more validity to claim your work was not thorough enough rather than your explanation simply being concise[0]. This is only exaggerated as we are in a publish or perish paradigm and publishing faster and in more competitive environments. But papers are like wizards: they're meant to be as long as they're meant to be. No more, no less. (At least that's how they should be if you're targeting fellow researchers in your niche)

It's an all too common mistake to believe that metrics are perfectly aligned with some well defined but abstract goal. Rather they are generally aligned with proxies that correlate with the desired goal. You'll find this everywhere from trying to measure the quality of LLMs to trying to exterminate cobras in Colonial India. Pay close attention or Goodhart will be turning in his grave.

I'd say more about the science communication aspect but I don't want to rant too much and I think one could guess a much lengthier response extrapolating from my thoughts above.

[0] Similarly papers get cut to fit the length and tough decisions are made about what goes in the front matter vs the appendix because reviewers are not required to read the appendix and a large portion simply do not (https://twitter.com/sarahookr/status/1660250223745314819).

[kristopolous]

The real silver bullet would be not to summarize for a general layperson but somehow, personally, for the context of "me", whoever that is. For instance, the amount of approximation language and jargon can vary to make it optimally accessible

I'm pretty sure this is achievable right now with just a lot of work.

[hhh]

I don’t think it’s that much work, if you have a general sense of your knowledge domains. I described the platform I work on in a few sentences, my knowledge level, and what I like out of a response and it’s been pretty great for that.

[kristopolous]

I was thinking of a more generalized one with "personas" which use things like embeddings and hypernetworks to "know" you.

Of course the presumption here is that more useful results would avail themselves under the added training load. It might be just as good as the simple usecase.

For instance, ideally it would know your strengths, weaknesses, blind spots and misconceptions so it will know what you don't know you don't know.

[mattfalconer]

That is what it tries to do, the title, bulletpoints, summary and 'FAQs' try to simplify the paper, but LLMs are hard to tame when given an entire paper in the context window.

[pclmulqdq]

I would assume that the existence of abstracts and titles is what allows tools like this to be effective at all. Abstracts are probably shorter than a useful summary of a paper should be, and this is by design. You would prefer that a summary have a few more things, like how experiments were actually conducted, but the abstract tells you what words to look for to find that.

[diffeomorphism]

Abstracts are often quite a bit shorter than proper summaries and function more like elevator pitches (mainly to reviewers and other people in the field). They are not intended to do what you say they should.

If you suggest to also add summaries, that is something I could get behind, but right now your criticism is a bit misplaced.

[SkyMarshal]

What I really want to see more than anything is an upfront explanation of what the paper’s novel new technique or idea is. Sometimes abstracts include that, but not always, but I wish it were SOP for all scientific or technical papers to include in the abstract. If AI could make it SOP that would be an improvement, but it may take AGI to be able to make that distinction.

[redwood]

It's interesting you say that, because I've always found abstracts from academic papers incredibly dense and hard to follow. They tend to really not summarize very well