[thicknavyrain]

I'm a popular science writer with eight year's experience doing exactly this (SciShow, Crash Course, Veritasium and recent winner of the Wellcome Collection Non Fiction Awards) without AI. Done right, the right coverage of even a pre-print reached hundreds of thousands/millions of people. But I've experimented with every SOTA model since 2022 with the most detailed and specific prompting I can think of (including multiple examples of transcripts of work already in the public domain) to see if it can replicate good quality science communication.

The content is usually reasonably strong but the tone is always off and it never quite understands what it is a reader/viewer needs to really get to grips with the topic if they don't already have a prior foundational understanding (though I notice this about a lot of other media outlets with professional science communicators too). It also has poor editorial thinking around what bits are most likely to be interesting and cohesive when considered as part of the whole piece.

But I'm still reasonably convinced as AI improves it ought to be able to replace me with the right workflow/context/prompting. I think there will always be a demand for my (and many other writers') talents as they are so it doesn't really bother me, but it'd be great to extend the work to all the many scientific discoveries that don't get the same attention. If anyone is serious about developing something like this, I'd be interested in partnering with them as someone with domain expertise on science communication and familiar with prompt engineering (email in bio).

[trevor-e]

That's super cool, I love the SciShow videos.

I think you're right about the editorial thinking + what do people find interesting parts. But that doesn't have to be solved by directly by AI, it's easy enough to sidestep the problem and provide a nice interface for the human-in-the-loop part. I'd imagine that would save you a ton of time by having a nice starting point depending on how much you have to rewrite for tone.

[thicknavyrain]

That's true, it could just turn the writer's role into more of an editorial role. The main time-saving I have so far is being able to upload papers and get it to fact check for me. The editorial guidelines at SciShow are stricter than any academic journal I've published in: any non-trivial statement has to be supported by a direct, findable quote in (most-of-the-time) peer-reviewed scientific literature. I once had to find a citation for the idea that heat + fuel + oxygen generates a fire! (for this video: https://www.youtube.com/watch?v=BEcaE0e0CZg)

LLMs make that much easier. As I collect primary sources during my drafting/writing phrase, I can type up any non-trivial claims I'm making in my script in a separate document, share that with the LLM and say "Quoting directly from the set of attached PDFs, identifying which document, and on which page the quote comes from, find content which directly supports each of these assertions" and it generally goes a great job. At any rate, I have to check each of those quotes for accuracy but the help in _finding_ those quotes in order to pass a stringent fact checking procedure is a huge help if I didn't scribble down the supporting quotes during my research phase. This is also, by the way, stricter than the fact checking process for most non-fiction publishing.

[thekracken]

>SciShow are stricter than any academic journal I've published in

Now there's a testimonial. I look forward to browsing the source links with each video!

[notahacker]

Feels like there might be an accuracy issue as well. Although that might make it perfectly suited to replacing whoever writes university press releases...

[thekracken]

>I've experimented with every SOTA model since 2022

>The content is usually reasonably strong but the tone is always off and it never quite understands what it is a reader/viewer needs

A SOTA model fine-tuned with your choice of transcripts could probably get you most of the way there. There might be a customized, open-weight model already on Huggingface that meets your needs.

[antonvs]

One question is whether the audience is discerning enough to care about the issues you're raising. This seems like it could be a variation of why the umpteenth Marvel movie beats out indie masterpieces at the box office. The audience for high quality becomes increasingly niche as the market's relatively low bar for quality is satisfied.

[kristianp]

> as AI improves it ought to be able to replace me with the right workflow/context/prompting

The bitter lesson may very well come for us all...