Hacker News new | ask | show | jobs
by SaidinWoT 758 days ago
Constructively using LLMs tends to require validating the quality of their output; even when not hallucinating content, they do hallucinate confidence. Increasing the size of the output dramatically increases the effort needed to validate it.

Both examples have sections where the model simply left in placeholders for concepts - in example 1, "Summary of Key Takeaways" repeatedly references `[Book Topic]`, while example 2 starts doing so much earlier in "Overview of the Book".

What is the goal of this project? If it is meant to be more than a learning exercise, I'd hope for a lot more investment on quality control for the final output (but then I'd perhaps be even more worried that people would trust that output uncritically).

1 comments

The goal was to showcase a task where Groq's speed would be useful while showing what current LLMs can and can't do with a task like book generation. That's why the placeholder content is mentioned in the limitations section. The end books are definitely not perfect, but I am impressed by the generations nonetheless, especially since the majority of the content is generated using Llama3b-8b.

I don't think a publishable-quality book can currently be generated this way. I do think it is a helpful tool for generating an entire book on any nonfiction topic you want to learn more about, no matter how specific.