The prototype runs through ollama on my laptop while I browse. I usually have few hundreds stories ready from the previous session, as new stories come in they get processed in the background while I browse the old ones. It processes stories faster than I consume them.
When I move it to the server I might consider waking it up periodically to pull and analyze new stories and perhaps notify me if something absolutely great shows up.
There are so many possibilities for tuning it. And I don't need to think how to make it secure (beyond the basics), ultra performant, fitting other people's tastes and so on because this program has audience of one.
I was basically throwing stuff at the wall and seeing what sticks and makes a pretty picture for me. Vector embeddings were just one of the things I threw, in hopes of boosting news items thematically similar to the stuff I already liked. I can't really tell how much good or bad are they doing. All I know is that they are fast (embeddinggemma through ollama) and their output is worth few points in some combined heuristics that AI concocted for me.
When I move it to the server I might consider waking it up periodically to pull and analyze new stories and perhaps notify me if something absolutely great shows up.
There are so many possibilities for tuning it. And I don't need to think how to make it secure (beyond the basics), ultra performant, fitting other people's tastes and so on because this program has audience of one.