[bigbird-media]

Yes, I was thinking of implementing that and probably will soon. Usability wise, it make sense (and supports our mission statement), but in practice that means we need a GPU always available to do that (currently using Colab, meaning GPUs are scarce and I only have time to generate full articles rather than be able to attend to users' needs).

If this project does go somewhere, that'll be my next feature.

[spicyramen]

Since you are student you should look into Google cloud credits and use preemptible instances with GPU

[bigbird-media]

I did :). And Google didn't give me any (mostly because I'm not an official "researcher" under a professor.

I do use GCP all the time though (make new accounts and utilize the sweet $300 in compute).

[kalium_xyz]

Try github student pack free DO hosting credit, needs a credit card tho.

[zitterbewegung]

Inference might be able to be done on a CPU . Try that first