I think patterns and best practices are still yet to be figured out. I believe you can create caller in a server component and it should work (https://trpc.io/docs/server/server-side-calls#create-caller), but the pages router appears to be a battle-tested solution.
Thank you I only briefly read about them before and didn't consider them properly.
Will try out calling the database directly there, but I kinda liked how with tRPC I can add validation/auth checks etc/ will have to see how I develop my own strategy for that w server actions.