[sjkoelle]

Following - we host our own models for a variety of architectures in vocal synthesis, and have tried using Replicate and Mystic as well.

Roll your own k8s? Predibase?

[DidISayTooMuch]

Thanks for the tip. Predibase has support for Zephyr-7B, but I wonder if they offer the same price per 1k token for a fine-tuned version of Zephyr-7B? Most likely, they will ask me to get a dedicated instance for that, which is the same as together.ai.

[DidISayTooMuch]

Just checked out mystic.ai, it looks like you only pay for usage on any model and not idle time. Might actually fit my requirements.