[ryanworl]

You would use two buckets in this case. Input bucket gets consumed by worker processes to do the transcoding (and validation) and then they upload into the output bucket. The output bucket is what you serve to clients (hopefully with a CDN in front).

[rawnlq]

Thanks! This is a great solution but none of the tutorials/blogs I read on pre-signed uploads mentions it.

Do you have links (or just keywords) to learn more? Will I need to add something like Cloud Pub/Sub to my stack? https://cloud.google.com/solutions/using-cloud-pub-sub-long-...

This is more complicated than I imagined so I am not sure the cost saving will still work out (factoring in development time and extra code maintenance cost).

[ryanworl]

No, I don’t, sorry. What I can promise you is that you’ll thank yourself for implementing it! There is hardly any additional complexity here because you’d probably be uploading the derived content somewhere anyway. Now you’re just putting in a different place than the source.

You can use whatever queue you’re comfortable with so long as you can pipe the upload events from the bucket into it. The pattern I’m outlining is just a physical separation of buckets to make access control much harder to screw up.

[TheReveller]

I can comment on using pub/sub - it's an immensely useful abstraction for these kinds of tasks and something that is quite difficult to implement yourself with the same level of guarantees that using the cloud service will provide. Any time you need to pass information or trigger events asynchronously messaging is the first choice IMO.