yea I think within the context of BigQuery the most sensible thing would be to do an aggregate per the column that would be considered a primary key. For example [0]. That said, Streaming API de-dupe window is very nice in practice.
I mentioned elsewhere on Google Cloud the most elegant way of doing this is with Google Cloud Dataflow [1]