Hacker News new | ask | show | jobs
by WXLCKNO 1430 days ago
Enjoyed this post! Djk447 would love to ask a question as well.

We've started working with Timescale to process historical time series data. However there is so much of it that we chose which parts we process.

It's possible that in the future we may need to go back and reprocess the dataset to add something we decided we want after all.

In your post it seems like this is handled automatically on a smaller time scale, meaning you could insert into the past without doing anything special.

What happens if you need to insert data at numerous points across two years worth of data instead? Do you have to use backfilling as described in your documentation? Or is it better to maybe rebuild the entire hypertable?

1 comments

NB: Post author

Yes. this is generally handled automatically, there may be times though where you want to essentially pause refreshing the view for a while while you do some backfilling and then eventually let it catch up, especially if you're overwriting the same time period multiple times in a row. If you can insert in time order then it just breaks up re-calculation into smaller segments, which can be quite useful rather than having to process the whole data set again.

This can be a little bit different if you're doing compression, but with continuous aggregates I think it should work fine. I'm not 100% sure that was what you were looking for, let me know if it's not.