Hacker News new | ask | show | jobs
by chadboyda 3738 days ago
The problem with autoscaling DynamoDB is that every time you scale up they shard your table into more partitions but they never merge them back when you scale down. You end up with less throughput per partition once you've scaled back down. The only way to correct this is to completely recreate your table. Does Neptune take this into consideration?
1 comments

@chadboyda: Good point, per-partition throughput reduces as the #partitions increases but if your load is uniform, it wouldn't impact your overall throughput. Neptune does take this into consideration, although we can't change the inherent DynamoDB behavior. We talk about how to address this problem proactively in our best practices. Our recommendation: create table with 12-month peak throughput and then immediately bring it down to what you want right now. If the table is already created, bump up the throughput to the 12-month peak just once and then bring it down to what you want right now. In either case, this will ensure DynamoDB doesn't change partitions internally when you scale up and scale down within in 12-month peak range. But if you range goes beyond the peak, you'd still run into the problem that you'd described. We've seen in many case, people can predict the highest peak with reasonably high confidence. (think database world where they'd always known this in the past for many years).

Refer to our first best practice point for more details: http://blog.neptune.io/dos-and-donts-of-dynamodb-autoscaling...