What is DynamoDB Hot Partition?
In this post learn what DynamoDB Hot Partition is, what it's application are and how it can help you with throttling quick fix
DynamoDB splits your table into smaller data chunks in order to speed up the read in large or quite large tables. This is known as partition. DynamoDB creates partitions for every 10GB of data, or in case the RCU or WCU limits for a single partition are exceeded. In case DynamoDB encounters a pattern of hot partition, it splices up the partition in order to fix the problem.
Items in a DynamoDB table are stored across several partitions based on a particular item’s partition key. Every partition consists of its share of RCU i.e read capacity unit and WCU i.e write capacity unit from the provision of the table.
When a request arises, it is routed to the appropriate partition for data. The capacity of this partition determines whether the request is accepted or throttled i.e rejected. A minor amount of throttling is normal and can be managed by your application.
However, in some cases, excess throttling becomes a major issue and is caused by hot partitions and limited capacity. Hot partitions refer to throttles that arise due to more than average requests in few partitions in the table. Whereas, in some cases, throttles arise due to the limited capacity of the table to service requests on several partitions.
Effects Of Hot Partition
DynamoDB Hot partition is an undesirable issue in your application and may lead to the following effects if left unchecked for long -
- Out of date data due to throttled writes but accepted reads
- Slow processing due to retrying throttled requests
- Loss of data due to failure of the application to retry throttled write requests
Throttling Quick Fix
In case the consumed RCU or WCU of your table is near or at the provisioned RCU or WCU, then the read and write throttles can be alleviated by increasing the provisioned capacity slowly.
This would require the knowledge of how partitioning in your DynamoDB works. For an application working at 100% capacity, it would take multiple capacity increases to find out the exact figure of the capacity.
However, increasing the capacity by a considerable size is not advisable since it may cause throttling issues because of the way partitioning works in indexes and tables.
Hot Partition Fix
1. Thorough Fix
In the case of hot partitions in DynamoDB, your application may experience throttling even with just a usage of 10% of the provisioned capacity. In order to fix throttling issues, it’s imperative to understand how partitions work.
- Partitions are never deleted when stored data or the capacity decreases.
- The capacity of a partition is 10GB of data.
- Only 1000 WCU or 3000 RCU can be accommodated in a partition.
- 2 new partitions are created when a partition is split and its data and current throughput are split in 2 as well.
- It is not necessary that all partitions will have the same provisioned throughput.
Some fixes -
Increase the capacity of the table or index.
Study the access patterns for your data and distribute access evenly across all keys to avoid a large number of requests on certain keys that might lead to hot partition.
Control the size of your growing tables using the Time To Live (TTL) feature of Dynamo.
Delete older data that is no longer relevant as this would reduce the load and eventually help with throttling due to hot partitions.
Make a new table that is designed with throttling in mind in order to ensure a uniform access pattern.
Deploy a lookup table in a relational database to manage query. Another option would be to use Amazon DynamoDB Accelerator (DAX) to help with reads.