Understanding DynamoDB’s Limitations To Design Better Databases
Learning about its limitations will let you build more efficient and scalable databases
Limitations in any tool or technology are always important to understand, as they help us decide on tradeoffs and assess alternate possible solutions.
In DynamoDB these limitations affect how you model your data therefore understanding them is crucial to designing good databases.
Item Size Limits
A single item in your DynamoDB table cannot contain more than 400KBs of data.
While this is sufficient for the great majority of use cases, it is smaller than what other database services offer.
Firebase has a 1MB item limit
MongoDB: 16MB item limit
Couchbase: 20MB item limit
ScyllaDB: 2GB item limit
Cassandra: 2GB item limit
Why is this size limit smaller than most other databases?
The reason for the smaller size limit is that DynamoDB wants to protect you against high read and write latency.
Large items take longer to read and write from/to your database.
Items that are large in size also allow for fewer concurrent requests.
Query Size Limits
When you query for multiple items in your database, DynamoDB will limit the query request to 1MB of data.
This is done again to deliver data at scale with the lowest latency possible.
If you have requests that exceed 1MB of data, you can simply paginate through the results by submitting a “follow-up” request.
Partition Throughput Limits
In DynamoDB, you have the choice to use provisioned throughput capacity or on-demand throughput capacity
Provisioned capacity lets you define a range of throughput you wish for your database to support (a minimum and maximum capacity for DynamoDB to stay within)
On-demand capacity mode lets DynamoDB dynamically provision throughput to match your traffic patterns
A read capacity unit is roughly 4KBs of data queried, a write capacity unit is 1KB of data queried.
A single partition — the storage node you define (with a partition key) for where to store your items — can have a maximum of 3000 read capacity units and 1000 write capacity units per second.
These limits are exceedingly generous as RCUs and WCUs work on a per-second basis and are applied to the partition level only, not the table as a whole.
Local Secondary Index Limits
Finally, another important limit to be aware of is with regard to local secondary indexes (or LSIs).
An item collection — a group of items with the same partition key — cannot be larger than 10GB in an LSI.
This could become problematic since if you exceed this limit, your writes will get rejected.
To solve this limitation, you can consider using a global secondary index (GSI).
Another possible solution would be to shard your data further.
With GSIs however, there is virtually no limit on the size an item collection can grow to, although it is bad practice to make an item collection too large.
Conclusion
Like with all other databases and tools, DynamoDB comes with its limitations.
But in DynamoDB, these limitations are mostly intentional; designed that way to not let you design a bad query or keep its promise of a single-digit millisecond latency database.
Understanding these limitations in DynamoDB will help you design better and more efficient databases capable of massive scalability.
👋 My name is Uriel Bitton and I’m committed to helping you master Serverless, Cloud Computing, and AWS.
🚀 If you want to learn how to build serverless, scalable, and resilient applications, you can also follow me on Linkedin for valuable daily posts.
Thanks for reading and see you in the next one!