Advertisement
Guest User

Untitled

a guest
Aug 29th, 2016
561
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 1.34 KB | None | 0 0
  1. To elaborate more on this, here's a quick explanation on how DynamoDB Stream shards work:
  2.  
  3. The amount of shards that your stream has, is based on the amount of partitions the table has. So if you have a DDB table with 4 partitions, then your stream will have 4 shards. Each shard corresponds to a specific partition, so given that all items with the same partition key should be present in the same partition, it also means that those items will be present in the same shard.
  4.  
  5. Now, one other thing that you need to keep in mind, is that DynamoDB Stream shards are closed after a specific period. When a shard is closed (becomes read only), a new child shard will be created and new records will be written to that new active shard. Old records that were previously written to the stream (before the new shard was created) will be present in the closed (read only) parent shard.
  6.  
  7. If you use the KCL to process your stream, it will handle all the heavy lifting for you. The KCL knows how many shards the stream has and it also knows when shards are closed and new ones created. So we always advise customers to use the KCL when possible. Lastly, Lambda will also handle all of this for you. Like the KCL, Lambda also knows when shards are closed and new ones opened. So customers usually won't need to program any application logic to handle that part of the work.
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement