Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- Hello,
- I've checked into your table and determined that you have indeed encountered a hot partition problem.
- As Rendy mentioned, sometimes you can get around hot key problems by moving your hot keys into different tables.
- There are a lot of caveats, however.
- If your Hash Key cardinality is and will remain low -- that is, you will only have a relatively small number of distinct values for, in this case, the 'domain' field, then you can divide your table into separate tables based on the domain.
- You can't create an unlimited number of tables, so if you have many distinct values for the hash key (and therefore many tables), this is unsustainable.
- If that's not a problem, you will still need to choose a different hash key with high cardinality. For example, if your new tables use the subdomain as a hash key (which seems like a natural choice), ideally this should have a large number of distinct values, and reads and writes should be evenly distributed across the keys.
- If those two attributes(high cardinality and even distribution) aren't met, you will still encounter hot partitioning problems.
- You may want to review the following documentation to determine the best design for your use-case.
- http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/BestPractices.html
- Can you provide an overview of your use-case? Unfortunately it is difficult to give detailed guidance without knowing the context of what you're trying to do with your table.
- Best regards,
- Jeff D.
- Amazon Web Services
- We value your feedback. Please rate my response using the link below.
- ===================================================
- Overview of use-case
- We are getting js url as request e.g `https://www.example.google-analytics.com/analytics.js`
- We are storing it in `tbl_scripts_20150312` in following columns
- pattern = `https://www.google-analytics.com/analytics.js`
- subdomain = `example.google-analytics.com`
- domain = `google-analytics.com`
- Before storing it we are checking existing entry for records in table on three different levels using `classification_applies_to` (which is also Global Secondary Index) value.
- So we first search in `tbl_scripts_20150312` with pattern = `https://www.google-analytics.com/analytics.js` and `classification_applies_to` = `script_only`, if we found record we use `script_id` for it and stop further search and storing.
- Else we search in `tbl_scripts_20150312` with subdomain = `example.google-analytics.com` , if we found record we use `script_id` for it and stop further search and storing.
- Else we search in `tbl_scripts_20150312` with domain = `google-analytics.com` if present, if we found record we use `script_id` for it and stop further search and storing.
- Else as we haven't found record in all levels we store the value in table.
- Now, We have planned to use `Elasticache` redis database to store values for `domain` and `subdomain` fields.
- We will use dynamodb to query values for `pattern` or store values if not found in `cache` or in `pattern`.
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement