Advertisement
Guest User

Untitled

a guest
Apr 1st, 2015
190
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 2.99 KB | None | 0 0
  1. Hello,
  2.  
  3. I've checked into your table and determined that you have indeed encountered a hot partition problem.
  4.  
  5. As Rendy mentioned, sometimes you can get around hot key problems by moving your hot keys into different tables.
  6.  
  7. There are a lot of caveats, however.
  8.  
  9. If your Hash Key cardinality is and will remain low -- that is, you will only have a relatively small number of distinct values for, in this case, the 'domain' field, then you can divide your table into separate tables based on the domain.
  10.  
  11. You can't create an unlimited number of tables, so if you have many distinct values for the hash key (and therefore many tables), this is unsustainable.
  12.  
  13. If that's not a problem, you will still need to choose a different hash key with high cardinality. For example, if your new tables use the subdomain as a hash key (which seems like a natural choice), ideally this should have a large number of distinct values, and reads and writes should be evenly distributed across the keys.
  14.  
  15. If those two attributes(high cardinality and even distribution) aren't met, you will still encounter hot partitioning problems.
  16.  
  17. You may want to review the following documentation to determine the best design for your use-case.
  18. http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/BestPractices.html
  19.  
  20. Can you provide an overview of your use-case? Unfortunately it is difficult to give detailed guidance without knowing the context of what you're trying to do with your table.
  21.  
  22. Best regards,
  23.  
  24. Jeff D.
  25. Amazon Web Services
  26. We value your feedback. Please rate my response using the link below.
  27. ===================================================
  28.  
  29. Overview of use-case
  30.  
  31. We are getting js url as request e.g `https://www.example.google-analytics.com/analytics.js`
  32.  
  33. We are storing it in `tbl_scripts_20150312` in following columns
  34. pattern = `https://www.google-analytics.com/analytics.js`
  35. subdomain = `example.google-analytics.com`
  36. domain = `google-analytics.com`
  37.  
  38. Before storing it we are checking existing entry for records in table on three different levels using `classification_applies_to` (which is also Global Secondary Index) value.
  39.  
  40. So we first search in `tbl_scripts_20150312` with pattern = `https://www.google-analytics.com/analytics.js` and `classification_applies_to` = `script_only`, if we found record we use `script_id` for it and stop further search and storing.
  41. Else we search in `tbl_scripts_20150312` with subdomain = `example.google-analytics.com` , if we found record we use `script_id` for it and stop further search and storing.
  42. Else we search in `tbl_scripts_20150312` with domain = `google-analytics.com` if present, if we found record we use `script_id` for it and stop further search and storing.
  43. Else as we haven't found record in all levels we store the value in table.
  44.  
  45. Now, We have planned to use `Elasticache` redis database to store values for `domain` and `subdomain` fields.
  46. We will use dynamodb to query values for `pattern` or store values if not found in `cache` or in `pattern`.
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement