Advertisement
Guest User

Counting the book formats that have an epilogue

a guest
Sep 29th, 2015
88
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 8.08 KB | None | 0 0
  1. Hello, I'm using the Java API and am having a hard time making a query that aggregates as intended.
  2.  
  3. I tried simplifying the problem data.
  4. - Our root document "authors" have "books" (nested).
  5. - Books have a format ("ebook" or "hardcover"), and "chapters" (nested).
  6. - Chapters have a "chapter_type" ("prologue", "regular", "epilogue").
  7.  
  8. Here are the mappings for that:
  9.  
  10. curl -XPOST localhost:9200/test_index -d '{
  11. "mappings": {
  12. "author": {
  13. "properties": {
  14. "books": {
  15. "type": "nested",
  16. "properties": {
  17. "format": {
  18. "type": "string",
  19. "index": "not_analyzed"
  20. },
  21. "chapters": {
  22. "type": "nested",
  23. "properties": {
  24. "chapter_type": {
  25. "type": "string",
  26. "index": "not_analyzed"
  27. }}}}}}}}}'
  28.  
  29. Here is test data:
  30.  
  31. curl -XPUT 'localhost:9200/test_index/author/Mr_a' -d '{
  32. "books": [
  33. { "format": "ebook",
  34. "chapters": [
  35. { "chapter_type" : "prologue" },
  36. { "chapter_type" : "regular" },
  37. { "chapter_type" : "regular" },
  38. { "chapter_type" : "epilogue" }
  39. ]
  40. },
  41. { "format": "hardcover",
  42. "chapters": [
  43. { "chapter_type" : "prologue" },
  44. { "chapter_type" : "regular" }
  45. ]
  46. }
  47. ]
  48. }'
  49.  
  50. I try to get the count per format (terms aggregation on books.format) given that I only want books that have an epilogue
  51. (In practice I have all sorts of other constraints and query parts on both "books" and "chapters".)
  52. I tried all sorts of combinations on the aggregation filters and nested filters without success. I either get counts for all books in one author, or nothing at all.
  53. Here's one of the queries I tried, It's not applying the filter constrains as intended, and I'm getting back counts for ebook:1, and hardcover:1, when what i'm expecting is ebook:1 only.
  54.  
  55. curl -XGET 'localhost:9200/test_index/author/_search' -d '{
  56. "query" : {
  57. "bool" : {
  58. "should" : {
  59. "nested" : {
  60. "path" : "books",
  61. "query" : {
  62. "filtered" : {
  63. "query" : {
  64. "bool" : { }
  65. },
  66. "filter" : {
  67. "and" : {
  68. "filters" : [ {
  69. "nested" : {
  70. "path" : "books.chapters",
  71. "filter" : {
  72. "and" : {
  73. "filters" : [ {
  74. "terms" : {
  75. "books.chapters.chapter_type" : [ "epilogue" ]
  76. }
  77. } ]
  78. }
  79. }
  80. }
  81. } ]
  82. }
  83. }
  84. }
  85. },
  86. "inner_hits" : {
  87. "name" : "books",
  88. "_source" : false
  89. }
  90. }
  91. }
  92. }
  93. },
  94. "_source" : false,
  95. "aggregations" : {
  96. "filtered_aggregation" : {
  97. "filter" : {
  98. "nested" : {
  99. "filter" : {
  100. "nested" : {
  101. "filter" : {
  102. "term" : {
  103. "books.chapters.chapter_type" : "epilogue"
  104. }
  105. },
  106. "path" : "books.chapters"
  107. }
  108. },
  109. "path" : "books"
  110. }
  111. },
  112. "aggregations" : {
  113. "sub_aggregation_books" : {
  114. "nested" : {
  115. "path" : "books"
  116. },
  117. "aggregations" : {
  118. "count_formats" : {
  119. "terms" : {
  120. "field" : "books.format"
  121. }}}}}}}}
  122. '
  123.  
  124. Here's the response:
  125.  
  126. {
  127. "took": 3,
  128. "timed_out": false,
  129. "_shards": {
  130. "total": 5,
  131. "successful": 5,
  132. "failed": 0
  133. },
  134. "hits": {
  135. "total": 1,
  136. "max_score": 1,
  137. "hits": [
  138. {
  139. "_index": "test_index",
  140. "_type": "author",
  141. "_id": "Mr_a",
  142. "_score": 1,
  143. "inner_hits": {
  144. "books": {
  145. "hits": {
  146. "total": 1,
  147. "max_score": 1,
  148. "hits": [
  149. {
  150. "_index": "test_index",
  151. "_type": "author",
  152. "_id": "Mr_a",
  153. "_nested": {
  154. "field": "books",
  155. "offset": 0
  156. },
  157. "_score": 1
  158. }
  159. ]
  160. }}}}
  161. ]
  162. },
  163. "aggregations": {
  164. "filtered_aggregation": {
  165. "doc_count": 1,
  166. "sub_aggregation_books": {
  167. "doc_count": 2,
  168. "count_formats": {
  169. "doc_count_error_upper_bound": 0,
  170. "sum_other_doc_count": 0,
  171. "buckets": [
  172. {
  173. "key": "ebook",
  174. "doc_count": 1
  175. },
  176. {
  177. "key": "hardcover",
  178. "doc_count": 1
  179. }
  180. ]
  181. }}}}}
  182.  
  183. Here's java code for some trials I have made:
  184.  
  185. AbstractAggregationBuilder aggregation = AggregationBuilders.filter("filtered_aggregation")
  186. .filter(FilterBuilders.nestedFilter("books", FilterBuilders.nestedFilter("books.chapters",
  187. FilterBuilders.termFilter("books.chapters.chapter_type", "epilogue"))))
  188. .subAggregation(AggregationBuilders.nested("sub_aggregation_books").path("books")
  189. .subAggregation(AggregationBuilders.terms("count_formats").field("books.format")));
  190.  
  191. AbstractAggregationBuilder aggregation2 =
  192. AggregationBuilders.nested("nested_in_books").path("books")
  193. .subAggregation(AggregationBuilders.nested("nested_in_chapters").path("books.chapters")
  194. .subAggregation(AggregationBuilders.filter("filtered_aggregations")
  195. .filter(FilterBuilders.termFilter("books.chapters.chapter_type", "epilogue"))
  196. .subAggregation(AggregationBuilders.terms("count_formats").field(
  197. "books.chapters.chapter_type"))));
  198.  
  199. AbstractAggregationBuilder aggregation3 =
  200. AggregationBuilders.nested("nested_in_books").path("books")
  201. .subAggregation(AggregationBuilders.filter("filter_on_books")
  202. .filter(FilterBuilders.nestedFilter("books.chapters",
  203. FilterBuilders.termFilter("books.chapters.chapter_type","epilogue")))
  204. .subAggregation(AggregationBuilders.terms("count_formats").field("books.format")));
  205.  
  206. Obviously there's something about nested aggregations and filter aggregations that I'm not getting right... can somebody help me?
  207.  
  208. Thanks!
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement