Guest User

Counting the book formats that have an epilogue

a guest
Sep 29th, 2015
141
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 8.08 KB | None | 0 0
  1. Hello, I'm using the Java API and am having a hard time making a query that aggregates as intended.
  2.  
  3. I tried simplifying the problem data.
  4. - Our root document "authors" have "books" (nested).
  5. - Books have a format ("ebook" or "hardcover"), and "chapters" (nested).
  6. - Chapters have a "chapter_type" ("prologue", "regular", "epilogue").
  7.  
  8. Here are the mappings for that:
  9.  
  10. curl -XPOST localhost:9200/test_index -d '{
  11. "mappings": {
  12. "author": {
  13. "properties": {
  14. "books": {
  15. "type": "nested",
  16. "properties": {
  17. "format": {
  18. "type": "string",
  19. "index": "not_analyzed"
  20. },
  21. "chapters": {
  22. "type": "nested",
  23. "properties": {
  24. "chapter_type": {
  25. "type": "string",
  26. "index": "not_analyzed"
  27. }}}}}}}}}'
  28.  
  29. Here is test data:
  30.  
  31. curl -XPUT 'localhost:9200/test_index/author/Mr_a' -d '{
  32. "books": [
  33. { "format": "ebook",
  34. "chapters": [
  35. { "chapter_type" : "prologue" },
  36. { "chapter_type" : "regular" },
  37. { "chapter_type" : "regular" },
  38. { "chapter_type" : "epilogue" }
  39. ]
  40. },
  41. { "format": "hardcover",
  42. "chapters": [
  43. { "chapter_type" : "prologue" },
  44. { "chapter_type" : "regular" }
  45. ]
  46. }
  47. ]
  48. }'
  49.  
  50. I try to get the count per format (terms aggregation on books.format) given that I only want books that have an epilogue
  51. (In practice I have all sorts of other constraints and query parts on both "books" and "chapters".)
  52. I tried all sorts of combinations on the aggregation filters and nested filters without success. I either get counts for all books in one author, or nothing at all.
  53. Here's one of the queries I tried, It's not applying the filter constrains as intended, and I'm getting back counts for ebook:1, and hardcover:1, when what i'm expecting is ebook:1 only.
  54.  
  55. curl -XGET 'localhost:9200/test_index/author/_search' -d '{
  56. "query" : {
  57. "bool" : {
  58. "should" : {
  59. "nested" : {
  60. "path" : "books",
  61. "query" : {
  62. "filtered" : {
  63. "query" : {
  64. "bool" : { }
  65. },
  66. "filter" : {
  67. "and" : {
  68. "filters" : [ {
  69. "nested" : {
  70. "path" : "books.chapters",
  71. "filter" : {
  72. "and" : {
  73. "filters" : [ {
  74. "terms" : {
  75. "books.chapters.chapter_type" : [ "epilogue" ]
  76. }
  77. } ]
  78. }
  79. }
  80. }
  81. } ]
  82. }
  83. }
  84. }
  85. },
  86. "inner_hits" : {
  87. "name" : "books",
  88. "_source" : false
  89. }
  90. }
  91. }
  92. }
  93. },
  94. "_source" : false,
  95. "aggregations" : {
  96. "filtered_aggregation" : {
  97. "filter" : {
  98. "nested" : {
  99. "filter" : {
  100. "nested" : {
  101. "filter" : {
  102. "term" : {
  103. "books.chapters.chapter_type" : "epilogue"
  104. }
  105. },
  106. "path" : "books.chapters"
  107. }
  108. },
  109. "path" : "books"
  110. }
  111. },
  112. "aggregations" : {
  113. "sub_aggregation_books" : {
  114. "nested" : {
  115. "path" : "books"
  116. },
  117. "aggregations" : {
  118. "count_formats" : {
  119. "terms" : {
  120. "field" : "books.format"
  121. }}}}}}}}
  122. '
  123.  
  124. Here's the response:
  125.  
  126. {
  127. "took": 3,
  128. "timed_out": false,
  129. "_shards": {
  130. "total": 5,
  131. "successful": 5,
  132. "failed": 0
  133. },
  134. "hits": {
  135. "total": 1,
  136. "max_score": 1,
  137. "hits": [
  138. {
  139. "_index": "test_index",
  140. "_type": "author",
  141. "_id": "Mr_a",
  142. "_score": 1,
  143. "inner_hits": {
  144. "books": {
  145. "hits": {
  146. "total": 1,
  147. "max_score": 1,
  148. "hits": [
  149. {
  150. "_index": "test_index",
  151. "_type": "author",
  152. "_id": "Mr_a",
  153. "_nested": {
  154. "field": "books",
  155. "offset": 0
  156. },
  157. "_score": 1
  158. }
  159. ]
  160. }}}}
  161. ]
  162. },
  163. "aggregations": {
  164. "filtered_aggregation": {
  165. "doc_count": 1,
  166. "sub_aggregation_books": {
  167. "doc_count": 2,
  168. "count_formats": {
  169. "doc_count_error_upper_bound": 0,
  170. "sum_other_doc_count": 0,
  171. "buckets": [
  172. {
  173. "key": "ebook",
  174. "doc_count": 1
  175. },
  176. {
  177. "key": "hardcover",
  178. "doc_count": 1
  179. }
  180. ]
  181. }}}}}
  182.  
  183. Here's java code for some trials I have made:
  184.  
  185. AbstractAggregationBuilder aggregation = AggregationBuilders.filter("filtered_aggregation")
  186. .filter(FilterBuilders.nestedFilter("books", FilterBuilders.nestedFilter("books.chapters",
  187. FilterBuilders.termFilter("books.chapters.chapter_type", "epilogue"))))
  188. .subAggregation(AggregationBuilders.nested("sub_aggregation_books").path("books")
  189. .subAggregation(AggregationBuilders.terms("count_formats").field("books.format")));
  190.  
  191. AbstractAggregationBuilder aggregation2 =
  192. AggregationBuilders.nested("nested_in_books").path("books")
  193. .subAggregation(AggregationBuilders.nested("nested_in_chapters").path("books.chapters")
  194. .subAggregation(AggregationBuilders.filter("filtered_aggregations")
  195. .filter(FilterBuilders.termFilter("books.chapters.chapter_type", "epilogue"))
  196. .subAggregation(AggregationBuilders.terms("count_formats").field(
  197. "books.chapters.chapter_type"))));
  198.  
  199. AbstractAggregationBuilder aggregation3 =
  200. AggregationBuilders.nested("nested_in_books").path("books")
  201. .subAggregation(AggregationBuilders.filter("filter_on_books")
  202. .filter(FilterBuilders.nestedFilter("books.chapters",
  203. FilterBuilders.termFilter("books.chapters.chapter_type","epilogue")))
  204. .subAggregation(AggregationBuilders.terms("count_formats").field("books.format")));
  205.  
  206. Obviously there's something about nested aggregations and filter aggregations that I'm not getting right... can somebody help me?
  207.  
  208. Thanks!
Advertisement
Add Comment
Please, Sign In to add comment