Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- # This paste shows that Elasticsearch (at least versions 7.17 and 8.1) ignore minimum_should_match clause in aggregations against rollup indices (using the rollup_search endpoint), despite the official documentation stating otherwise.
- # The paste creates all necessary components and has a clean up section too.
- # Create sample index with data
- PUT cardata
- PUT cardata/_mapping
- {"dynamic":"true","properties":{"amount":{"type":"long"},"vendor":{"type":"keyword"},"currency":{"type":"keyword"},"date":{"type":"date"},"car.original_state":{"type":"keyword"}}}
- ## Put data into the sample index
- PUT cardata/_doc/1
- {"amount":1000,"currency":"USD","vendor":"Ford","date":"2022-04-01T12:10:00Z","car.original_state":"CLEAN"}
- PUT cardata/_doc/2
- {"amount":5000,"name":"John Smith","currency":"USD","vendor":"Chevrolet","date":"2022-04-02T12:10:00Z","car.original_state":"WRECKED"}
- PUT cardata/_doc/3
- {"amount":7500,"currency":"HUF","vendor":"Chrysler","date":"2022-04-03T12:10:00Z","car.original_state":"CLEAN"}
- PUT cardata/_doc/4
- {"amount":65,"currency":"GBP","vendor":"Vauxhall","date":"2022-04-03T12:10:00Z","car.original_state":"WRECKED"}
- PUT cardata/_doc/5
- {"amount":17,"currency":"GBP","vendor":"Rover","date":"2022-04-03T12:10:00Z","car.original_state":"CLEAN"}
- # Create and start rollup jobs, run every minute
- PUT _rollup/job/car_states
- {"index_pattern":"cardata*","rollup_index":"rollup-cardata","cron":"*/1 * * * * ?","page_size":1000,"groups":{"date_histogram":{"field":"date","fixed_interval":"1d"},"terms":{"fields":["currency","amount","car.original_state"]}},"metrics":[{"field":"amount","metrics":["min","max","sum"]}]}
- POST _rollup/job/car_states/_start
- # Query data
- ## Query live index data, should return 1 doc with key GBP and sum value 17.0
- GET cardata/_search?size=0
- {"query":{"bool":{"filter":[{"term":{"currency":"GBP"}}],"should":[{"term":{"car.original_state":"CLEAN"}},{"term":{"car.original_state":"UPGRADED"}}],"minimum_should_match":1}},"aggs":{"currency_agg":{"terms":{"field":"currency"},"aggs":{"amount_agg":{"sum":{"field":"amount"}}}}}}
- ## Query rollup data with minimum_should_match (identical query to the live data query), but somehow returns 2 docs with key GBP and sum value 82.0
- ## *************** THIS IS WHERE YOU CAN SEE THAT minimum_should_match is ignored *****
- ## Query rollup data without minimum_should_match
- GET rollup-cardata/_rollup_search?size=0
- {"query":{"term":{"currency":"GBP"}},"aggs":{"currency_agg":{"terms":{"field":"currency"},"aggs":{"amount_agg":{"sum":{"field":"amount"}}}}}}
- ## Query rollup data with minimum_should_match returns the same result, even though it filters for CLEAN cars paid in GBP. (So should return what the 'live index data' search does.)
- ## It's IDENTICAL to the 'live index data' query.
- GET rollup-cardata/_rollup_search?size=0
- {"query":{"bool":{"filter":[{"term":{"currency":"GBP"}}],"should":[{"term":{"car.original_state":"CLEAN"}},{"term":{"car.original_state":"UPGRADED"}}],"minimum_should_match":1}},"aggs":{"currency_agg":{"terms":{"field":"currency"},"aggs":{"amount_agg":{"sum":{"field":"amount"}}}}}}
- ## *************** ^^^ THIS IS WHERE YOU CAN SEE THAT minimum_should_match is ignored *****
- # Clean up
- ## Clean up rollup job
- POST _rollup/job/car_states/_stop
- DELETE _rollup/job/car_states
- DELETE rollup-cardata
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement