Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- book 6
- book,eraser 2
- book,pen 4
- book,pen,pencil 3
- book,pencil 5
- chalk 3
- chalk,eraser 2
- chalk,eraser,pen 2
- chalk,pen 2
- eraser 6
- eraser,pen 5
- eraser,pen,pencil 2
- eraser,pencil 3
- pen 8
- pen,pencil 5
- pencil 7
- This will be my input to Mapper 1 and Mapper 2
- In Mapper 1 I will be generating rules with this input file
- //OUTPUT
- Rule 1: book => eraser 2
- Rule 0: eraser => book 2
- Rule 1: book => pen 4
- Rule 0: pen => book 4
- Mapper2 emits the same input file
- What calculation I have to do is
- P(A union B)/P(B)
- ie for
- book => eraser = 2/6
- eraser => book 2/6
- book => pen 4/6
- pen => book 4/8
- So my logic is
- in mapper1
- context.write(book,book => eraser 2)
- context.write(eraser,eraser => book 2) etc
- AND
- in mapper 2
- context.write(book,6)
- context.write(book,eraser,2)
- So in Reducer we get
- key -- book
- values---book => eraser 2
- 6
- similary for
- key -- eraser
- values----eraser => book 2
- 6
- So i can clavulate as explained in my Stackoverflow question
- So while doing secondary sorting I have to emit a composite key right?
- so if i am emitting composite key ,I will not be able to get values in reducer as expected
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement