50 Shades of BLEU

alvas@ubi:~/git/mosesdecoder/scripts/training/propy$ cat hyp.txt
It is a guide to action which ensures that the military always obeys the commands of the party
he read the book because he was interested in world history
alvas@ubi:~/git/mosesdecoder/scripts/training/propy$ cat ref1.txt
It is a guide to action that ensures that the military will forever heed Party commands
he was interested in world history because he read the book
alvas@ubi:~/git/mosesdecoder/scripts/training/propy$ python3 pybleu.py -r ref1.txt -t hyp.txt -w '0.25 0.25 0.25 0.25' --smooth 0 --segment-level
0.41180376356915777
0.7400828044922853
BLEU = 48.02, 316/5/63/91/2/147/5 (BP=1.0, ratio=1.074, hyp_len=29, ref_len=27)
alvas@ubi:~/git/mosesdecoder/scripts/training/propy$ python3 pybleu.py -r ref1.txt -t hyp.txt -w '0.25 0.25 0.25 0.25' --smooth 1 --segment-level
0.41180376356915777
0.7400828044922853
BLEU = 48.02, 316/5/63/91/2/147/5 (BP=1.0, ratio=1.074, hyp_len=29, ref_len=27)
alvas@ubi:~/git/mosesdecoder/scripts/training/propy$ python3 pybleu.py -r ref1.txt -t hyp.txt -w '0.25 0.25 0.25 0.25' --smooth 2 --segment-level
0.45763189808605753
0.8210967436686386
BLEU = 48.02, 316/5/63/91/2/147/5 (BP=1.0, ratio=1.074, hyp_len=29, ref_len=27)
alvas@ubi:~/git/mosesdecoder/scripts/training/propy$ python3 pybleu.py -r ref1.txt -t hyp.txt -w '0.25 0.25 0.25 0.25' --smooth 3 --segment-level
0.41180376356915777
0.7400828044922853
BLEU = 48.02, 316/5/63/91/2/147/5 (BP=1.0, ratio=1.074, hyp_len=29, ref_len=27)
alvas@ubi:~/git/mosesdecoder/scripts/training/propy$ python3 pybleu.py -r ref1.txt -t hyp.txt -w '0.25 0.25 0.25 0.25' --smooth 4 --segment-level
0.41180376356915777
0.7400828044922853
BLEU = 48.02, 316/5/63/91/2/147/5 (BP=1.0, ratio=1.074, hyp_len=29, ref_len=27)
alvas@ubi:~/git/mosesdecoder/scripts/training/propy$ python3 pybleu.py -r ref1.txt -t hyp.txt -w '0.25 0.25 0.25 0.25' --smooth 5 --segment-level
0.4905328138015114
0.8132249827094292
BLEU = 48.02, 316/5/63/91/2/147/5 (BP=1.0, ratio=1.074, hyp_len=29, ref_len=27)
alvas@ubi:~/git/mosesdecoder/scripts/training/propy$ python3 pybleu.py -r ref1.txt -t hyp.txt -w '0.25 0.25 0.25 0.25' --smooth 6 --segment-level
0.18015078767591494
0.3961311151812937
BLEU = 48.02, 316/5/63/91/2/147/5 (BP=1.0, ratio=1.074, hyp_len=29, ref_len=27)
alvas@ubi:~/git/mosesdecoder/scripts/training/propy$ python3 pybleu.py -r ref1.txt -t hyp.txt -w '0.25 0.25 0.25 0.25' --smooth 7 --segment-level
0.4905328138015114
0.8132249827094292
BLEU = 48.02, 316/5/63/91/2/147/5 (BP=1.0, ratio=1.074, hyp_len=29, ref_len=27)