AI evaluation

# Evaluation results taken from:
# https://github.com/EleutherAI/pythia/tree/main/results (Pythia, OPT, BLOOM)
# https://github.com/EleutherAI/gpt-neo#model-evaluations (GPT-Neo 125M, 350M, 1.3B, and 2.7B)
# https://huggingface.co/EleutherAI/gpt-j-6B (GPT-J 6B)
# https://blog.paperspace.com/gpt-neox-20-multi-gpu/#model-evaluation (GPT-NeoX 20B)
# https://blog.eleuther.ai/announcing-20b/ (GPT-NeoX 20B)
# https://huggingface.co/BlinkDL (RWKV-4)
# https://github.com/facebookresearch/llama/blob/main/MODEL_CARD.md (LLaMA)
# https://huggingface.co/cerebras/Cerebras-GPT-6.7B (Cerebras-GPT)

# If I've made an error, please let me know. There were some inconsistencies when models were listed on different sites.
#

# BLOOM                 LAMBDA          PIQA            WINOGRANDE
560M:                   34.10%          63.66%          50.43%
1.1B:                   42.58%          67.19%          54.70%
1.7B:                   46.23%          68.55%          57.22%
3.0B:                   51.80%          70.78%          58.64%
7.1B:                   57.62%          72.63%          64.40%


# GPT-NEO
125M:                   37.36%          63.06%          50.43%
350M:                   47.27%          65.07%          51.14%
1.3B:                   57.23%          71.11%          55.01%
2.7B:                   62.22%          72.14%          56.50%
6B:                     69.7%           76.5%           64.01%
20B:                    72.09%          77.42%          66.10%

# OPT
125M:                   37.90%          63.00%          50.28%
350M:                   45.16%          64.36%          52.33%
1.3B:                   57.93%          71.71%          59.75%
2.7B:                   63.59%          73.88%          61.01%
6.7B:                   67.71%          76.28%          65.27%
13B:                    68.64%          75.95%          65.19%
30B:                    71.45%          77.58%          68.19%
66B:                    73.90%          78.78%          68.75%

# PYTHIA DEDUPED (zero-shot)
70M:                    24.78%          60.50%          52.64%
160M:                   37.05%          63.60%          51.07%
410M:                   51.10%          67.25%          53.67%
1.0B:                   58.39%          71.06%          55.49%
1.4B:                   59.54%          72.58%          57.30%
2.8B:                   67.15%          74.76%          61.09%
6.9B:                   69.24%          75.52%          63.77%
12B:                    71.01%          76.06%          63.85%

# RWKV-4
169M:                   32.99%          65.07%
430M:                   45.16%          67.52%
1.5B:                   56.43%          72.36%
3B:                     63.94%          73.72%
3B (ctx-len 4096):      63.96%          74.16%
7B:                     67.18%          76.06%
14B:                    71.05%          77.42%          62.98%

# LLAMA
7B:                                     79.8%           70.1%
13B:                                    80.1%           73.0%
33B:                                    82.3%           76.0%
65B:                                    82.8%           77.0%

# CEREBRAS-GPT
111M:                   19.4%           59.4%           48.8%
256M:                   29.3%           61.3%           51.1%
590M:                   36.6%           62.7%           49.8%
1.3B:                   46.2%           66.4%           52.1%
2.7B:                   56.7%           70.1%           55.9%
6.7B:                   63.6%           73.9%           60.2%
13B:                    69.6%           76.6%           64.6%

# If you want to further understand what these benchmarks represent:
# Lambada: https://zenodo.org/record/2630551#.YFJVaWT7S_w
# PIQA: https://yonatanbisk.com/piqa/
# Winogrande: https://winogrande.allenai.org/