Tiny LLM
pico-GPT
https://jaykmody.com/blog/gpt-from-scratch/
https://github.com/jaymody/picoGPT
https://www.dipkumar.dev/becoming-the-unbeatable/posts/gpt-kvcache/
minGPT/nanoGPT
https://github.com/karpathy/mingpt
https://github.com/karpathy/nanoGPT
pemtoGPT
https://github.com/keyvank/femtoGPT
nanoT5
https://github.com/PiotrNawrot/nanoT5/tree/main
tiny vicuna 1B
https://blog.stackademic.com/tiny-vicuna-1b-is-the-lightweight-champion-of-the-tiny-models-f05e459b8018
nanoRWKV
https://github.com/BlinkDL/nanoRWKV
https://github.com/BlinkDL/ChatRWKV/blob/main/RWKV_in_150_lines.py
https://github.com/BlinkDL/ChatRWKV/blob/main/RWKV_v5_demo.py
T5
https://medium.com/mlearning-ai/past-present-perfect-flan-t5-stands-the-test-of-time-e1db93ff7846
LaMini
https://medium.com/@fabio.matricardi/lamini-power-when-a-small-guy-can-beat-the-giants-b176d0f7558a
2024-01 TinyLlama: An Open-Source Small Language Model
2024-01 StableLM-2-1.6B
2024-01 Monte Carlo Tree Search for Recipe Generation using GPT-2
2023-12 Speeding up the GPT - KV cache
2023-01 GPT in 60 Lines of NumPy