LLM: MoE
mixtral-8x7B
https://towardsdatascience.com/run-mixtral-8x7b-on-consumer-hardware-with-expert-offloading-bd3ada394688
Maixtchup: Make Your Own Mixture of Experts with Mergekit