====== LLM: MoE ======

  * mixtral-8x7B
    * https://towardsdatascience.com/run-mixtral-8x7b-on-consumer-hardware-with-expert-offloading-bd3ada394688
  * [[https://towardsdatascience.com/maixtchup-make-your-own-mixture-of-experts-with-mergekit-87cc46401587|Maixtchup: Make Your Own Mixture of Experts with Mergekit]]