Base merges and moerges in the context of Hugging Face refers to a technique used in the field of machine learning, specifically for large language models (LLMs). This technique involves combining two or more LLMs into a single model¹. It’s a relatively new and experimental method that has produced many state-of-the-art models on the Open LLM Leaderboard¹.

• Various, depending on the specific models or datasets being merged.

The term “base merges and moerges” is a category on the LLM Leaderboard for models that have been created using this merging technique. For example, the model palmer-002.5 is listed as the “Best 🤝 base merges and moerges model of around 65B on the leaderboard!.

• Merge Large Language Models with mergekit (huggingface.co)
• Pretrained Model