Base merges and moerges in the context of Hugging Face refers to a technique used in the field of machine learning, specifically for large language models (LLMs). This technique involves combining two or more LLMs into a single model¹.??It???s a relatively new and experimental method that has produced many state-of-the-art models on the Open LLM Leaderboard¹.

• Various, depending on the specific models or datasets being merged.

The term ???base merges and moerges??? is a category on the LLM Leaderboard for models that have been created using this merging technique.??For example, the model??palmer-002.5 is listed as the “Best ???? base merges and moerges model of around 65B on the leaderboard!.

• Merge Large Language Models with mergekit (huggingface.co)
• Pretrained Model