MiMo 7B: Xiaomi’s Open Source AI Is A Tough New Rival To ChatGPT

Xiaomi has officially entered the artificial intelligence space with the release of its first open-source reasoning model, MiMo 7B. Developed by the company’s newly established Big Model Core Team, MiMo represents a strategic shift from Xiaomi’s traditional hardware focus to advanced AI research and development.

Despite having just 7 billion parameters, MiMo has demonstrated strong performance in complex reasoning tasks, surpassing much larger models like OpenAI’s o1-mini and Alibaba’s 32B QwQ-Preview. This performance has placed Xiaomi in the spotlight for producing a highly efficient and compact model capable of rivaling more resource-intensive competitors.

Benchmark Performance and Design Strategy

MiMo 7B has outperformed its peers on benchmarks such as AIME 24-25 (a test for mathematical reasoning) and LiveCodeBench v5 (a programming challenge dataset). This success is attributed to a well-structured development process consisting of both pre-training and post-training innovations.

Pre-training Strategies Included:

• Rich Reasoning Corpus: Focused on extracting and integrating complex reasoning knowledge.
• Synthetic Data Generation: Produced approximately 200 billion tokens of expert-level reasoning data to improve training depth.
• Progressive Difficulty Training: Employed a three-phase training method with increasing levels of difficulty.
• Extensive Token Exposure: Trained across a vast 25 trillion tokens, ensuring comprehensive learning.

Post-training Enhancements:

• Test Difficulty-Driven Rewards: Introduced a novel approach for addressing reward sparsity in complex algorithmic tasks.
• Data Re-sampling Techniques: Applied to stabilize reinforcement learning processes.
• Seamless Rollout System: Increased training efficiency by 2.29 times and validation speed by 1.96 times, streamlining the reinforcement learning (RL) pipeline.

A Broader Vision for AI

The release of MiMo underscores Xiaomi’s broader ambitions in the AI sector. While previously known for consumer electronics and smart devices, Xiaomi is now positioning itself as a serious player in AI model development. By open-sourcing MiMo, Xiaomi is contributing to the collaborative AI research ecosystem, fostering innovation beyond proprietary development.

Developers and AI researchers can now access MiMo 7B and its full technical documentation via Xiaomi’s official Hugging Face repository, offering a valuable tool for further experimentation and development. This move reflects Xiaomi’s intent to build a strong presence in AI while supporting the open-source community.

Filed in Cellphones >Computers >Robots. Read more about AI (Artificial Intelligence), ChatGPT and Xiaomi.