Xiaomi CEO Lei Jun’s upcoming address approaches and Xiaomi has revealed its latest achievement in artificial intelligence (AI): a large-scale AI model. On August 11, Xiaomi Corporation introduced the MiLM-6B pre-training language model on GitHub, as reported by Titanium Media App. This model boasts a significant parameter capacity of 6.4 billion and has been evaluated in C-Eval and CMMLU benchmarks.
Xiaomi’s MiLM-6B secures the 10th spot in C-Eval’s ranking. It leads its parameter category with an average score of 60.2, outdoing models like Alibaba Cloud Qwen-7B and OpenAI’s ChatGPT (data from May). This closely resembles Anthropic company’s Google-backed Claude 1.0 version.
This marks Xiaomi’s debut in the large-scale model domain and represents their first step into GPT-style large-scale models.
In April this year, Xiaomi formed the Xiaomi AI Lab Large Model Team, headed by Luan Jian and reporting to Wang Bin. This team is vital to Xiaomi’s AI strategy. Wang Bin, with over two decades of NLP (Natural Language Processing) expertise, including three years at Xiaomi, leads the large model team.
Xiaomi’s MiLM-6B AI Model Ranks 10th in C-Eval List
Xiaomi’s CEO, Lei Jun, mentioned the company’s focus on unveiling innovative technologies and products once perfected. Lu Weibing, President of Xiaomi Group, noted that the company’s AI team comprises over 1,200 professionals. While they plan to integrate large-scale models extensively into their operations, their approach will differ from OpenAI’s.
In June of this year, Wang Bin revealed that Xiaomi was crafting a large-scale universal language model without directly releasing a ChatGPT-like product. He emphasized Xiaomi’s initial goal of tens of billions of parameter bases. The envisioned applications for Xiaomi’s large model encompass domains like Xiaoai, loT, autonomous driving, and robotics. These scenarios will offer valuable feedback to enhance the large model’s capabilities.
Four months after its founding, Xiaomi introduced the MiLM-6B large-scale model. This model performed remarkably well in both C-Eval and CMMLU benchmark tests.
In the C-Eval evaluation, MiLM-6B achieved an average score of 60.2. Scores varied from 42 to 71.7 across different subjects and difficulty levels (e.g., STEM, social sciences, humanities). In the CMMLU assessment, MiLM-6B attained an average score of 60.37 and 57.17 in the zero-sample and five-sample tests, indicating substantial knowledge and reasoning capabilities.
Tsinghua University, Shanghai Jiaotong University, and the University of Edinburgh collaboratively developed the C-Eval list, a comprehensive Chinese model evaluation suite. Similarly, CMMLU represents a comprehensive Chinese model benchmark.
Xiaomi will hold the 2023 Lei Jun annual address on August 14. It’s unclear if they’ll reveal MiLM-6B tech during the event. Xiaomi hasn’t responded to MiLM-6B testing and availability queries.
Read Also: Xiaomi Pad 6 Max 14 Launch on August 14 with Snapdragon 8+ Gen1.
