Exploring Meituan’s LongCat-Flash-Lite: A Lightweight MoE Model with N-Gram Embedding

The development of intelligent large-scale models has been a focal point in artificial intelligence, with MoE (Mixture of Experts) architecture leading the way. However, as the number of experts in MoE increases, challenges like diminishing marginal returns and higher system communication costs arise. Meituan's LongCat team has introduced LongCat-Flash-Lite, a groundbreaking model designed to address these issues through N-gram embedding extension and a lightweight approach to MoE architecture.

What is LongCat-Flash-Lite?

LongCat-Flash-Lite is an advanced, lightweight MoE model comprised of 68.5 billion parameters, but with a dynamic activation of only 2.9 to 4.5 billion parameters per inference. By emphasizing N-gram embedding expansion instead of merely increasing the number of experts, this model streamlines efficiency while maintaining high performance.

Key Innovations and Features

🌟 N-gram Embedding for Enhanced Semantic Understanding

One of the core innovations in LongCat-Flash-Lite lies in its N-gram embedding layer, which improves the model's ability to capture local semantic context. This is achieved by mapping sequences of tokens into N-gram embedding vectors using a hash function, effectively reducing semantic misunderstandings. For instance, it distinguishes similar phrases like "open terminal input command" and "open file" to accurately identify the usage context, such as programming versus daily tasks.

🚀 Dynamic Activation and Sparse Efficiency

Unlike traditional MoE models needing to activate all experts for every inference, LongCat-Flash-Lite employs a dynamic activation mechanism that only activates selective parts of the model. This design minimizes computational redundancy and reduces operational costs without compromising model performance.

System-Level Optimizations

To make the most out of its sparse model architecture, LongCat-Flash-Lite comes with several system optimizations:

• Smart Parameter Allocation: Allocating nearly 46% of the parameters (31.4 billion) to N-gram embedding layers enhances efficiency and resolution accuracy while reducing communication overhead.
• N-gram Cache: A specialized caching mechanism inspired by KV Cache directly manages N-gram IDs on GPU devices, significantly lowering input/output latency.
• Optimized CUDA Kernels: Custom CUDA kernels and kernel fusion techniques improve GPU utilization and reduce processing delays.
• Collaborative Speculative Decoding: This allows for the increased batch size and improved inference performance by incorporating a predictive model and avoiding unnecessary computational redundancies.

Performance Highlights ✨

LongCat-Flash-Lite performs exceptionally well across key domains:

1️⃣ Smart Agent Tools Utilization

In VitaBench evaluations, LongCat-Flash-Lite consistently achieves high scores, excelling in diverse industries like telecom (72.8), retail (73.1), and aviation (58.0).

2️⃣ Programming and Code Generation

• Code Repair: It achieves an accuracy of 54.4% on the SWE-Bench benchmark, demonstrating robust troubleshooting skills for bug fixes and feature implementations.
• Terminal Commands: It significantly outperforms other models on the TerminalBench, boasting an impressive 33.75 score in simulating accurate command-line operations for developers.
• Multi-language Competence: LongCat-Flash-Lite scores a remarkable 38.10% accuracy in multi-language code generation tasks, showcasing its adaptability and problem-solving capabilities across programming languages.

3️⃣ General Knowledge and Reasoning

The model delivers top-notch performance on various benchmarks:

• MMLU: 85.52 points, competitive with leading AI models.
• C-Eval and CMMLU: 86.55 and 82.48 points, respectively, indicating strong performance in Chinese-language tasks.
• Mathematics: Achieves 96.8% accuracy in fundamental math problems and excels in competitive-level math challenges, with scores of 72.19 in AIME24 and 63.23 in AIME25.

Applications and Practical Impact

LongCat-Flash-Lite sets a new bar for AI deployment in various real-world applications. Its lightweight design makes it suitable for:

• Long document analysis and summarization.
• Large-scale code analysis for debugging and crafting new algorithms.
• AI-driven automation in industries like e-commerce, telecommunications, and aviation.

Open Ecosystem and Future Collaboration

Meituan has made LongCat-Flash-Lite's weights and technical details freely available for developers globally. The project is open-sourced to foster collaboration and innovation. Explore these resources:

• Hugging Face: LongCat-Flash-Lite on Hugging Face
• Modelscope: LongCat-Flash-Lite on Modelscope
• GitHub: SGLang-FluentLLM on GitHub
• Technical Report: LongCat-Flash-Lite on arXiv
• Official Website: LongCat AI
• API Open Platform: LongCat API Access

Conclusion

Through its innovative N-gram embedding, smart parameter allocation, and advanced system optimization, LongCat-Flash-Lite pushes the boundaries of MoE models. Its ability to combine high efficiency with cutting-edge AI performance makes it a standout solution across industries, from intelligent agents to programming tasks. By making the technology open source, Meituan invites developers globally to collaborate, explore, and deploy innovative solutions based on this exceptional model.