Bengaluru-based startup Sarvam AI has introduced two new large language models, as it expands its role in India’s effort to build domestic AI systems.Speaking at the India AI Impact Summit in Delhi, Sarvam AI Co-founder Pratyush Kumar said the company has trained a 30-billion-parameter model and a 105-billion-parameter model from scratch, using a mixture-of-experts (MoE) architecture to balance scale and efficiency.The MoE architecture boosts efficiency by activating only a subset of the total parameters for each input, rather than the whole model.Kumar said Sarvam had previously developed a 3-billion-parameter dense model but it had to scale further. The 30B model, he explained, activates only 1 billion parameters per token despite having 30 billion parameters in total. He said this reduces inference costs and improves efficiency, particularly for reasoning tasks. The model supports a 32,000-token context window and was trained on 16 trillion tokens.Efficiency, Kumar said, is central to the company’s approach as it aims to build AI that delivers population-scale impact. Sarvam also presented a 105-billion-parameter MoE model that activates 9 billion parameters and supports a 128,000-token context window. Kumar said the system is designed for more complex reasoning and agentic use cases. Kumar said, on most benchmarks, the 105B… Read MoreYourStory RSS Feed
