Sarvam AI, an Indian startup, introduces OpenHathi-Hi-0.1, marking the debut of an open-source Hindi language model. This release kicks off a series aiming to foster innovation in Indian language AI by contributing open models and datasets to the ecosystem.
Utilizing Meta AI’s Llama 2-7B model as its foundation, Sarvam AI’s blog asserts that the model matches GPT-3.5 for Indic languages. The challenge of tokenization, especially costly in Hindi due to limited training text, was addressed through cost-effective methods during the model’s two-phase training.
Testing encompassed various benchmarks, including standard metrics like translation and novel assessments such as toxicity and text classification. The base model is now accessible on the Hugging Face platform for developers to fine-tune and apply it to specific use cases.
Co-founders Pratyush Kumar and Vivek Raghavan, previously engaged with AI4Bharat, collaborated with them, leveraging language resources and benchmarks to train OpenHathi.
With approximately 18 employees, Sarvam AI aims to develop extensive language models with voice as a universal interface, tailored to the diverse demands of the Indian market.
Securing $41 million in Series A funding led by Lightspeed Ventures, with contributions from Peak XV and Khosla Ventures, the five-month-old startup continues its stride.
Alongside, the company is actively crafting enterprise-grade models on its comprehensive Generative AI platform, slated for future release.