AI4Bharat: AI4Bharat releases Hindi LLM ‘Airavata’

Airavata (Sanskrit word for ‘elephant’) was created using SarvamAI’s foundational Hindi model OpenHathi with diverse, instruction-tuning Hindi datasets to make it better suited for assistive tasks, the research wing said in a blogpost.

Also read |

“We release Airavata, an open-source instruction tuned model for Hindi that shows encouraging performance on a wide range of tasks compared to other open-source models,” it said.

“This is a first step towards building high-quality open-source LLMs for Indian languages that encompass large pre-training datasets, diverse instruction tuning datasets and high-quality models.”

SarvamAI’s OpenHathi is an extension of Meta’s Llama2-7B model and boasts GPT-3.5-like performance for Indic languages.

Discover the stories of your interest

“Currently, Airavata supports Hindi, but we plan to expand this to all 22 scheduled Indic languages,” it said.Given that there is scarcity of Hindi training sets, AI4Bharat has developed its model by translating well-constructed English-supervised instruction-tuning datasets into Hindi. Along with the model, AI4Bharat has also shared the instruction tuning datasets to enable further research for IndicLLMs.

“We rely on human-curated, license-friendly instruction-tuned datasets to build ‘Airavata’. We do not use data generated from proprietary models like GPT-4 etc. We think this is a more sustainable way of building instruction-tuned models at scale for most Indic languages, where relying on distilled data from commercial models would increase costs and restrict their free usage in downstream applications due to licensing restrictions,” it said.

Some of the data sources used to build Airavata include 27k articles curated from Wikihow. It also uses Anudesh, a crowd-sourced collection of prompts accompanied by responses generated from the Llama-2 70B model.

AI4Bharat has also highlighted the challenges and limitations related to its model.

“These include a possibility for hallucination, leading to fabricated information, and may struggle with accuracy in complex or specialized topics. There’s also a risk of producing objectionable or biased content. Its grasp of cultural subtleties and effectiveness in mixed-language situations may be limited,” it said.

Airavata (Sanskrit word for ‘elephant’) was created using SarvamAI’s foundational Hindi model OpenHathi with diverse, instruction-tuning Hindi datasets to make it better suited for assistive tasks, the research wing said in a blogpost.

Also read |

“We release Airavata, an open-source instruction tuned model for Hindi that shows encouraging performance on a wide range of tasks compared to other open-source models,” it said.

“This is a first step towards building high-quality open-source LLMs for Indian languages that encompass large pre-training datasets, diverse instruction tuning datasets and high-quality models.”

SarvamAI’s OpenHathi is an extension of Meta’s Llama2-7B model and boasts GPT-3.5-like performance for Indic languages.

Discover the stories of your interest

“Currently, Airavata supports Hindi, but we plan to expand this to all 22 scheduled Indic languages,” it said.Given that there is scarcity of Hindi training sets, AI4Bharat has developed its model by translating well-constructed English-supervised instruction-tuning datasets into Hindi. Along with the model, AI4Bharat has also shared the instruction tuning datasets to enable further research for IndicLLMs.

“We rely on human-curated, license-friendly instruction-tuned datasets to build ‘Airavata’. We do not use data generated from proprietary models like GPT-4 etc. We think this is a more sustainable way of building instruction-tuned models at scale for most Indic languages, where relying on distilled data from commercial models would increase costs and restrict their free usage in downstream applications due to licensing restrictions,” it said.

Some of the data sources used to build Airavata include 27k articles curated from Wikihow. It also uses Anudesh, a crowd-sourced collection of prompts accompanied by responses generated from the Llama-2 70B model.

AI4Bharat has also highlighted the challenges and limitations related to its model.

“These include a possibility for hallucination, leading to fabricated information, and may struggle with accuracy in complex or specialized topics. There’s also a risk of producing objectionable or biased content. Its grasp of cultural subtleties and effectiveness in mixed-language situations may be limited,” it said.