Indian colleges accelerate work on Indic language Gen AI

Generative AI platforms have been a craze since the second half of last year, and Microsoft and Google have pushed these programs into their existing services. Even the Ministry of Electronics and Information Technology (MeitY) said on February 3 that it is “cognizant” about the emergence and spread of generative AI and noted that AI could be a “kinetic enabler” for growth in India .

However, researchers at the institutes outline several challenges for generative AI projects in academia, the biggest of which lies in sourcing enough data for Indian languages, the cost of such projects, and the scale of computing power required. Indian researchers have been working on such projects for more than three years.

Tapas Kumar Mishra, assistant professor of computer science engineering at National, said, “In academia, we are using techniques from language models called transformer architectures for various tasks such as classification of data, answering questions, machine translation and building chatbots.” Are.” Institute of Technology (NIT), Rourkela.

Transformer AI is the underlying algorithm for model generative AI tools. They can process conversational human language input and generate output after understanding the context. While the global platforms work mostly in English, Mishra said researchers under him are working on languages ​​like Hindi, Bangla and Kannada, building models that can take queries and generate outputs in these languages . In English. They are not using OpenAI’s tools for this, but have received a “very good” score according to the industry standard Bilingual Evaluation Understudy (BLEU) test.

He said that NIT Rourkela has scored between 25 and 30 marks in Hindi to English and 19 marks in Bangla to English. For reference, OpenAI’s GPT-4 model has a score of 22.9 in English to French output. The institute last month published a research paper on translation from Hindi to English in collaboration with the Association for Computing Machinery – an American scientific academic community that publishes research work on natural language processing (NLP).

NIT Rourkela is not alone in doing so. Students of the Indian Institute of Technology (IIT) Madras have also taken up such projects. Harish Guruprasad, assistant professor of computer science engineering at IIT Madras, said one such project involves “better translated YouTube videos in Tamil”.

“The students mostly adopted it to compare their own research language models with GPT-4, and eventually published a paper on new ways to translate videos into Indian languages,” he said. languages.

For example, Debanga Raj Neg, assistant professor, data science and AI at IIT Guwahati, said the institute is currently working on “creating an economical visual animation model that studies eye and facial movements from open-source visual databases”. And uses it to replicate it.” process.” IIT Guwahati is also working on a research paper on this.

Professor Mausam, founding head of the Yardi School of Artificial Intelligence at IIT Delhi, said that in 2022, he along with Anup Krishnan, associate professor and a team of students, created a language model called ‘MatSciBert’ – specifically for the field of materials science Research. “The ultimate goal is to discover new materials with the help of AI. The first step is to process scientific articles and extract knowledge about the materials and their properties from them. We developed MatSciBert in 2022 – a language model that is more efficient at reading materials science papers more effectively than other common language models such as BERT. MatSciBert has been downloaded nearly 100,000 times in the last year and has been found useful by many groups around the world for a variety of materials science tasks,” said Weathers, who goes by one name.

However the major problem for most researchers is computing power. NIT Rourkela has 13 machines with 24 GB Graphic Processing Unit (GPU). Weathers said the scale of compute power required is “excessive and prohibitive”.

“For example, a training run of GPT-3 would cost $4.6 million, not accounting for any errors during training and re-testing. Any academic institution or any Indian company other than top tech firms regularly cannot train such large models. It is premature to train India-specific language models, unless we create large-scale computer infrastructure in the country,” said Mausam of IIT Delhi.

A senior executive, who previously worked on government technology projects, said on condition of anonymity that “Access to India’s supercomputer infrastructure owned by the MeitY-backed Center for the Development of Advanced Computing (C–) There is a lack of clarity in terms of enabling DAC.” India’s supercomputing power also lags far behind global systems, Mint reported on July 6 last year.

The executive said several top institutes, including IIT Delhi, have been consulted to use the infrastructure for their research initiatives, but not much progress has been made in this regard.

Data availability is another problem for India. For example, NIT Rourkela uses various public datasets, such as the parallel database released by IIT Madras. “It included the following resource language pairs of Indian languages. We are also using our own dataset by scraping newspapers and converting them into different languages ​​- and then working on it. We are also using publicly available data, such as state government-supported local language data repositories,” Mishra said.

To accelerate AI research in India, MeitY in May last year launched ‘Bhasini’ – an Indic language database that can be tapped by institutions.

However, access to the scale of data required for such projects remains an issue. “Transformer architectures can generate great translation efficiency when a large amount of data is available in a language. But, working with small amounts of data is difficult. For example, translating from Oriya to Hindi, such The models are not very efficient,” said Guruprasad of IIT Madras.