As competition among artificial intelligence (AI) chatbots intensifies, Chinese researchers are making significant strides in developing Chinese-language AI models. Leading the pack is ChatGLM, which rivals ChatGPT in many capabilities and even outperforms it in Chinese, according to its developers.
“Basically, ChatGLM is a ChatGPT alternative,” said Jie Tang, a computer scientist at Tsinghua University in Beijing, at the International Conference on Learning Representations (ICLR 2024) in Vienna on May 9.
Since OpenAI released ChatGPT in November 2022, the enthusiasm for large language models (LLMs) has surged globally. Tech giants, start-ups, and universities worldwide are now developing LLMs that produce human-like text responses. While many of these models, including ChatGPT, primarily operate in English, ChatGLM is designed to be bilingual, functioning seamlessly in both Chinese and English.
“It’s one of the star models in China,” says Wang Yu, a computational biologist at Peng Cheng Laboratory in Shenzhen, China.
Developed by Tsinghua University and its spin-off company Zhipu AI, ChatGLM and its underlying model, the General Language Model (GLM), are the product of extensive collaboration. Over 700 researchers and engineers at Zhipu AI and around 100 students at Tsinghua University are involved in the project, which has caught the attention of the international research community.
The Need for Chinese-Specific AI
ChatGPT is not available in China, but the drive to develop local alternatives goes beyond availability. Chinese-oriented LLMs, like ChatGLM, are tailored to reflect the unique needs and preferences of Chinese users, providing relevant financial, educational, and cultural information. “Models tailored to different languages avoid oversimplifying or neglecting the specific characteristics of certain languages and cultures,” says Adina Yakefu of Hugging Face.
ChatGLM's developers trained it specifically on Chinese datasets, using feedback from native speakers to optimize its conversational abilities. However, challenges remain, such as the lack of publicly available Chinese data and the complexity of tokenization in the Chinese language.
At ICLR 2024, Zhipu AI shared data showing that the latest version of their model, GLM-4, scores within 90% of OpenAI’s GPT-4 on various benchmarks. This includes tests of general knowledge, common sense, and mathematics, where ChatGLM also surpasses GPT-4 in Chinese language optimization.
The Broader Landscape of Chinese LLMs
A public version of ChatGLM is available online, with interfaces in both Chinese and English. While some GLM products, like GLM-130B and ChatGLM-6b, are open source, the inner workings of more advanced models remain closed, similar to GPT-4.
China is witnessing a boom in LLM development, with over 100 AI language models released in 2023 alone. Major tech companies like Baidu and Alibaba are also entering the fray with their own chatbots. These developments occur under strict regulations designed for generative AI systems, emphasizing transparency, reliability, and adherence to core socialist values.
The Future of AI in China
Jie Tang envisions making ChatGLM and GLM-4 even more advanced, aiming towards artificial general intelligence (AGI) — AI with human-level capabilities across a wide range of tasks. While the pursuit of AGI is a contentious and highly debated topic, Tang remains hopeful that his team can lead the way.
Beyond AGI, AI holds the promise of addressing significant global challenges, such as climate change and pandemic prevention. With substantial investments in AI infrastructure and expertise, China is positioning itself to leverage AI for industrial optimization and broader societal benefits.
As Wang Yu states, “We think we have a really good chance to optimize our whole industry with AI — and to do it well. It is not only of benefit to the Chinese. If you can reduce the use of energy and emission of CO2, it’s good for everyone.”
