FT Chinese Column

As large language models (LLM) are being applied to an increasing number of sectors, striking a balance and fostering cooperation between China and the U.S. will be the key development in the forthcoming years.

Since the launch of ChatGPT, LLMs have swiftly become the spotlight in the global technology race. These models not only demonstrate extraordinary capabilities in interpreting dialogue and analogy, processing data, and handling creative tasks, but are also perceived as a crucial step in moving towards “Artificial General Intelligence”, which will possess cognitive and deduction capacities like human beings. LLMs sparked an emerging investment trend worldwide, taking the AI race between China and the U.S. to a new level. According to the “Artificial Intelligence Index Report 2024” published by Stanford University, the U.S. continues to outpace other regions in the development of foundational models, while China ranked top in the number of AI patent applications and successfully granted patents (see 【Note 1】). This reflects that the two countries adopt different paths and strategies when it comes to technological innovation and development. Against this backdrop, a comprehensive understanding and comparison of the latest LLM developments between China and the U.S. not only deciphers patterns and strategies in this competition, but also provides fresh perspectives and new opportunities for future international cooperation.

The multilingual performance of a LLM is an important metric for assessing its global competitiveness. Although ChatGPT excels in English, its capabilities in other languages require further evaluation (see 【Note 2】). Similarly, LLMs created by Chinese teams or originating from Chinese models perform well in local language environments (see 【Note 3】), but are still inadequate when it comes to language tests in English. A comprehensive understanding of the performance of these LLMs in different language environments is of vast importance. For such, we compared the Chinese and English performance of 16 representative LLMs through systemic assessment frameworks in the first half of 2024 (see 【Note 4】 and 【Note 5】). These models originate from tech giants, top universities and AI start-ups from China and the U.S.

In English language testing environment, the GPT-4 Turbo ranked first thanks to its natural language processing and subject matter expertise, followed by Gemini Pro and Llama 2. China’s ERNIE Bot 4 is the best performing China-made model when it comes to English language test, ranking fifth overall, slightly above Claude 2 and GPT-3.5 Turbo, but still unable to surpass GPT-4 (see 【Note 5】). In the Chinese language testing environment, ERNIE Bot 4 outperforms GPT-4 Turbo, ranking first and has the best overall performance (see 【Note 4】). Overall, China’s models outperformed other models in the Chinese language environment, but still have room for improvement in performance in other language environments.

As LLM technology continues to mature, LLMs are rapidly expanding into multimodal models and cross-disciplinary applications, becoming a new blue ocean for AI development. The multimodal capabilities allow models not only to process text but also to understand and generate images, audio and video content, significantly broadening their application scenarios. For example, OpenAI’s latest GPT-4o can simultaneously process text, audio and video messages, offering new possibilities for augmented reality, intelligent surveillance and auto-pilot systems.

At the same time, the cross-disciplinary applications of LLMs are also accelerating. Microsoft and OpenAI worked together to integrate GPT-4 into office software, helping users to enhance efficiency at work. Baidu’s ERNIE Bot is not only used for search engines but is also widely incorporated into scenarios such as customer services and smart home systems. On the other hand, LLMs in the vertical sector emerge to impress the industry with the latest technology. For instance, the OpenMEDLab2.0 – a medical multimodal model co-founded by the Shanghai Artificial Intelligence Laboratory and Ruijin Hospital — aims at empowering applications in intelligent image diagnosis, virtual surgery and intelligent clinical decision-making, in a bid to create future AI hospitals. These applications not only showcase the diverse potential of LLMs, but also give rise to a stronger demand for high-performing and high-security AI models.

Looking ahead, the future development of LLMs will focus on further deepening their multi-modality, expanding cross-disciplinary application, enhancing security and ethical responsibility. Currently, the U.S. has a clear advantage in developing new fundamental technologies and innovative applications. Its models often excel at the technological forefront. Meanwhile, China’s models place greater emphasis on optimising their local language environment and their adaptability to practical applications. As LLMs are applied to an increasing number of fields, finding a balance between competition in both countries and promoting collaboration will become a key to development in the coming years.

The competition and cooperation between China and the U.S. not only influence their respective technological ecosystems, but also create long-lasting impacts on the development of the global AI industry. Hong Kong, with its unique international background, and advantages in finance, technology, and location, is poised to become an important bridge in global competition and collaboration. By promoting multi-faceted cooperation in technology, talent and policy, Hong Kong will play a significant role in international research exchanges, technology transfer, and industrial collaboration, leading the exploration of the limitless potential of artificial intelligence.

【Note 1】：https://aiindex.stanford.edu/wp-content/uploads/2024/05/HAI_AI-Index-Report-2024.pdf
【Note 2】：http://arxiv.org/abs/2302.04023
【Note 3】：https://cevalbenchmark.com/static/leaderboard_zh.html
【Note 4】：https://www.hkubs.hku.hk/aimodelrankings/report
【Note 5】：https://www.hkubs.hku.hk/aimodelrankings/report/en

Professor Zhenhui Jack Jiang,
Professor of Innovation and Information, HKU Business School

Jiaxin Li
Ph.D. Student in Innovation and Information Management, HKU Business School

This article was also published on September 18, 2024 on the Financial Times’ Chinese website.