Evaluating the Reasoning Capabilities of Large Language Models in Chinese-language Contexts

Evaluation of Olympiad Reasoning for Large Language Models in the Chinese Contexts / 中文语境下的大语言模型奥赛推理评测

by Zhenhui(Jack) Jiang¹, Yi Lu¹, Yifan Wu¹, Haozhe Xu², Zhengyu Wu¹, Jiaxin Li¹ / 蒋镇辉¹，鲁艺¹，吴轶凡¹，徐昊哲²，武正昱¹，李佳欣¹
¹HKU Business School，²The School of Management, Xi'an Jiaotong University

The full report can be accessed HERE.

Ranking	Model Name	Correctness	Logical Coherence	Methodological Innovation	Overall Weighted Score
1	GPT-5 (Thinking)	48	47	44	48
2	Gemini 2.5 Pro	48	39	36	44
3	GPT-o3	36	42	39	38
4	Claude 4 Opus (Thinking)	39	36	39	33
5	Gemini 2.5 Flash	35	28	31	32
5	GPT-o4 mini	32	33	33	32
7	Qwen 3 (Thinking)	29	25	28	28
7	Step R1-V-Mini	26	33	22	28
9	GLM-Z1-Air	27	31	22	27
9	SenseChat V6 (Thinking)	27	28	22	27
11	Qwen 3	25	31	17	26
12	Ernie 4.5-Turbo	25	25	19	24
13	Grok 3 (Thinking)	21	28	25	23
14	GPT-5	22	22	28	22
14	DeepSeek-V3	26	14	22	22
16	Claude 4 Opus	22	17	31	21
17	Doubao 1.5 Pro (Thinking)	22	17	22	20
17	DeepSeek-R1	17	25	22	20
19	Grok 3	20	19	17	19
19	Grok 4	19	17	25	19
21	Ernie XL-Turbo	17	19	14	17
21	Hunyuan-T1	17	17	19	17
21	Hunyuan-TurboS	17	17	19	17
21	Kimi-k1.5	17	19	11	17
25	Doubao 1.5 Pro	16	17	19	16
26	GLM-4-plus	12	17	8	13
27	GPT-4o	13	8	19	12
27	Spark 4.0 Ultra	13	11	14	12
29	Baichuan4-Turbo	8	19	11	11
29	GPT-4.1	11	8	17	11
31	Kimi	6	14	27	9
31	Llama 3 3.70B	7	14	6	9
33	Yi-Lightning	6	11	14	8
33	SenseChat V6 Pro	8	8	6	8
35	MiniMax-01	5	11	8	7
35	Step 2	6	8	8	7
35	360 Zhiniao 2-o1	7	6	8	7