XunZi-MLLM: a multimodal large language model for ancient text and image recognition. - Zhu, Dongmei; Liu, Chang; Zhao, Xue; Zhao, Zhixiao; Shen, Si; Wang, Dongbo (Digital Scholarship in the Humanities, 2025)
Photocopies of ancient works, as valuable cultural heritage of China, can be digitized through the integration of multimodal large language models (MLLMs). This approach allows for more vivid representations of these historical documents, fostering the preservation and advancement of traditional culture. We propose a MLLM specifically dedicated to the digitization of ancient works photocopies. Specifically, we first use web crawling technology to efficiently gather ancient works data, followed by data cleaning and format conversion to ensure the data were suitable for model training. Next, we construct an unlabeled pre-training dataset and an ancient text dialogue dataset for fine-tuning based on the collected data. Finally, we further pre-train and fine-tune the MiniCPM-v-2.6-chat baseline model, enhancing its understanding of ancient works and its conversational ability. Experimental results indicate that the Xunzi-MiniCPM-v-2.6-chat model surpasses existing models in metrics such as BLEU, ROUGE, accuracy, recall, and F1 score. This model demonstrates strong performance in recognizing ancient texts and images, effectively processing multimodal information.