How AI is helping to open a new chapter for ancient Chinese literature

For generations, working with ancient Chinese texts has been a painstaking task. Specialists would spend years deciphering unfamiliar characters, copying manuscripts by hand, adding punctuation to texts written without it, comparing different editions and tracing references scattered across thousands of years of records. But artificial intelligence (AI) is now changing that process, not only by helping to preserve fragile manuscripts but also by transforming how ancient texts are recovered, studied and shared. As China accelerates its efforts to digitise a vast collection of ancient works, universities, technology companies and volunteers are taking part in a nationwide effort to turn the country’s rich literary heritage – once largely the preserve of specialists – into an open digital resource. An example of how this works in practice is Shidian Guji, an online database of more than 70,000 ancient works, all of them freely available to the public, that was launched in 2022 and jointly developed by ByteDance and Peking University. The database provides tools for organising, searching and reading classical works, including modern language versions of classical texts and chatbot-based interactions that help users to explore them. Last month, it published a collection of almost 2,000 traditional scientific works covering fields such as astronomy, mathematics, agriculture and engineering. Sun Xianbin, a researcher at the Institute for the History of Natural Sciences at the Chinese Academy of Sciences, told Shangguan News the digital archive contained centuries of accumulated knowledge and formed part of China’s “indigenous knowledge system”. This concept has gained increasing prominence in recent years, as Beijing tries to promote ideas rooted in China’s own history and intellectual traditions rather than imported Western concepts. The platform forms part of China’s broader push to preserve and pass on its literary heritage, a policy included in the latest five-year plan for national development. Such a push has particular significance in a country where ancient texts are closely tied to concepts of cultural confidence and civilisational continuity stretching across thousands of years. Ancient texts are regarded by Beijing as a cornerstone of this civilisational history, with an estimated 200,000 works surviving in various editions, according to state news agency Xinhua. President Xi Jinping has thrown his full weight behind such efforts, saying technology could help promote the “philosophical ideas, humanistic spirit, values and moral principles” embedded in classical texts and “promote the creative transformation and innovative development” of traditional culture. Last month the National Cultural Heritage Administration published its latest five-year plan. This included a drive to recover cultural relics from overseas and calls to explore how AI could assist in these efforts. For China, AI-driven digitisation offers a way to facilitate the “digital return” of ancient manuscripts from overseas. Millions of volumes are now kept in collections in neighbouring countries such as Japan and South Korea, while others are kept in North America – partly as a consequence of war and social upheaval. A number of projects have been set up to make these works available to the general public, including a partnership between Sichuan University and the University of California, Berkeley, to digitise nearly 10,000 rare pre-1912 titles from the American university’s C.V. Starr East Asian Library. The project, set up in 2021, was funded by the Alibaba Foundation, a non-profit founded by the company that owns the South China Morning Post. Berkeley said the texts were being given a “new digital life” that marked “the next phase in a great journey through time and space”, by providing a trove of easily searchable treasures to scholars, enthusiasts and amateurs alike. “The project is also notable for uniting institutions that are half a world apart for a common cause,” it said. Another such project is run by the Harvard-Yenching Library, which has made digital copies of more than 6,000 rare Chinese books available in its online collection. These include some works that are particularly valuable because no surviving copies exist in mainland China. Many digitised texts already exist online, but they are difficult for non-specialists to read. Tsui Lik-hang, an associate professor from the Department of Chinese and History at City University of Hong Kong, said AI was helping to remove “genuine barriers” for modern readers by adding punctuation, providing glossaries and translating ancient texts. “Much premodern Chinese material is inaccessible not because it is locked away, but because it is unreadable to most people,” he said. Researchers and museums have also begun experimenting with avatars that visitors can question and interact with directly, based either on real historical figures or fictional characters designed to represent a certain time and place. Tsui said the main value of this approach was that it helped stimulate people’s curiosity and imagination, but warned users to be clear about which elements were supported by historical sources and which were “reconstruction or conjecture”. For Tsui, the value of AI lies beyond simply preserving ancient texts. “Preservation often gives you a digital image of a text, but actual scholarly research requires a text you can work with,” he said. The more difficult stage comes after digitisation, he said, because “interpretive operations” were needed to create usable research materials. These operations can include punctuating the text, identifying persons, offices, places and dates, as well as annotating allusions and quotations. Such tasks were “until recently mostly done by hand”, but machine learning has become increasingly useful in assisting with them. Handwritten text recognition was also improving rapidly with AI models, he said. For historians, the use of AI changes “the speed at which familiar questions are answered and the range of questions worth asking”, he said. Tsui said historians had always sensed there were connections between certain authors, texts and different historical periods – but proving them often needed a course of reading and note-taking beyond the ability of even the most dedicated scholar. He said AI tools could help him shift from studying a single written letter to finding patterns across “hundreds of documents by various different authors”. He gave the example of letters written by members of the Song dynasty literary class, saying AI could help researchers trace the hidden links between people and ideas, and identify stylistic similarities across texts and images. But the technology can only help so far in revealing patterns, and Tsui said historians must still do much of the hard work. He also warned that AI-generated historical interactions carried risks. Large language models sometimes made up names, dates or citations, and their “confidence” could make it hard for non-specialists to spot the problems. More importantly, Tsui warned: “Interpretation in the humanities is multiple and contested by nature. An AI model returning one confident reading could flatten that. “Classical Chinese is especially exposed: unpunctuated, unreferenced, deliberately subtle, often obscure and vast in the amounts of texts. Meaning frequently depends on recognising what is being quoted and why.” Tsui also said that AI should not become a substitute for directly engaging with historical sources. “Historical research is … a trade in which scholars find pleasure in reading, interpreting, explaining,” he said. “If we always let a predictive machine sit between ourselves and the sources, we lose that.” Historians, he added, should also play a role in shaping these AI systems, adding that it was wrong to ask whether they could replace historians. “Historians and humanities scholars are trained critics and readers of all kinds of sources. We weigh evidence, argue over meaning and context, and go deep,” he said. “This is precisely what we need in the age of AI models.”

How AI is helping to open a new chapter for ancient Chinese literature

.css-1c6uqr6{color:inherit;font-weight:inherit;font-size:inherit;font-family:inherit;line-height:inherit;overflow-wrap:break-word;}The technology is being used for everything from digitising rare manuscripts to helping break down barriers to understanding classical texts

The technology is being used for everything from digitising rare manuscripts to helping break down barriers to understanding classical texts