China Telecom Unveils AI Breakthrough with Domestically Powered TeleChat2-115B Model Amid US Sanctions
October 1, 2024China Telecom's Institute of AI has successfully developed two large language models (LLMs), including TeleChat2-115B, using only domestically produced chips, marking a significant advancement in China's semiconductor technology amid ongoing US sanctions.
This achievement highlights China's growing self-sufficiency in LLM training and represents a new phase in its innovation and self-reliance in artificial intelligence.
TeleChat2-115B features a parameter count of 100 billion, which is lower than other leading models such as Llama and OpenAI's models, but it demonstrates that effectiveness isn't solely determined by parameter size.
The relatively lower parameter count suggests that TeleChat2-115B required less computing power for training compared to its more complex counterparts.
The model was trained on a dataset comprising 10 trillion tokens from high-quality Chinese and English texts, showcasing its potential for diverse applications.
Training was conducted on Huawei's Ascend Atlas 800T A2 training server, which is compatible with Kunpeng processors, indicating a reliance on domestic technology.
China Telecom utilized tens of thousands of domestically produced chips, reportedly from Huawei's Ascend processors, for the training of its models.
This development comes at a time when the US has tightened restrictions on China's access to advanced semiconductors, including Nvidia's AI chips.
Despite these sanctions, China Telecom's progress underscores its capability to advance its AI ambitions through scale and domestic resources.
In comparison to other models, TeleChat2-115B's parameter count of 100 billion is significantly lower than Llama's 400 billion and OpenAI's 200 billion parameters.
In addition to TeleChat2-115B, the institute is also developing an unnamed model with a staggering 1 trillion parameters, further pushing the boundaries of AI technology in China.
China Telecom operates with substantial resources, boasting revenues over $70 billion and serving more than half a billion subscriptions, which enables it to leverage considerable computational power.
Summary based on 4 sources
Get a daily email with more Tech stories
Sources
South China Morning Post • Sep 30, 2024
China Telecom say AI model with 1 trillion parameters trained with Chinese chipsThe Register • Oct 2, 2024
China trains 100-billion-parameter AI model on home grown infrastructureSilicon UK • Sep 30, 2024
China Telecom Trains 1 Trillion-Parameter AI Model Using Domestic ChipsNewsBytes • Oct 2, 2024
China develops 100-billion-parameter AI model using domestic tech