Large-scale model competition heats up, AI chip regulation is difficult to limit development

The International Institute for Strategic Studies (IISS), a well-known research institution, recently released a report stating that although the United States restricts the export of high-performance AI chips to China, these control measures may push Chinese artificial intelligence researchers towards “fields with lower computing requirements” and Guide them to develop a “new competitive advantage”.

In recent years, the capabilities of large language models have improved significantly, and OpenAI’s creation of GPT-3 in 2020 is an important milestone, the report said. These improvements are attributed to the creation of larger, more general model architectures and increases in dataset sizes as well as the amount of money technology companies are spending to increase the computing power to train models. Empirical studies show that there is a strong relationship between dataset size, computational overhead, and parameter count for a given model, and in practice computational overhead is the strongest constraint on model improvement.

Furthermore, large language models are being developed by an increasing number of actors, with diffusion across multiple dimensions. Research on large-scale language models, for example, has largely taken place in the US, but researchers in other countries—notably China, and specific research institutions elsewhere, such as DeepMind in the UK—have devoted significant resources to building their own models. Furthermore, the types of institutions developing language models have expanded to include both large tech companies such as Google and Microsoft, as well as scattered collectives of researchers. The report argues that the proliferation of large model technology has two near-term implications for security: These models may produce higher quality and more content for disinformation, and competition for the development of large models may increase geopolitical tensions.

The report also analyzed last October, when the U.S. government announced new export controls on advanced semiconductor chips flowing to China, in part because they are critical to the development of artificial intelligence. While the stated intent of export controls is to limit the development of artificial intelligence systems for surveillance or military applications, language models are also highly dependent on these advanced semiconductors. Part of the real motivation for imposing export controls may be to preserve the U.S. advantage in language models, either as part of a broader strategy to compete in AI technology or because the government specifically wants to curb Chinese language model development.