policy report
Published by Convergence Analysis, this series is designed to be a primer for policymakers, researchers, and individuals seeking to develop a high-level overview of the current state of AI regulation.
AI Model Registries
What are model registries? Why do they matter?
Model registries, in the context of AI regulation, are centralized governance databases of AI models intended to track and monitor AI systems in real-world use. These registries typically mandate the submission of a new algorithm or AI model to a governmental body prior to public release.
Such registries will usually require basic information about each model, such as their purpose or primary functions, their computational size, and features of their underlying algorithms. In certain cases, they may request more detailed information, such as the model’s performance under particular benchmarks, a description of potential risks or hazards that could be caused by the model, or even certification that they have passed safety assessments designed to prove that the model will not cause harm.
Model registries allow governmental bodies to keep track of the AI industry, providing an overview of key models currently available to the public. Such registries also function as a foundational tool for AI governance – enabling future legislation targeted at specific AI models.
These registries adhere to the governance model of “models as a point of entry”, allowing governments to focus their regulations on individual AI models rather than regulating the entire corporation, access to compute resources, or creating targeted regulations for specific algorithmic use cases.
As these model registries are an emerging form of AI governance with no direct precedents, the requirements, methods of reporting, and thresholds vary wildly between implementations. Some registries may be publicly accessible, providing greater accountability and transparency, whereas others may be limited to regulatory use only (e.g. when model data contains sensitive or dangerous information). Some may enforce reporting of certain classes of AI algorithms (such as China), whereas others may only require leading AI models with high compute requirements (such as the US).
What are some precedents for mandatory government registries?
While algorithm and AI model registries are a new domain, many precedent policies exist for tracking the development and public release of novel public products. For example, reporting requirements for pharmaceuticals is a well-established and regulated process, as monitored by the Food and Drug Administration (FDA) in the US and the European Medicines Agency (EMA) in the EU. Such registries typically require:
Many of these structural requirements will transfer over directly to model reporting, including a focus on transparent reporting, pre-deployment safety testing by unbiased third-parties, and postmarket surveillance.
What are current regulatory policies around model registries?
China
The People’s Republic of China (PRC) announced the earliest and still the most comprehensive algorithm registry requirements in 2021, as part of its Algorithmic Recommendation Provisions. It has gone on to extend the scope of this registry, as its subsequent regulations covering deep synthesis and generative AI also require developers to register their AI models.
The EU
Via the EU AI Act, the EU has opted to categorize AI systems into tiers of risk by their use cases, notably splitting permitted AI systems into high-risk and limited-risk categorizations. In particular, it requires that high-risk AI systems must be entered into an EU database for tracking.
The US
The US has chosen to actively pursue “compute governance as an entry point” –- that is, it focuses on categorizing and regulating AI models by the compute power necessary to train them, rather than by the use-case of the AI model.