Skip to main content

Artificial Intelligence Risk Framework

A risk framework addressing societal-level risks due to Artificial Intelligence (AI). This was developed for discussion in the AI Risks chapter of the Risk-First Software Development book.

If you want a much more comprehensive view of what is going on in the world of risk management for AI, please take a look at the MIT AI Risks Database which covers hundreds of different AI Risk Frameworks. One of the reasons for such proliferation is that there are many different domains that are impacted by AI.

Here, we are narrowly considering a broad "society" level domain.

Outcomes​

  • Understand the main risks we face as society nurturing AI.
  • Understand which risks can be managed (i.e. have mitigating practices), which perhaps can't.

Risks​

Loss Of Diversity

A single AI system dominates globally, leading to catastrophic consequences if it fails or contains errors.

Emergent Behaviour

AI develops unforeseen behaviours, capabilities, or self-replication that could lead to unpredictable consequences.

Unintended Cascading Failures

AI interacting with critical systems (finance, infrastructure, etc.) may trigger global-scale unintended consequences.

Loss Of Human Control

AI systems operating autonomously with minimal human oversight can lead to scenarios where we cannot override or re-align them with human values.

Practices​

AI As Judge

Using the outputs of one (trained) AI to measure the performance of another

Ecosystem Diversity

Encouraging the development of multiple, independent AI models instead of relying on a single dominant system.

Global AI Governance

International cooperation is necessary to prevent AI firms from evading national regulations by relocating to jurisdictions with lax oversight.

Interpretability

Developing tools to analyse AI decision-making processes and detect emergent behaviors before they become risks.

Kill Switch

Fail-safe systems capable of shutting down or isolating AI processes if they exhibit dangerous behaviours.

National AI Regulation

Governments can implement policies that ensure AI-driven firms remain accountable to human oversight.

Public Awareness

Equip citizens with media literacy skills to spot deepfakes and manipulation attempts.

Replication Control

Replication control becomes relevant when an AI system can duplicate itself—or be duplicated—beyond the reach of any central authority.

Transparency

Requiring AI developers to publish model architectures, data sources, generated data and decision-making rationales.

Monitoring

Continuous observation and tracking of a system, team or person, perhaps with respect to performance, security or availability.

Redundancy

Ensuring backup systems are in place to prevent failure.

Security Testing

Ensuring the application is secure by identifying vulnerabilities.