OpenAI o1 System Card: Safety Evaluations for o1-preview and o1-mini Models

OpenAI has published a detailed safety report outlining the evaluations conducted prior to releasing the OpenAI o1-preview and o1-mini models. This report covers external red teaming efforts and frontier risk evaluations, all conducted in alignment with OpenAI’s Preparedness Framework.

Comprehensive Safety Evaluations

Before deploying any new model in ChatGPT or the API, OpenAI rigorously assesses potential risks and implements safeguards. In conjunction with the release of the OpenAI o1 System Card, the company is sharing its Preparedness Framework scorecard to offer a transparent and thorough evaluation of safety risks and the measures taken to address them. These assessments aim to tackle current safety challenges and manage frontier risks effectively.

Building on the lessons learned from previous model releases, OpenAI has paid particular attention to o1‘s advanced reasoning capabilities. Evaluations measured risks related to disallowed content, demographic fairness, hallucination, and potentially dangerous capabilities. In response to these findings, safeguards like blocklists and safety classifiers have been applied at both the model and system levels to mitigate these risks.

Enhanced Safety Through Advanced Reasoning

One key finding from these assessments is that o1’s advanced reasoning helps the model apply safety rules more effectively. This makes it more resilient to generating harmful or inappropriate content. Based on the Preparedness Framework, o1 has been assigned an overall “medium” risk rating and is deemed safe to deploy. Its risk profile includes:

• “Low” risk levels for Cybersecurity and Model Autonomy
• “Medium” risk levels for CBRN (Chemical, Biological, Radiological, and Nuclear risks) and Persuasion

The OpenAI Safety Advisory Group, the Safety & Security Committee, and the OpenAI Board have thoroughly reviewed these safety and security measures, approving o1 for release after an in-depth Preparedness evaluation.

The Role of Chain-of-Thought Reasoning

The o1 model series utilizes large-scale reinforcement learning to enhance reasoning through a process known as chain-of-thought reasoning. This allows the models to apply safety policies within the context of a conversation, leading to better performance on benchmarks designed to test against illicit content generation, stereotypical responses, and known jailbreaks. However, this increased reasoning capability also improves performance in dual-use scenarios, where heightened intelligence can raise new risks.

As a result, OpenAI acknowledges the need for robust alignment methods and continued stress-testing to ensure safety remains a priority. This ongoing risk management is critical to balancing the model’s reasoning capabilities with its safety performance.

This report offers a full overview of the pre-deployment assessments for OpenAI o1-preview and o1-mini, ensuring the models meet the necessary safety standards through comprehensive evaluations and external reviews.

Read more here