AI Policy Primer (#6)

The Hiroshima process, scientists' views on AI, and the UK AI Safety Summit

Welcome to the latest edition of AI Policy Perspectives: a rundown of the topics that Google DeepMind’s policy team has been reading, thinking about and working on over the past few weeks.

This month's version looks at the G7’s International Code of Conduct, a recent study examining how scientists are using AI, and new approaches to evaluations as the AI Safety Summit gets underway. 

As ever, feedback and questions are very welcome. We’re planning a few updates to AI Policy Perspectives in the future, so make sure to watch this space.

Policymakers taking action

G7 announces International Code of Conduct

• What happened: The G7 this week announced an International Code of Conduct for Organisations Developing Advanced AI Systems as a result of its “Hiroshima Process.” The principles are largely modelled on previously agreed commitments including those companies made at the White House in July – including measures to limit misuse; invest in cybersecurity; and identify vulnerabilities, including through red-teaming. Additional principles focus on sharing risk management policies and implementing controls on models’ data inputs and outputs.

• What’s interesting: The nascency of AI policy is enabling fast progress: from a meeting with four CEOs at the White House in May, to company commitments in July to G7 leaders agreeing an international set of principles by October. That's leaving aside the extensive Executive Order that the White House also published this week. Amidst the supposed demise of multilateralism, the G7 process provides a welcome reminder that governments can collaborate constructively on shared global issues.

• Developing interoperable, international frameworks for AI safety is one of our policy priorities, and we’ve been supportive of more international backing for the practices we committed to the White House. As other fora also consider these topics – such as the upcoming UK AI Safety Summit and discussions at the UN – we will continue sharing our perspective on things like potential new institutions.

• Looking ahead: We could see the OECD and upcoming Italian G7 Presidency collaborate closely in 2024 to operationalise the principles in alignment with the proposed EU AI Act, for which trilogue negotiations are still ongoing. 

Study watch

Scientists share hopes and concerns about AI

• What happened: Two recent Nature surveys shed light on how scientists and postdocs are using AI, including various generative AI tools.

• What’s interesting: Scientists’ use of AI is growing relatively quickly, although it is not yet transforming most practitioners’ research. While 8% of the UK public use generative AI tools at work, a third (31%) of postdocs do so, mainly to refine their writing, debug code, adapt content for different formats (e.g. LaTex), and summarise literature. Looking ahead, scientists hope that using machine learning in their research will enable them to speed up data processing, use new kinds of data, and tackle otherwise prohibitive research problems. Smaller shares of scientists expect AI to directly generate new research hypotheses or make new discoveries. 

• Scientists also worry that GenAI tools will lead to more misinformation, fraud, and inaccuracies, and compound issues relating to biases in datasets - although AI may also help address some of these challenges, such as detecting fraudulent images in papers.  When it comes to using machine learning in their research, scientists worry that over-reliance on pattern recognition may come at the expense of deeper understanding. Although new interpretability research focussed on the chess playing AI system AlphaZero, also suggests that we may one day be able to expand human knowledge by studying how AI systems learn. 

• When asked about obstacles to using AI in their work, scientists placed the lack of skills, training, and funding above a lack of compute and data, but also worried that only a small number of companies and universities could operate at the cutting edge. 

• Looking ahead: In the next 1-2 years, scientists could be among a group of professions, including programmers, whose use of AI will outpace that of the broader labour force. This may provide early insights into the broader benefits and risks of AI. 

What we’re thinking about 

The UK AI Safety Summit

• What happened: At the UK AI Safety Summit, which began yesterday, evaluations and “effective model risk assessments” will be a priority discussion item on day one. In the run up to the Summit, the Frontier Model Task Force announced that it was partnering with Humane Technology’s Rumman Chowdhury to expand its capacity to evaluate societal impacts from AI. 

• What’s interesting: Our recent paper identifies three main types of sociotechnical evaluations of AI safety risks: (a) those that assess a model's capabilities; (b) those that assess risks stemming from how people interact with an AI model; and (c) those that evaluate longer-term societal effects, such as employment or environmental effects, as AI becomes more widely used across society. 

• The paper surveys the current state of AI evaluations and identifies gaps, particularly for non-text modalities, human-AI interaction and societal impact evaluations. This tallies with the efforts of the Frontier Model Task Force, which, although focussed primarily on assessing AI models’ dangerous capabilities, recently partnered with the Collective Intelligence Project to conduct social evaluations of powerful models. 

• Evaluations are challenging to conduct. Arvind Narayanan and Sayash Kapoor recently described evaluating LLMs as a ‘minefield’ due to what they characterised as prompt sensitivity - results depending on prompts rather than model properties;  construct validity - failing to adequately model the real world; and data contamination - when training data isn’t properly separated from test data.  This wide range of challenges highlights how policymakers, AI labs and civil society groups will need to significantly step up AI evaluation efforts and co-design new evaluations approaches. 

• Looking ahead: In the next year, the number of society-focused AI evaluations may increase, but will remain vastly under-studied relative to capability-focused approaches.