AI Policy Primer (October 2024)

Issue #15: Data governance, legal services, and agent communication

Visualising AI by Google DeepMind

In the October edition of the AI Policy Primer, we look at research exploring a shift towards data governance and away from compute governance, how AI is being used in the legal sector, and new work outlining a communication protocol for AI agents. As usual, leave a comment or let us know if you have any feedback at aipolicyperspectives@google.com.

What we’re reading 

From compute governance to data governance   

• What happened: A team at Berkeley announced a new initiative, and an accompanying paper, which calls for AI governance efforts to shift away from relying on ‘compute’ to identify a ‘frontier’ or risky AI model, and towards approaches that centre ‘data’ as well. 

• What’s interesting: 

  • Several AI governance initiatives, such as the EU AI Act and the US Executive Order on AI, use some measure of compute, such as total parameter count and/or FLOPs, to identify the most powerful ‘frontier’ AI models. These models are in turn subject to various governance measures and safety assessments. Other forms of ‘AI compute governance’ include export controls on certain chips. 

  • The Berkeley team argue’s that relying on compute to gauge the risk posed by an AI model is imperfect because advances in efficiency and distributed models of training may ‘decouple’ model performance from computational cost. Some state-of-the-art models are also relatively small. In image segmentation, for example, the authors note that a smaller model by researchers in China outperforms the much larger PaliGemma model from Google on the RefCOCO dataset. In AI biology, smaller models like AlphaFold outperform much larger models like ESM-3 on tasks like protein structure prediction. 

  • The authors also argue that it is increasingly the quality and use of data, both training data and deployment data (such as prompts or RAG documents) that determine model performance. They propose that it will be the combination of models exposed to specific datasets that will lead to risk, rather than models alone. In response, the group suggests making better use of existing data regulations to respond to risks posed by AI, and calls on AI labs to invest more in techniques to evaluate, red-team, and filter datasets – including to assess the incremental uplift they provide to models. 

• Looking ahead: Compute will remain the primary metric that policymakers use to identify the most powerful new AI models. But as datasets become more important, we may see new efforts to tweak existing regulations as well as more market solutions – such as new types of AI licensing regimes. 

Sector spotlight

AI and legal services

• What happened: AI continues to impact different services. One of the most interesting of these is the legal profession, where AI increasingly is becoming a key tool for lawyers and paralegals. On July 29th 2024 the American Bar Association published Formal Opinion 512, which lays out the ground rules for lawyers who want to use generative AI. This is representative of a key trend in which various professions carefully consider the technology, issue guidance and come together around sector norms. 

• What’s interesting: 

  • The opinion addresses six key areas: competence (lawyers must understand AI's capabilities and limitations and independently verify its output); confidentiality (lawyers must evaluate risks of disclosure and obtain informed consent before inputting client information); communication (lawyers must inform clients about AI use in certain circumstances); duties to courts (lawyers must verify AI outputs for accuracy in court submissions); supervision (law firms must establish clear AI policies and ensure proper training); and fees (lawyers must charge reasonable fees for AI use and cannot bill clients for time saved through AI efficiency or for learning to use AI tools). 

  • The opinion acknowledges AI's potential to improve legal service efficiency while emphasising that AI cannot replace professional judgment and that lawyers remain fully responsible for their work product. It also recognises that AI technology is rapidly evolving and that further guidance may be needed as these tools develop. 

• Looking ahead: Formal Opinion 512 is an example of how professional social norms regulate the use of artificial intelligence. This kind of regulation usually gets much less attention than legislation, but remains one of the most important tools for a society adapting to the complexity that new technology introduces in different fields. The legal industry will become a core arena for shaping AI regulation alongside many other high-risk areas.  

What we’re reading 

Agora: Enabling agent collaboration

• What happened: 

  • Building collaborative AI systems where multiple LLMs specialise in different tasks is challenging. Imagine trying to coordinate a team where everyone speaks a different language and with different cultural expectations – a similar issue arises when diverse LLMs attempt to interact. To study this problem, researchers at Oxford University have introduced Agora, a new communication protocol designed to enable more efficient and scalable collaboration between large language models. 

  • The communication bottleneck described by the authors stems from what they call the ‘agent communication trilemma’, which captures three distinct challenges: agents vary significantly in their architecture and training data (heterogeneity); language models are general-purpose tools, making it impractical to define and pre-program every possible interaction scenario (generality); and agents are computationally expensive (cost). 

• What’s interesting: 

  • Agora aims to break this trilemma, which makes it hard to design cost-effective ways for agents to communicate across different types of scenarios. It employs pre-defined routines, similar to APIs, for common tasks. This greatly reduces the computational overhead compared to relying solely on language models for every interaction. For less frequent communications, Agora leverages structured data like JSON, which seeks to offer a balance between flexibility and efficiency.

  • Only in rare cases, such as unexpected errors or the need for complex negotiation, does Agora fall back on natural language. A key innovation of Agora is the use of “protocol documents” (PDs) – machine-readable descriptions of communication protocols. Agents can share and learn these PDs, allowing them to automatically adapt their communication strategies without human intervention. 

• Looking ahead: We expect the research and production of agents to continue to grow in the coming months and years. As such we anticipate the need for streamlined communication between agents will only grow over time, and this will in turn incentivise the development of specialised protocols like Agora. Which specific protocol will be ultimately adopted by industry remains to be seen.