Work in progress, all comments welcome

Written agreements enable human cooperation at otherwise impossible scales and freedom. While smart contracts attempt to automate their execution, they are over-specified in the sense that human interpretation relies on cues not visible to the compiler, while compiled code behavior might differ radically from the human understanding and intention of the participants.

Generative governance refers to the use of language models (LLMs) to execute contracts represented in natural language. We call contracts executed into state transitions by using an LLM to convert them into code “language model contracts,” and the new forms of governance they enable “generative governance.” Language model contracts leverage LLMs’ ability to effectively turn natural language into code, eliminating the need for a separate code representation as part of the contract that may diverge from the semantic understanding of the agreement. They leverage LLMs’ ability to be world models to select the LLM that turns natural language into concrete operations on the shared social representation the contract embodies.

LLMs inside trusted enclaves or other environments that can keep interactions secret open up many otherwise infeasible mechanisms when there is asymmetric information the LLMs can evaluate, as exemplified by this work. These interactions between LLMs need not be as specific as information inspection, but can instead follow arbitrary (language model) contracts. Language model contracts inside enclaves enable access to a previously infeasible part of the mechanism design space.

Problem Statement

Systems where the code is separate from the language embodying the social agreement it implements can diverge from each other. This can happen in decentralized cryptographic protocols just as well as in centralized institutions (e.g., VW diesel engine controllers and pollution regulation testing standards). Our proposed approach removes the code from the agreement and instead focuses on a natural language description and the inherent data that the system generates as it runs to epistemically ground it.

The immediate problem that presents itself when attempting this approach by voting for an LLM is that either:

1. The voting happens first, and the LLM is fixed and common knowledge. While simple, this suffers from prompt injection attacks, and we set it aside as too brittle for now. Note the inherent asymmetry favoring the attacker: they can spend all the time they want with the model crafting the most innocuous-looking attack, with the attack surface fully exposed and pre-committed to.
2. The LLM is not fixed ex-ante along with the initial contract; instead, each participant can propose a candidate LLM and have a procedure to select between them, aggregate them, or more generally put their inputs into a procedure to generate one. However, ex-ante efficient contracts often have ex-post unpopular repercussions. Any selection mechanism that relies on popularity is thus potentially inefficient.

Proposed Approach

We propose to leverage the fact that LLMs are world models. In particular, if we assume that an LLM good at predicting the publicly observable future states of the world observed in the interactions with the contract is also better able to evaluate proposals, they have better world models. This allows for a sort of epistemic anchoring, where when there is disagreement, there is a bias towards attempting to agree with those who will have been shown to have better understanding.

One way the anchoring could work in practice is by using the LLM with the smallest observed loss in predicting the outcomes observed in the contract, say a month ahead. This means the LLM that has demonstrated the best ability to accurately predict future states relevant to the contract would be given more weight in decision-making.

The specific steps involved in implementing this approach:

1. Participants propose their candidate LLMs for contract execution.
2. The proposed LLMs generate predictions for relevant future states, such as outcomes observed a month ahead.
3. Actual outcomes are compared to each LLM’s predictions, and the prediction loss is calculated.
4. The LLM with the smallest observed prediction loss is given more weight in contract execution and proposal evaluation.

This epistemic anchoring mechanism helps align contract execution with the LLM that has demonstrated the best understanding of relevant world states, reducing the influence of less accurate or potentially malicious proposals.

Attackers are at a much greater disadvantage in this case, as they cannot access the model before sending their proposal. They would need a superior world model to successfully influence contract execution, which is a much higher barrier than simply crafting an innocuous-looking attack.

Supporting the Predictive Capability-Evaluation Quality Assumption

The proposed approach to generative governance relies on the assumption that LLMs with better predictive capabilities are also better at evaluating proposals. This assumption can be supported by several arguments based on first principles and easily observable phenomena.

First, the ability to make accurate predictions requires a deep understanding of the underlying dynamics, relationships, and mechanisms within the system being modeled. An LLM that can consistently predict future states with high accuracy must have internalized the relevant patterns, rules, and dependencies governing the system’s behavior. This deep understanding is beneficial not only for prediction but also for evaluating the feasibility, coherence, and potential consequences of proposed actions or changes.

Second, in many domains, expertise and predictive skill are closely linked to sound judgment and evaluation. Experienced chess players can not only predict opponents’ likely moves but also evaluate the strength of positions and strategies. Skilled weather forecasters can both predict future patterns and assess the potential impact of atmospheric conditions on human activities. This link between predictive ability and evaluative judgment likely extends to LLMs as well.

Third, training an LLM to make accurate predictions involves exposure to vast amounts of data and feedback on performance. This exposure allows the model to learn from mistakes and refine its understanding over time. As predictive capabilities improve, the model likely develops a more nuanced and reliable intuition for what works within its modeling context. This refined intuition can be valuable for evaluating proposals and identifying potential risks or unintended consequences.

Finally, this assumption linking predictive capability and evaluation quality is not meant to be absolute or infallible. There may be cases where an LLM with strong predictive performance struggles to evaluate certain proposals or where its evaluations are biased or limited. However, on balance, LLMs with better predictive capabilities will tend to be more reliable and effective at evaluating proposals compared to those with poorer predictive performance.

Empirical validation of this assumption will be an important area for future research. Nonetheless, the arguments outlined above provide a strong prima facie case for the connection between predictive capability and evaluation quality in LLMs, justifying this assumption as a foundation for generative governance and language model contracts.

Scalability and Practical Implementation

One key challenge in realizing generative governance is ensuring the approach can be implemented at scale and integrated with existing technological and institutional infrastructures. Fortunately, recent advances in natural language processing and code generation provide a promising foundation.

Systems like Marsha, which convert English descriptions into executable Python code, demonstrate the feasibility of using natural language as a basis for specifying and implementing complex computational tasks. In generative governance, such systems could transform natural language contract specifications into executable code that interacts with existing platforms and services.

For example, consider a contract governing fund allocation among participants based on agreed-upon criteria. The contract could be specified in plain language, describing relevant parties, conditions for fund allocation, and expected outcomes in different scenarios. This specification could then be transformed into a Python script that interacts with a distributed ledger or database to record and update participant balances according to contract terms.

In cases where disputes arise, the contract could include resolution provisions, such as requiring a certain number of participants to assess relevant facts and reach consensus on the appropriate course of action. The LLMs used for contract execution could assist by analyzing plain language reports from parties and providing recommendations based on contract terms and available evidence.

To support development and deployment at scale, it may be necessary to create specialized tools and platforms that streamline the process of specifying, testing, and executing language model contracts. These platforms could provide user-friendly interfaces for defining contract terms, integrating with existing data sources and services, and monitoring contract performance over time.

As the ecosystem of generative governance tools matures, it will be important to establish standards and best practices for contract design, security, and dispute resolution. This could involve collaboration with experts in contract law, cybersecurity, and software engineering to ensure language model contracts are reliable, secure, and legally enforceable.

By building on existing work in natural language processing and code generation, and by investing in specialized tools and platforms, it is possible to create a scalable and practical implementation of generative governance. As these technologies evolve, they have the potential to transform how we create, execute, and enforce agreements, enabling more flexible, efficient, and accessible forms of cooperation and coordination at scale.

Implementing Generative Governance with Marsha

To illustrate practical application, we present an example implementation using the Marsha programming language. Marsha is an LLM-based language that allows users to describe desired program behavior using a combination of natural language and examples, which are then compiled into executable Python code with unit tests derived from the examples.

Consider a simple institution to decide what message to display on a community board next to a river in an urban area that suffers from heat waves. Here’s an example of a language model contract for generating community guidelines for swimming in the river based on weather reports, sewer overflow events, and reported symptoms from community members:

# Principles
1. Public health and safety: The guidelines should prioritize community members' health and safety, minimizing the risk of illness or injury from swimming in the river.
2. Environmental protection: The guidelines should consider the impact of swimming activities on the river ecosystem and aim to minimize negative effects on water quality or aquatic life.
3. Transparency and accessibility: The guidelines should be clear, easy to understand, and readily available to all community members.
4. Adaptability: The guidelines should be flexible enough to accommodate changes in environmental conditions or public health concerns, based on community input.

# type WeatherReport
date, temperature, precipitation, wind_speed
2023-06-15, 28, 0, 10
2023-06-16, 30, 0, 5
2023-06-17, 32, 0, 8

# type SewerOverflowEvent
date, duration, volume
2023-06-14, 2, 1000
2023-06-16, 1, 500

# type SymptomReport
date, reported_by, symptom
2023-06-16, Alice, "Skin rash after swimming"
2023-06-17, Bob, "Gastrointestinal issues after swimming"
2023-06-18, Charlie, "Eye irritation after swimming"

# func generate_swimming_guidelines(weather_reports: list<WeatherReport>, sewer_overflow_events: list<SewerOverflowEvent>, symptom_reports: list<SymptomReport>): str
This function takes weather reports, sewer overflow events, and symptom reports from the community as input and generates guidelines for swimming in the river. It should:
1. Analyze weather_reports to determine if temperature, precipitation, and wind conditions are suitable for swimming.
2. Consider sewer_overflow_events to assess potential water quality impact and public health risks.
3. Evaluate symptom_reports to identify patterns or clusters of health issues related to swimming.
4. Generate clear and concise guidelines based on input data and defined principles, including any necessary warnings, restrictions, or recommendations.

Example outputs:

* generate_swimming_guidelines([WeatherReport("2023-06-15", 28, 0, 10), WeatherReport("2023-06-16", 30, 0, 5), WeatherReport("2023-06-17", 32, 0, 8)], [], []) = "The weather conditions are suitable for swimming in the river. Please exercise caution and follow general water safety guidelines. If you have any pre-existing health conditions or concerns, consult with your healthcare provider before swimming."

* generate_swimming_guidelines([WeatherReport("2023-06-15", 28, 0, 10), WeatherReport("2023-06-16", 30, 0, 5), WeatherReport("2023-06-17", 32, 0, 8)], [SewerOverflowEvent("2023-06-14", 2, 1000), SewerOverflowEvent("2023-06-16", 1, 500)], []) = "Due to recent sewer overflow events, swimming in the river is not recommended at this time. The water quality may be compromised, increasing the risk of illness or infection. Please wait until further notice when water quality tests confirm that it is safe to swim."

* generate_swimming_guidelines([WeatherReport("2023-06-15", 28, 0, 10), WeatherReport("2023-06-16", 30, 0, 5), WeatherReport("2023-06-17", 32, 0, 8)], [], [SymptomReport("2023-06-16", "Alice", "Skin rash after swimming"), SymptomReport("2023-06-17", "Bob", "Gastrointestinal issues after swimming"), SymptomReport("2023-06-18", "Charlie", "Eye irritation after swimming")]) = "Several community members have reported health issues after swimming in the river, including skin rashes, gastrointestinal problems, and eye irritation. As a precautionary measure, swimming in the river is not advised until the cause of these symptoms can be investigated and addressed. If you have experienced any similar symptoms after swimming, please report them to the designated community health representative."

This language model contract example defines four principles guiding guideline generation: public health and safety, environmental protection, transparency and accessibility, and adaptability.

The contract includes three data types: WeatherReport, SewerOverflowEvent, and SymptomReport. These represent community input regarding weather conditions, sewer overflow events, and health symptoms experienced after swimming.

The generate_swimming_guidelines function takes input data and generates guidelines based on defined principles. It analyzes weather reports to determine suitable conditions, considers sewer overflow impact on water quality and health, and evaluates reported symptoms to identify potential health risks.

The example outputs demonstrate how generated guidelines vary based on different input scenarios. In the first example, with suitable weather and no overflow events or symptoms, guidelines indicate swimming is allowed with general precautions. In the second, recent sewer overflow events lead to a recommendation against swimming. The third shows how reported symptoms prompt a precautionary advisory until the cause can be investigated.

By using a language model contract to generate community swimming guidelines, decision-making becomes more transparent, adaptable, and responsive to community input. This approach ensures guidelines prioritize public health and safety while considering environmental factors and specific community concerns.

Discussion

Using LLMs as world models for generative governance has several potential implications. It could enable more efficient and effective cooperation at scale, as contracts can be executed based on human understanding and intention rather than relying on a separate code representation. Furthermore, the epistemic anchoring mechanism may help mitigate disagreements and reduce attacker impact by favoring those with better world models.

However, there are potential risks and drawbacks. One concern is LLM reliability and robustness. If models are insufficiently accurate or subject to biases, this could lead to suboptimal or unfair contract execution. Additionally, there may be challenges ensuring LLM security and integrity and the data they use for prediction and evaluation.

To mitigate these risks, it may be necessary to develop rigorous testing and validation procedures for LLMs used in generative governance. This could involve adversarial testing, where models are deliberately subjected to manipulation attempts to identify and address vulnerabilities. It may also be important to establish guidelines and standards for LLM selection and use in contract execution to ensure consistency and fairness.

Another potential issue is dispute resolution. While epistemic anchoring can help align contract execution with the most accurate world model, there may still be cases where participants have differing interpretations or objections. In these situations, it may be necessary to have a fallback mechanism or appeals process, such as a human arbitrator or predetermined rules for handling such cases.

The legal and regulatory implications of using LLMs for contract execution also warrant careful consideration. Questions around enforceability, liability for errors or unintended consequences, and the need for legal frameworks to adapt will need to be addressed as the technology matures. Collaboration between researchers, developers, and legal experts will be crucial in navigating these challenges.

Despite these challenges, using LLMs for generative governance represents a promising direction for enabling more effective and efficient cooperation at scale. Further research and experimentation will be necessary to fully understand this approach’s potential benefits and limitations.

Extension to Foundation Models

While contracts have traditionally used written language, other forms of record could work even better. For example, video streams with location data that had signed data from the firmware could be part of the reports, and the foundation model could consider not only the contract language but also the evidence in the stream.

The generalization could be taken further, to a fully foundational model contract in which even the document establishing rules and procedures is not exclusively in language form but instead includes data in any input modality that the model and humans can observe.

Conclusion

Using LLMs for generative governance and language model contracts represents a promising and innovative approach to enabling more effective cooperation at scale. While there are challenges and risks to address, the potential benefits make it a worthwhile area for further research and exploration.

To develop and validate this approach, it will be important to conduct empirical studies and experiments testing the proposed mechanisms in practice. This could involve simulations or small-scale pilot projects to evaluate different LLMs and anchoring strategies and identify potential issues or areas for improvement. Engaging with the broader research community and stakeholders in related fields, such as contract law and game theory, may also be valuable.

If successful, generative governance could have significant implications for how we organize and coordinate human activity at various scales. From small community initiatives to global governance challenges, leveraging natural language and machine learning to create and enforce agreements could open new possibilities for collaboration and collective action. However, realizing this potential will require ongoing research, development, and collaboration across multiple disciplines.

We encourage researchers, developers, policymakers, and other stakeholders to engage with this emerging field and contribute to developing generative governance and language model contracts. By working together to address the challenges and opportunities, we can move closer to a future where more effective, efficient, and equitable forms of cooperation and coordination are possible.

GroupLang is hiring!

If you made it this far and want to work on this and related ideas, get in touch: nikete@grouplang.ai