How to Hold General-Purpose AI Producers Accountable

For the fourth year in a row, MIT Sloan Management Review and Boston Consulting Group (BCG) have assembled an international panel of AI experts that includes academics and practitioners to help us understand how responsible artificial intelligence (RAI) is being implemented across organizations worldwide. Last year, our experts weighed in on a wide range of topics, including organizational risk management capabilities to address RAI, readiness for the European Union’s AI Act, the importance of AI disclosures, and recommendations for overcoming the globally fragmented landscape for AI governance.

This year, we begin with a focus on accountability for general-purpose AI (GPAI) producers. Also called a foundation model, a GPAI refers to an AI model or system capable of performing a wide range of cognitive tasks that’s easily integrated into various downstream systems or applications.1 Generative AI models that can be adapted and used across a variety of applications are considered GPAI systems. AI systems may integrate GPAI models to execute a specific task, such as to provide customer service through an AI chatbot.

Given the power and growing influence of GPAI systems, accountability for their development and use is essential to achieving responsible AI (and business) outcomes. However, establishing accountability for the development of these systems is challenging because, unlike static product offerings, they evolve with use. GPAI deployments, inevitably, involve GPAI development. Is it even possible to hold GPAI producers accountable for their products?

In the context of this challenging accountability environment, we asked our panel to react to the following provocation: General-purpose AI producers (e.g., companies like DeepSeek, OpenAI, Anthropic) can be held accountable for how their products are developed. A clear majority (73%) agree or strongly agree with the statement, arguing that GPAI producers can and must be held responsible for their products, while emphasizing that doing so will require a multifaceted approach to account for complex AI value chains and multiple ecosystem actors.

Below, we share insights from our panelists and draw on our own RAI experience to offer recommendations on how to hold organizations developing general-purpose AI accountable for their products.

All companies can be held accountable, and GPAI producers are no exception. Our experts overwhelmingly believe that GPAI producers can and must be held accountable for their products. Jeff Easley, general manager of the RAI Institute, explains, “Just as pharmaceutical companies are accountable for drug safety or automotive manufacturers for vehicle standards, AI companies can reasonably be expected to implement rigorous testing, safety measures, and ethical guidelines during development.” Yan Chow, global health care lead at Automation Anywhere, agrees, stating, “General-purpose AI producers can and should be held accountable for how their products are developed, just as companies selling any other goods and services are held accountable for labor and manufacturing practices, product design, safety risks, marketing promises, and waste recycling.”

The scale of GPAI producers’ impact is an important factor for considering accountability, according to Richard Benjamins, founder and CEO of RAIght.ai. “Any company can and should be held accountable for how products are developed and even more so with general-purpose AI systems, which are being used by millions of people across the world with a huge impact on business, society, the planet, and work,” he says. Amnesty International’s Damina Satija agrees. “In other industries where technology can have serious impacts on individual and community welfare or rights (e.g., food, pharmaceuticals, aviation), regulation and governance are not left to private-sector actors,” she notes.

At minimum, holding GPAI producers accountable requires binding regulations. Despite this consensus on the need for accountability, some experts are skeptical that meaningful accountability for GPAI producers is achievable without regulation. As Satija argues, “The global discourse on AI governance is heavily weighted toward these producers self-governing through nonbinding principles on ethical or responsible use of AI that can be rolled back with zero accountability.” Armilla AI’s Philip Dawson, head of AI policy, agrees that “most obligations on foundation model developers exist within high-level, voluntary frameworks.” Tshilidzi Marwala of United Nations University echoes this, pointing out that “AI systems are often provided ‘as is’ with no guarantees of accuracy, safety, or reliability, effectively shifting responsibility to users.” This is despite users or downstream AI producers often lacking insight into how GPAI models are developed or the data used to train them.

Simon Chesterman, professor and vice provost at the National University of Singapore, highlights this disparity in oversight, noting, “If a system capable of acting as a therapist or friend were developed in a university, institutional review boards and ethics panels would oversee the process, but no such safeguards apply to private AI companies.” He warns, “Without stronger regulation, transparency mandates, or external oversight, accountability for AI developers will remain more theoretical than real.”

Fortunately, GPAI is increasingly subject to regulation. Franziska Weindauer, CEO of TÜV AI.Lab, cites measures like the European Commission’s General-Purpose AI Code of Practice, the new EU AI Act, and the EU’s General Data Protection Regulation as “crucial to ensure transparency, accountability, and trust.” Similarly, Rainer Hoffmann, chief data officer at EnBW, highlights the EU AI Act’s requirement that GPAI producers supply technical information about their models to the European AI Office and downstream providers, along with a summary of the training data used as ways to achieve a level of transparency that holds providers accountable.

While Shelley McKinley, chief legal officer of GitHub, agrees that the need for regulating general-purpose AI products is no different from the need for regulating other product-producing companies and industries, she believes regulation should “place responsibility with producers of commercial products and services, not the developers building open-source componentry across the tech stack that may be integrated into these systems.” She warns that failing to protect developers could harm innovation. Reflecting this concern, the EU AI Act includes a limited exception for AI systems released under free and open-source licenses and relaxes some requirements for open-source GPAI models.2

Beyond regulation, meaningful accountability requires a multifaceted approach to GPAI. Jai Ganesh, chief product officer at Harman Digital Transformation Solutions, proposes a combination of “industry-led ethical guidelines and standards, regulatory frameworks to establish clear guidelines and regulations for AI development, audits and assessments to help identify potential biases and errors in AI systems, and [frameworks for] transparency and explainability of AI systems.” Unico IDtech’s Yasodara Cordova believes that “independent audits and external oversight can strengthen accountability, while incentives like compliance certifications and funding for privacy-preserving AI can encourage ethical development.” Carnegie Endowment for International Peace’s Nanjira Sambuli also advocates for a combination of tools, including “audits, evaluations, and open sourcing.” Finally, Ryan Carrier, founder of ForHumanity, adds that the decisions that procurers and consumers make surrounding model safety can further promote accountability for GPAI producers.

But Amit Shah, founder and CEO of Instalily, believes that “blaming AI developers for every unintended application or malicious use of their creations ignores the agency of those who actually deploy these systems and the complexity of real-world scenarios.” He argues, “Much like a car manufacturer cannot be held accountable for every reckless driver, AI companies cannot feasibly control every wayward algorithmic outcome.” But Sameer Gupta, chief analytics officer at DBS Bank, sees the car analogy differently, contending that just as “manufacturers need to meet safety standards, distributors need to [ensure] cars are fit for sale, and drivers need training to operate the vehicle,” GPAI producers must proactively mitigate risks, companies that use AI products must establish their own governance frameworks, and users need education on AI risks and risk mitigation. As Rebecca Finlay, CEO of Partnership on AI, sums it up, “Safety is a team sport.”

GPAI producers can be held accountable for what they foreseeably control. Stefaan Verhulst, cofounder of GovLab, advocates for “distributing accountability across the AI ecosystem based on control and influence,” emphasizing that “GPAI developers should ensure transparency, technical safeguards, and model security.” Elizabeth Anne Watkins, research scientist at Intel Labs, would hold GPAI producers accountable for “gathering, labeling, sorting, and curating training data sets responsibly,” while Chow would focus on “training-content selection, bias mitigation, content filtering, and comprehensive documentation of known limitations.” Finlay says GPAI providers “can curate and filter their training data to remove potentially harmful content, conduct internal evaluations of their models through red teaming, and provide documentation, like model cards, to downstream developers.” But Marwala cites “a significant gap between ethical responsibility and legal accountability.”

As a result, several experts also emphasize the need to translate accountability into legal liability. Gupta explains, “While there are well-established guidelines for car accidents, the same can’t be said about AI-related incidents.” Dawson agrees, but cautions, “Given the complexity of these systems, tracing errors back to their source and proving causation can be exceedingly difficult.” Ben Dias, chief AI scientist at IAG, agrees: “It remains challenging to determine whether an issue that arises when using a general-purpose AI model is a result of how the producers developed the product or a result of how the deployer and/or user used the product in their specific use case.” Consequently, Dawson says, “foundation model developers remain largely insulated from liability in practice.”

Despite these challenges, Bruno Bioni, Data Privacy Brasil’s founding director, believes liability, like accountability, should be “analyzed in light of principles such as the duty of care, the foreseeability of risks, and the implementation of appropriate technical and legal safeguards — a precautionary framework proportional to the degree of control and influence each actor has over the system.” Finlay agrees that the responsibility of GPAI producers “should be understood within the broader AI value chain.” Ultimately, Carrier points to a need for “judicial decisions interpreting laws and liability” for GPAI producers.

Recommendations

In sum, we offer the following recommendations for how different ecosystem actors can help promote accountability for GPAI producers:

1. GPAI producers should hold themselves accountable. Accountability for GPAI systems begins with those who develop GPAI in the first place. Because GPAI producers cannot be held accountable “for every outcome of what they develop,” notes Katia Walsh, AI lead at Apollo Global Management, it is “even more critical to incorporate ethics and responsible principles from the very beginning and set the foundation for the technology to advance humanity.” In this regard, GPAI producers can adhere to established ethical AI guidelines and industry standards; conduct regular audits and impact assessments, as well as independent third-party assessments of model performance, fairness, and security; prioritize transparency, including by publishing transparency reports and model cards; ensure compliance with applicable laws and regulations, including privacy laws; and accept liability for incidents and harm caused by their models.

2. Businesses should boost their engagement with policy makers to shape regulations. It is crucial for businesses to actively take part in multistakeholder groups established by regulators and policy makers in order to help shape the implementation of existing regulations and guide the direction of future ones. This type of engagement can help hold all relevant actors in the AI ecosystem accountable for their respective roles in the AI value chain. For example, there has been a notable lack of industry involvement in key working groups in the EU that are influencing the development and interpretation of regulatory frameworks, such as the General-Purpose AI Code of Practice. Without greater business engagement, GPAI developers will remain overrepresented in these groups, potentially undermining efforts to hold them accountable.

3. Businesses should better collaborate with each other to define transparency requirements. To promote the accountability of GPAI producers, businesses should collectively define their transparency needs. Rather than focusing on strict standards, businesses can offer a view on what inputs are required for effective integration of GPAI models. For example, their recommended transparency requirements could address model interpretability, data usage, fairness, and the decision-making criteria that guide model development by GPAI developers.

4. Law and policy makers should ensure meaningful regulations are a baseline for GPAI accountability. Binding regulations are necessary but not sufficient for meaningful accountability for GPAI producers. Translating these regulations into practical accountability frameworks and legal liability may also require policy makers to establish independent, third-party watchdog organizations with specialized technical knowledge and adequate resources for oversight. At minimum, these watchdogs should mandate and oversee transparency reporting, enforce regular audits and impact assessments, and investigate complaints. They should also encourage cross-border collaboration on accountability frameworks, promote industry best practices and standards, and support public engagement and representative participation across a wide array of stakeholders.