AI Remixes: Who's Tweaking Your Favorite Model, and Should We Be Worried?

AI Remixes: Who's Tweaking Your Favorite Model, and Should We Be Worried?

We've all heard about powerful AI models like the ones that can write stories, create images, or answer complex questions. Companies that build these "foundation models" are starting to face rules and regulations to ensure they are safe. But what happens after these models are released? Often, other people and companies take these models and customize them – they "fine-tune" or "modify" them for specific tasks or uses. These are called downstream AI developers.

Think of it like this: an upstream developer builds a powerful engine (the foundation model). Downstream developers are the mechanics who take that engine and adapt it – maybe they tune it for speed, or efficiency, or put it into a specific kind of vehicle. They play a key role in making AI useful in many different areas like healthcare or finance, because the original developers don't have the time or specific knowledge to do it all.

There are a huge number of these downstream developers across the world, ranging from individuals to large companies, and their numbers are growing rapidly. This is partly because customizing a model requires much less money than building one from scratch.

How Can These Modifications Introduce Risks?

While many downstream modifications are beneficial, they can also increase risks associated with AI. This can happen in two main ways:

1. Improving Capabilities That Could Be Misused: Downstream developers can make models more capable in ways that could be harmful. For example, techniques like "tool use" or "scaffolding" can make a model better at interacting with other systems or acting more autonomously. While these techniques can be used for good, they could also enhance a model's ability to identify software vulnerabilities for cyberattacks or assist in acquiring dangerous biological knowledge. Importantly, these improvements can often be achieved relatively cheaply compared to the original training cost.
2. Compromising Safety Features: Downstream developers can also intentionally or unintentionally remove or bypass the safety measures put in place by the original developer. Research has shown that the safety training of a model can be undone at a low cost while keeping its other abilities. This can even happen unintentionally when fine-tuning a model with common datasets. Examples include using "jailbreaking" techniques to override safety controls in models from major AI labs.

The potential risks from modifications might be even greater if the original model was highly capable or if its inner workings (its "weights") are made openly available.

While it can be hard to definitively trace real-world harm back to a specific downstream modification, the potential is clear. Modifications to image models, for instance, have likely made it easier to create realistic deepfakes, which have been used to create non-consensual harmful content and spread misinformation. The fact that upstream developers include disclaimers about liability for downstream modifications also suggests concerns exist.

Why is Regulating This So Tricky?

Addressing these risks is a complex challenge for policymakers.

• Undermining Upstream Rules: Modifications by downstream developers can potentially sidestep the rules designed for the original model developers.
• Limited Visibility: Downstream developers might not have all the information they need about the original model to fully understand or fix the risks created by their modifications. On the other hand, upstream developers can't possibly predict or prevent every single modification risk.
• Sheer Number and Diversity: As mentioned, there are a vast and varied group of downstream developers. A single set of rules is unlikely to work for everyone.
• Risk to Innovation: Policymakers are also worried that strict rules could slow down innovation, especially for smaller companies and startups that are essential for bringing the benefits of AI to specific sectors.

What Can Policymakers Do?

The sources discuss several ways policymakers could try to address these risks:

1. Regulate Downstream Developers Directly: Put rules directly on the developers who modify models.
   • Pros: Allows regulators to step in directly against risky modifications. Could provide clarity on downstream developers' responsibilities. Could help regulators learn more about this ecosystem.
   • Cons: Significantly expands the number and diversity of entities being regulated, potentially stifling innovation, especially for smaller players. Downstream developers might lack the necessary information or access to comply effectively. Enforcement could be difficult.
   • Potential Approach: Regulations could be targeted, perhaps only applying if modifications significantly increase risk or involve altering safety features.
   <