Creative Assets

This week we talk about AI chatbots, virtual avatars, and romance novels.

We also discuss Inkitt, Galatea, and LLM grooming.

Recommended Book: New Cold Wars by David E. Sanger

Transcript

There’s evidence that the US Trump administration used AI tools, possibly ChatGPT, possibly another, similar model or models, to generate the numbers they used to justify a recent wave of new tariffs on the country’s allies and enemies.

It was also recently reported that Democratic mayoral candidate Andrew Cuomo used AI-generated text and citations in a plan he released called Addressing New York’s Housing Crisis. And this case is a bit more of a slam dunk, as whomever put the plan together for him seems to have just copy-pasted snippets from the ChatGPT interface without changing or checking them—which is increasingly common for all of us, as such interfaces are beginning to replace even search engine results, like those provided by Google.

But it’s also a practice that’s generally frowned upon, as—and this is noted even in the copy provided alongside many such tools and their results—these systems provide a whole lot of flawed, false, incomplete, or otherwise not-advisable-to-use data, in some cases flubbing numbers or introducing bizarre grammatical inaccuracies, but in other cases making up research or scientific papers that don’t exist, but presenting them the same as they would a real-deal paper or study. And there’s no way to know without actually going and checking what these things serve up, which can, for many people at least, take a long while; so a lot of people don’t do this, including many politicians and their administrations, and that results in publishing made-up, baseless, numbers, and in some cases wholesale fabricated claims.

This isn’t great for many reasons, including that it can reinforce our existing biases. If you want to slap a bunch of tariffs on a bunch of trading partners, you can ask an AI to generated some numbers that justify those high tariffs, and it will do what it can to help; it’s the ultimate yes-man, depending on how you word your queries. And it will do this even if your ask is not great or truthful or ideal.

These tools can also help users spiral down conspiracy rabbit holes, can cherry-pick real studies to make it seem as if something that isn’t true is true, and it can help folks who are writing books or producing podcasts come up with just-so stories that seem to support a particular, preferred narrative, but which actually don’t—and which maybe aren’t even real or accurate, as presented.

What’s more, there’s also evidence that some nation states, including Russia, are engaging in what’s called LLM grooming, which basically means seeding false information to sources they know these models are trained on so that said models will spit out inaccurate information that serves their intended ends.

This is similar to flooding social networks with misinformation and bots that seem to be people from the US, or from another country whose elections they hope to influence, that bot apparently a person who supports a particular cause, but in reality that bot is run by someone in Macedonia or within Russia’s own borders. Or maybe changing the Wikipedia entry and hoping no one changes it back.

Instead of polluting social networks or Wikis with such misinfo, though, LLM grooming might mean churning out websites with high SEO, search engine optimization rankings, which then pushes them to the top of search results, which in turn makes it more likely they’ll be scraped and rated highly by AI systems that gather some of their data and understanding of the world, if you want to call it that, from these sources.

Over time, this can lead to more AI bots parroting Russia’s preferred interpretation, their propaganda, about things like their invasion of Uk