Unity exec: AI tools are "disruption" for video game industry

The news of Unity's new AI-driven tools and marketplace is another mile marker on the controversial technology's advancement in the video game industry. Soon developers of all sizes using Unity will have the ability to create game content using text prompts and build games that can integrate AI models.

Such technology is apparently so powerful that it and tools from other providers will be "disruptive" for the video game industry, according to Marc Whitten, president of Unity's Create division. In public statements and an interview with Game Developer, Whitten painted a vision of Unity's AI tools as a natural evolution of the company's long-running mission to help "creators" of all stripes to create 3D content at a faster and higher-quality pace.

We wanted to know more about how Unity is approaching topics like data security, the "chum" of rapidly-produced, low-quality AI content, and the backlash over games like High on Life and Firmament using AI technology in development.

Whitten took our questions—and shared insights into Unity's journey into AI.

Some of Unity's AI technology has been years in development

A question many developers are asking of generative AI toolmakers is whether they are operating from a longstanding practice of making game development tools or are hopping on the recent investor fervor over tools like ChatGPT and Midjourney.

Whitten explained that Unity has been experimenting with machine learning and generative AI-like tools for years. The "animate" tool within Unity Muse which lets animators interpolate frames in Animation, has been one Unity has been working on for years. AI model inference tool Unity Sentis is built on Unity's Project Barracuda, which was first released to developers in 2021.

But it was advancements in natural language inputs, infill painting, and low-ranking adaptation processes that drove Unity to think of "a fundamental platform that connects" its varying machine learning tools.

Questions about the use of large data models quickly turn to the topic of sourcing the data these models draw from. Currently tools like ChatGPT and Midjourney face criticism for their reliance on datasets that scraped the internet for content.

Whitten declined to summarize the exact source of Unity's datasets, but offered some context for what developers can expect in terms of data ownership. He said that Unity Muse Chat is built on "other large language models" from outside of Unity that the company licensed, and which have been "heavily tuned" around Unity's unique information. That way developers trying to troubleshoot issues in Unity can interact with a chatbot interface whose responses are weighted by knowledge of Unity's documentation.

When it comes to worries over content scraping for visual assets, Whitten said that Unity is "very focused" on making sure it is building "ethical and transparent" models around these processes. Unity's objective is that if a developer create assets using Unity Muse, it can be included in a full game license that would let you or your studio have full ownership over the content.

"We are making sure that we build them on models where we have the rights and clear understanding of where the data came from...to make sure they work," he said.

AI models are able to produce more and more reliable output when trained on an ever-evolving set of data, which is what's led some toolmakers like Adobe to draw new image data from platforms like Adobe Stock. That practice has led some creators to worry about the use of their content for AI model training without their consent.

Whitten did say that Unity will be improving its models by adding more data to them, but that the company's focus is "definitely not taking someone else's art and training on top of it." "We're going to make sure that creators are in control, that they understand what's going on, that it's transparent, and that we have the right options for people." He wasn't able to confirm if the models powering Unity Muse would be automatically trained on content made by Muse, but reiterated that automatically designing the system that way would "break" the promise of data transparency.

What does Unity think of backlash to AI?

As mentioned up top the deployment of generative AI in video games has been met by no small amount of backlash from fellow developers and players alike. Whitten declined to comment on the examples of High on Life and Firmament, saying that most developers he's interacted with are excited about the tech's potential, and that Unity's offerings are at the core, about speeding up workflows and "democratizing" the game development process.

"I find the demarcation between an AI tool and a non-AI tool to be a very fuzzy thing," he said, using vegetation-modeling middleware SpeedTree as a comparable example. "No one using SpeedTree is out there designing trees that show up in the game leaf-by-leaf, the whole beauty of it is that you give it an intent, and it procedurally fills the world with realistic-looking vegetation."

"That's exactly what we're doing with our generative tools."

Whitten said that in the company's work with developers, they've mostly seen use cases for AI tools where developers rapidly "ping-pong" between generative content and hand-tuned tweaking.

What Whitten was less keen to comment on was the flood of low-quality AI-generated "chum" that has begun to fill up different marketplaces. Developers working with AI tools may have two different markets to contend with, as rapidly-created AI games on Steam could have a similar impact to their performance on mobile game stores, and creators building their livelihood by making middleware or assets for the Unity Asset store could be undercut by similar spam.

Whitten said there's a "deeper question" to the topic of AI spam, and that it's that game developers are having a harder and harder time finding players even if their game is exceptionally high-quality. He said it's a problem that Unity is trying to help developers with through Unity's Grow division, and that the company is always spending time "listening to people putting content into the Asset Store" along with developers to make sure it's improving that experience.

Unity's comments and boasting of the power of generative AI don't seem likely to assuage the concerns of developers put off by the technology. The company's language around Unity Muse is so similar to the boasts we've heard from other executives that it's easy to see backlash brewing.

But Unity Sentis may be the platform's ace in the hole when it comes to the platform's AI tech. Whitten said he's already talked to developers interested in using the access to neural networks (minus the higher cloud-computing costs) to do tasks like automating day-night cycles or create a broader range of responses for NPCs.

Developers do seem keen to make games using large AI models that can be new, unique experiences, and a reliable, cost-efficient way to integrate those models may lure some developers into Unity's ecosystem—which is sure to be a boon for its overall business model.