FYI - For Your Innovation: Unlocking the Power of AI in Gaming with Unity’s Marc Whitten

ARK Invest ARK Invest 8/10/23 - Episode Page - 44m - PDF Transcript

Welcome to FYI, the four-year innovation podcast.

This show offers an intellectual discussion on technologically-enabled disruption, because

investing in innovation starts with understanding it.

To learn more, visit arc-invest.com.

Arc Invest is a registered investment advisor focused on investing in disruptive innovation.

This podcast is for informational purposes only and should not be relied upon as a basis

for investment decisions.

It does not constitute either explicitly or implicitly any provision of services or products

by Arc.

All statements may regarding companies or securities are strictly beliefs and points

of view held by Arc or podcast guests, and are not endorsements or recommendations by

Arc to buy, sell, or hold any security.

Clients of Arc investment management may maintain positions in the securities discussed in this

podcast.

Hi everyone, welcome to another episode of For Your Innovation by Arc Invest.

A podcast on all things related to disruptive technologies.

I'm Andrew Kim, research associate covering consumer internet and fintech, and I'm joined

by Nick Groose, associate portfolio manager.

Today we have the great privilege of speaking with Mark Whitton, president of Create at

Unity.

Hi Mark, thanks so much for your time today.

Yeah, for sure, happy to be here.

And for our listeners, we think it'd be great if you can just start with an introduction

and how you ended up where you are today.

So I'm Mark Whitton, and as you said, I'm the president of Create at Unity.

What Create is, is all the tools and services that we provide for creators to build in real

time 3D, whether that's the Unity engine and editor, whether it's services like Unity

gaming services, whether it's tools like Weta Digital, so that people can create games and

other experiences in real time 3D.

I've been at the company about two and a half years now, and prior to that, I was most recently

at Amazon as a vice president of a bunch of the devices work, but I spent the majority

of my career at Microsoft, whereas one of the first employees on Xbox and spent 15 years

building the original Xbox, Xbox 360, Xbox Live, and Xbox One on the platform side, sort

of running a big chunk of those businesses.

So I spent a lot of time around games, I've spent a lot of time around 3D and developers

and creators, and as far as what brought me to the company, I knew our CEO quite well

for my time in Xbox, and we had about a 20 second conversation on the mission of Unity,

which is we believe the world is better with more creators in it, and this opportunity

that they saw, obvious to me, something that I care super, super deeply about.

But then as you got closer to the company, got to meet the extraordinary people inside

of there, is like one of those things where I went from like, yeah, yeah, I'm happy to

have a conversation, John, to I'm at the wrong place.

These are my people, and I need to get here as fast as possible, and I haven't looked

back.

That's awesome.

Thank you.

And obviously, Unity has been at the forefront of AI in the past couple of months.

That's obviously in the minds of many, many people, AI everywhere.

Are people talking about AI a lot?

They say, supposedly.

Yeah, it might be a big thing here.

And you guys talked about, I guess, Muse and Sentis as like the primary AI products, and

as for our listeners, I think it'd be great if you can just provide us with an introduction

to let's start with Muse, you know, what does it do today, and what use cases do you aim

to support in the future?

Yeah, well, let me give you like a top liner introduction to both because they're related,

and then we can talk about Muse in a little bit more detail.

Because frankly, it starts with the opportunity and the need.

It seems clear to us that over time, everything in every game, every quest, every boss, every

object, every pixel, every system is going to be touched by AI at create time, creators

using it deeply inside of their workflow, but also is going to be touched at runtime.

Actual inference happening on every pixel being displayed in every frame of a game or

other real-time 3D experience.

And so as we started thinking about that, it became clear to us that what we needed to

deliver was a platform approach for each half of that.

So how do we enable great AI workflows at create time to make it creators more productive,

to give them more capabilities, and to make it easier for more creators to create?

And then also, how do we unlock the real power of AI at runtime when the game's actually

being played?

So Unity Muse is our platform for creation side AI, and Unity Centus is our platform

for runtime side AI.

When you look at the two segments, where are the most challenges when you say, you know,

we're at X point today in this journey of AI being a part of every aspect of a game,

which of the two segments poses the most challenge?

And also, where are we in that journey?

Like AI has had this huge run up in the last six, eight months, it feels like it's starting

to seep into every sector, gaming being one of those, and there's so much innovation happening

so fast.

It almost feels like, hey, we've made so much striding, but I'm curious, where are we in

this journey, and then where are the challenges ahead?

Yeah, those are great questions.

And it's funny, I was sitting there saying, well, I don't know, which one is actually

the bigger challenge?

Because they're kind of an orthogonal set of challenges, and I'm not sure if I could

figure out which one's the right magnitude, because there's some really, really key problems

that you're trying to solve that are a little different on each side of that, and they're

both important.

I think there's a lot more intention being spent on the create side.

When people talk about generative AI today, they generally are talking about how to use

AI tools to create assets or help with coding or whatever those things are.

And I think, at least when we talk about this, people, when we talk about Unity Muse, people

are like, oh yeah, yeah, that's exactly what I expect that Unity would be doing.

When we talk about Unity Sentence, they're like, oh my god, I didn't even think about

this is what's going to be the implication of artificial intelligence as it applies to

gains and other real-time 3D experiences.

As for where we are, I think we're, first thing I'd say is AI has been used in a lot

of ways.

We've used AI inside of Unity for a lot of our tools for many, many years.

Obviously there's been this sea change because of the availability of extremely large language

models of techniques like diffusion and the availability of those in a lot of broad class

that really brought generative AI to the forefront.

But in another sense, especially on the create time, this is kind of like a continuum from

manual placing of pixels through procedural tools through full inference-based creative

tools.

It's a little more analog than digital in going between those states.

A good example of a non-what people think of as AI tool today at Unity that serves a

lot of the same purpose is something like SpeedTree.

I like to joke that actually most video games are just SpeedTree simulators because if you

see vegetation in a video game, there's a really good chance that it was done by using

SpeedTree as a tool, also in a bunch of film and other things as well.

Well, okay, no one's sitting around drawing individual trees and they're not even picking

from a class of trees or whatever.

Instead what they're doing is they're defining what the environment is that they're trying

to fill with vegetation and then allowing the smarts of SpeedTree to create variation,

so that you walk through a forest that's been created inside of it and it looks real and it

looks varied enough to not feel like you're in that old style of seeing sort of the same

object over and over again.

That's procedural though.

It's not an AI tool.

Now you're moving more into where you're using these generative and inference-based techniques,

but they've been heavily used for many years by artists, programmers, developers in real-time

3D.

Now, there's a bunch of challenges to now take it to the next level, which is what we're

spending time on.

So, I guess the generation of environments becomes a lot easier by combining aspects

of text to 3D models and also procedural generation as you've just mentioned, but I guess with

respect to more complex dynamic malleable assets, like a human for example, obviously

you can't just generate a holistic human.

You want it to move in the way a human would, and I'm just wondering, what are the puts

and takes with respect to training such a complex AI model that has to reflect how it

should behave in any given environment, if that makes sense?

It totally makes sense, and there's several challenges there, and there's not one system

of training that solves all problems.

In fact, part of the reason we think of Muse as a platform is in some level, what it is

now and what it will become even more so over the future is really orchestration of AI techniques.

What you want to do is unlock natural input, whether it's language or sketching or anything

like that, to give input into what you're trying to accomplish and then allow a variety

of AI, or in some cases procedural techniques, to then take over and help get you further

along where you're trying to get.

In large environments, in very complex environments, there's a couple of big problems that we're

spending time on.

The first is, there's actually a lot of research that's going on in, as you talked about, text

to 3D, but one problem that often comes from a model is models can start to become better

and better about generating a 3D mesh, but we call it this triangle suit, because what it is

is a sort of undifferentiated set of triangles.

Where does the ground end and the chest begin?

Can you open the chest?

Typically today, that's a much harder problem, because you're actually forward-generating

what the behavior of an actual object is in the environment, not just the generation of

the triangle.

We're spending a lot of time on how do you think about problems like that.

The other thing is, what you're going to see is a lot of expert-trained AI systems for

specific characteristics.

You talked about humans, which is a really, really great example.

We spent a lot of time, and in fact acquired a company called Zeba Dynamics that's really

focused on ML and AI-based training to solve this problem.

The old way of creating a high-quality character was to create it and then you hand-rig every

point that's potentially animatable and then hand-move them through every particular scene.

It's extremely time-consuming, which is one reason why the quality of characters that

you often see, especially in real time, has not increased at the same level that the performance

capabilities of GPUs or your hardware has.

We take a different approach with Zeba.

What we've done is we've trained an AI model that understands a huge number of data points

and tens of thousands of different emotional states around a character's facial animation

and deformation.

Then we run it through an advanced deformation solver so that you can move to instantaneous

inference of what the animation should be versus having to hand-rig it.

Something that might take a team of artists three months suddenly can take a server run

about 10 minutes to get you back something that you can use in real time and interactively.

It's not pre-canned animations.

It's not pre-canned rigging or anything about that.

The fundamental is still going to come back to the same thing, which is there's not one

AI technique or AI system that's going to apply to all of these because they're all

really, really complex behaviors.

Our focus is instead, how do you create orchestration in a platform layer so that we can plug in

all of the expert AI systems in for a particular area?

You mentioned a really interesting stat there that three months to 10 minutes.

I want to zoom out a bit here and talk just about productivity gains at large.

When you look at the video gaming space, obviously, AAA games take years and years to develop.

I'm curious, what do you think are the areas to focus on when you talk about productivity?

Where are you hearing stories from customers already saying, we are just rocking and rolling

now with all of these services you're putting out there, this is taking us half the time.

You mentioned three months to 10 minutes on this animation styling.

I'm curious, where are the other areas where you're seeing these productivity gains?

First off, we're still early in some of these tools rolling out.

We're learning with our customers and we'll continue to move them forward, but I'll step

back a little bit of the curvature of the earth and then talk about that.

The truth is that we have a crisis of content creation.

It's not possible to make the amount of 3D content and assets that video games and other

real-time 3D experiences demand.

When everybody was so hyped about things like the metaverse, one thing that many of us were

talking about is like, this is going to be a bunch of really empty worlds because there's

not enough content in content time, a thing that you often see in video games in particular,

especially from larger games, is if you look at the announced video of a game and then

the final game, they often ended up not necessarily achieving their vision.

Well, that's not about the tech.

It's actually about the content creation and the assets.

It's one of the things that takes the most time.

One of the reasons the games are delayed or become less ambitious or fail to reach their

goals.

There's a desire for much more of that content.

We have to solve the productivity.

The interesting point about the three months to ten minutes is it's actually zero to ten

minutes because everybody looks at the three months and say, we can't afford it, so we

don't do it, or we're going to use a much worse model and it looks worse or whatever.

We're able to change the quality bar and also give them productivity gains.

On the asset side, we see this as two ways.

Number one, what we want to do is make sure that these tools enable a broader class of

creators to have access to technology and become creators.

We believe the world is better with more creators than it is our core mission.

Number two, to create productivity gains for professionals so that they get their job done

faster and they feel more in control of the end outcome.

This isn't about generate some art for me.

It's about give me a generation and then all of the knobs and the nodes that I need in

order to turn that into something truly special for the game that I'm trying to build.

In the end, I think it will mean a significant more high quality content in significant more

games.

I actually think it's going to mean more net creators and sort of be expansive for the

industry, but also allow people to make bigger bets on the types of games that they want

to be willing to try because they can get there a lot faster in prototyping to understand

what the challenges are and these other areas because they have these techniques.

Wow.

Yeah, that's really fascinating to hear it.

It's an expanding market.

Embedding AI can actually expand the market.

You're talking about potentially even new content types as well because-

I strongly believe that's what's actually going to happen.

It doesn't mean there won't be changes and that people will have to learn new tools.

I actually think it will be expansive of the types of things that people are trying to

accomplish as they go create and especially in real time in 3D.

So I guess you've iterated more content and more high quality content, but I guess looking

at the other side of the equation just on the consumer front, given that people have

limited time or limited resources, how do you think the market will evolve as it may

look more competitive going forward for developers in general to get the game in front of consumers?

Yeah.

Gaming is already extraordinarily competitive to find and keep your audience.

It's also extraordinarily lucrative and so with or without AI, it is only going to continue

to increase in the level of competition as people look at it and go try to do great things.

Now, to be clear, that's one of the longest uses of AI that we've had.

If you look at the ad monetization in Unity, the ad business is really about how do you

help people find their customers to keep those customers engaged and then obviously be able

to monetize their game so that they can be successful.

That's all based on using AI models and neural networks in order to better target so you

find the specific player that would most like their game is most likely to stick around

for day 7 or day 30 or whatever your retention metrics that you're looking at.

So AI will play a continuous role in doing that.

However, there's another trend.

It actually sort of leads into the other half of the AI stuff that Unity sent us that we've

been mentioning, but I'll start it by saying the other fundamental part of that is all

of these games are becoming what we call live games or live ops games.

It's not about the initial bits that you shipped, but it's about did I build this community

and am I maintaining the game that I've built for them to sort of continue to be engaging

and robust so that I hold on to them for a long period of time.

And I'm a big Marvel snap player as an example and every time a patch comes out, I'm sitting

there saying, oh, I wonder how I'm going to change my deck and this is going to shake

up the meta and let me keep playing it.

Well, the core reason they're doing that is they want to keep the game engaging for their

existing players as well as try to drive new players in.

One of the big opportunities in AI at runtime is to create deeper, richer living worlds

that actually feel much more alive to the players inside of them.

And there's some key examples that we've been talking about.

If you saw a video of our alien that we did with the Sentis one, you can see a little

bit of this of like an NPC that you can truly have a conversation with and sort of use that

to sort of shape the game experience in a way that's very different than everybody else.

If you think of Skyrim, one of the best games ever, in my opinion, there's things like you'll

go talk to someone and they'll say, I used to be an adventurer like you, but then I

took an arrow to the knee.

And it's a total meme.

And the reason it's a total meme is because millions of gamers have heard it dozens of

times because they don't, you know, going back to the content creation thing, you can't

have every, you can't go create content for every guard to say something different to

everybody every time.

Well, when you switch that out where you're still in control of the game world and its

lore, but I might have an extremely different experience than you had.

In fact, maybe I talked this guard into like coming along with me or giving me some different

information, then you can suddenly do like it's going to feel a lot more alive than

anything that we have today.

And that's an example with NPCs, but that plays out actually in a lot of the systems

that you'll think about inside of your games.

I guess just following up on intelligent NPCs and like how far are we from a fully interactive

Skyrim and like what, like how big do the models have to be to be able to run that locally?

Yeah, it's a really good question.

And I think one of the things there's there's kind of an economic question and then there's

a capability question.

I don't think we're very far away.

In fact, you can see lots of examples, including people hacking Skyrim to plug in LLMs and

try them out and getting cool results.

Hacking, it's not not sort of the way that you think about a finished game yet.

But the first problem you have to solve is running a bunch of cloud inference can be very

expensive.

It's one thing if it's at create time, because it's a relatively small number of people for

for a defined period of time doing the creation.

If instead you said I have to pay every time you talk to an NPC as a cloud inference times

number of concurrent users that I have and you have a successful game, well, that could

actually the cost of that could actually scale in a way that's not just not manageable.

And so part of what we're trying to do a sentence is to make it easier for you to imagine

using neural network based features at runtime with deterministic cost.

The other problem that so then you're like, oh, I'm just going to run it locally.

Well, the problem with running it locally is number one, the tech stack for most neural

networks and the languages behind them, whether it's, you know, TensorFlow or PyTorch and

things like Onyx and things like that is not is designed for a very specific architecture.

Typically, it's designed for cloud based PCs.

And so the capabilities and both the hardware language support and all of those sorts of

things don't exist broadly.

And so now you have to say only people on this particular device can play the game.

Well, game creators say that's too small of a market.

If I start, you know, sort of willing it down.

So what we're trying to do a sentence is to make it so that you can take any standard model,

assuming it fits in, you know, has to fit in local memory, you can make one that's too big.

And we will make sure that works anywhere the unity runtime runs.

So we will take take care to sort of transpile the PyTorch into whatever we need to

locally to run on the GPU or CPU or the best possible performance that you can get.

So that suddenly says, as my design palette, I can imagine the model running locally.

So now I can start to design for it.

Then you get into the size of models.

There's a lot of research going on that says that you started specifically about intelligent

NPCs. It's unclear that you need massively large models to make that happen.

You might have small models that work for most people.

And then maybe you go to the cloud for like some, you know, boss or the head wizard or

whatever that you need more, you know, deeper inference from.

But the other thing I would say is it's not just about those types of features.

There's I believe that we're going to enter this golden age of inference based

techniques used at runtime.

And many of those are going to be very small models because the other fundamental thing

it's actually been interesting talking about AI runtime with game creators is there's one

hard truth in game development.

You have 16 milliseconds to get your frame done and then it's time for your next frame.

You know, that's not much time.

So that often means what am I able to do?

Oh, I want to do this advanced lighting thing that I want to do.

Oh, I can't because I'm on a piece of hardware where I don't have enough time to run the

math to render that lighting technique in my 16 mil, however many of the 16 milliseconds

that I can spare for that particular lighting technique, because I have to do everything

for the game, the whole simulation input, the entire thing and every 16 millisecond turns.

Well, inference means that you can take some of those problems and turn them into inference

instead of instead of mathematical generation or it's different, it's different math, I guess,

but use a use a model instead of doing direct calculation.

And what we're hearing is that people are super interested in having all sorts of models,

models that do lighting, you know, to use that specific example.

Instead of calculating the physics, think about ray tracing.

That's a very high performance physics calculation to generate how does a photon get to my eyeball

to turn many types of techniques into I'm just going to do a bunch of training.

And now with a very small model, because it's very purpose based, I can do extremely fast

inferences and actually create it as a feature that I can run across every platform that I'm trying to deliver.

Gotcha.

And do you think as we enter this like golden age of AI, as you've said, like, does this accelerate

like growth in memory, do you think?

Like, because like if you think about the like iPhone, you know, unified memory, it's been growing

for like the past decade, but then starting in the, I would say the mid 2010s, it kind of slowed down,

right? And that, I assume, presents a bottleneck for more performant models running locally.

And I'm just wondering, like, is that like a meaningful constraint in your opinion in the longer term?

I think on a core RAM, probably, it probably will end up sort of helping drive additional

increases there. The thing that is super clear is that there's going to be a lot of energy put

into continued evolution of AI specific silicon, that is, whether it's a module, I mean, you know,

use a GPU for a lot of the silicon, they're becoming more and more tailor made to run an

inference versus run a render pass. And I think that will continue. And that will be on every SOC,

you know, everybody's going to be thinking about like, what's their kind of inference engine that

they're embedding in a piece of silicon. And that will then probably drive this cycle about like,

okay, well, what does that change in terms of how we think about hardware cache or RAM size or

other things to get the most out of that, just like GPU evolution has core GPU evolution for 3D

graphics has for the last, you know, 20 years.

Mark, I want to change gears here a bit. And first, just congratulate Unity on the partnership with

Apple. But my congratulations is also the question. And, you know, could you give us a bit of background

on the on this partnership with Apple? And, you know, what's happening now that they have the

Apple Vision Pro out in the market, or, you know, it'll be coming later this year, early next year?

Yeah, you know, we've collaborated with Apple for a very long period of time across many things.

You might remember that when they originally launched their M1 silicon, Unity was there, not just

the runtime capable of running on it, but the full editor already ready to go on M1 silicon. And

obviously, we've worked very, very closely on iOS and iPhone for many years, huge, huge part of the

gaming market. And so, you know, when we first heard, you know, what they were thinking about

doing, we were really excited to see where they were going to go. They've really, I think, gone super

deep into how do you think about spatial computing and what they've built inside of Vision OS. And

it's been really exciting working with them on it. There were two things that we were really excited

to do. The first one is, I'll call it Unity standard platform approach. Our goal is to make sure as

much of Unity just works for you as you target whatever platform matters, so that we want to make

it as easy as possible for the way that you think about building your game, your workflows and all

that sort of stuff for you, then say, I want to now target platform X, Y or Z. So that's standard,

make sure it's just easy to use Unity on day one of the launch. But the other thing that I think is

very unique about Apple's approach to spatial computing and the Vision OS is shared mode in the

experiences. And that meant that we needed to work quite closely with Apple in order to think about

how to render, how to build and deliver multiple Unity experiences that can be resident and running at

the same time, whether they ever knew about each other or not, and that those experiences could be

running next to an Apple experience or a Reality Kit experience. And that's different than what we've

done in any other platform. And I think it really, we're excited to see what creators do with that,

because I think it creates this opportunity. You can imagine there's this crazy game or experience

that I've built. And next to it is another experience that a video screen where you and I are

chatting or sort of can see each other as we're doing it. And then maybe I have something off to the

side that I'm just kind of slightly paying attention to. That's cool. And then what's also cool is you

can start to imagine those experiences being able to interact with each other over time. And what we

tend to find is whenever there's this type of paradigm shift that game creators and real time 3D

creators were like this tip of the spear of like figuring out kind of like what new sort of modalities

or interesting ways that you could use that technology to create something fun or novel or

different than what exists elsewhere. So yeah, we're super, super excited about it.

Yeah, it sounds amazing. I'm so excited for this collaboration. I have a follow up question and this

ties it back to what we were originally discussing with Muse. Do you think as Muse scales, it will be

possible for developers to deploy on a device like Vision Pro, even if they didn't design for that

platform originally?

Well, I mean, that's always our goal to make that as easy, regardless of whether you're using AI or

not, we're going to do everything we can to make it easy. I think there's two parts about that. And I

think AI likely makes part of it easier. There's the technical part is my game capable of running on a

particular platform. But then there's a second part, which is game design, which encompasses a lot,

because it just turns out that different types of platforms lend themselves to different ways that you

think about presenting information and taking input from the user. Think about a PC game or PC

application, it's extremely information dense, because you have a large screen, you know, in one of

things people learned when they first started doing things like game streaming of, let's say console

games on a phone is, you know, it's really cool to play Assassin's Creed on a phone. There's a lot of

text, like your screen is like covered with text, because it was designed to be on a 50 inch display,

suddenly it's on a five inch screen you hold in your hand. Twin stick is different than mouse and

keyboard is different than touch. And those aren't just how do you translate between one versus the

other, they often can go really deeply into how you design the gameplay loop to be fun, because you

could translate between mouse and keyboard to twin stick, but that doesn't mean that it feels great on

twin stick or that it's the right mechanic for you. And so that part AI will help because it will help

you explore the space and it will help you get farther. But you will still have to fundamentally

understand the type of game or experience that you're trying to build and how a user is going to

interact with it.

That makes sense. Very excited to see what gets built for the Apple Vision Pro.

Yeah, me too. It's going to be really fun to watch.

I think your point on like shared experiences for the Vision Pro is really interesting, right, because at

least from what I gathered from the Vision Pro demo, right, and it seems like the Quest Pro, right,

sorry, the Quest is taking a very different approach to gaming compared to the Vision Pro, right, because

you have 2D in a spatial environment versus like full immersive 3D for the Quest. And I'm just wondering,

do you think those two experiences will like service different markets? Do you think like Vision Pro

eventually will kind of also collide with like that kind of full 3D use case? Like how do you think

about like developers developing for either device?

What's exciting about Unity is that we end up, you know, working with a lot of different platforms. And

for us, we're always looking at how does that create more opportunity for creators? Because we think all

those spaces will get filled. If there's a high quality device and a company that's really out there sort of

building really cool stuff for that, then game creators will come in and fill out interesting use cases

inside of it. And so I think that, you know, whether you're talking about a device like the Quest, or

you're talking about Vision Pro, or you're talking about Xbox or a PC, all of them are going to have

really, really robust ecosystems, and people are going to explore them. Now, if we talk about Vision Pro,

particularly, it's extraordinarily powerful device. And it will definitely have really rich 3D, full

3D experiences. It's more than capable of doing it. One of the things that was exciting in working with

Apple is just the amount of time that they spent on thinking about whether it's the sensors, the display

quality, or, you know, the power of their in silicon to drive, you know, really amazing results. So I

think you're going to see some some very interesting stuff. And again, what will be exciting is when

people start to use that kind of palette of toys that they have, I always think of it as how many

crayons do they have in their crayon box to start, you know, building something, they're going to go do

something that none of us think that the device is actually built for. One of my favorite first

examples on the iPhone when they launched the App Store was from a company called Smule, who built

this app called Ocarina, which was essentially, your iPhone's a flute, and you blow into the

microphone, and then you use your fingers to fret. And the thing that's interesting about that is you

know, you know, no one at Apple, no one at Apple thought that they had developed the Woodwind API.

And developers didn't eat weren't thinking about that. But suddenly they came out and did it. And you

could see everybody at Apple go, Oh, and you can see every other game creator go, Oh, and it's that way

that the space gets broader and broader, because people, it's actually, you know, probably telling a

little excited about this kind of stuff. Like, I've been doing platforms for a long time. And I

always like the questions when people ask me, what do you think people are going to really do with

this? Like, I can tell you that whatever idea I have is so low on the creativity scale. Because the

beauty of being a platform creator is you unleash the creativity of so many brilliant people to sort

of really, you know, tease away and people do amazing things. There were people on the original

PlayStation one, PlayStation one was out forever, it lasted a long time. They were people using the

audio chip to run extra graphics, because they just wanted to do these crazy things. I mean, creators

are inventive, they're brilliant, and they're going to do amazing stuff. If you give them new

techniques and new tools to play with.

And that's certainly possible now with the Apple Vision Pro and the Unity collaboration. And I think

we'd be remiss to say, but the beta just opened, right?

That's right. We just started our first closed beta. We've had extraordinary interest since the WWDC

announcement and continuing with people signing up, both in games and non games. And on Wednesday of

this week, we started putting our first closed beta. And now, obviously, our goal is to expand as soon as

possible to make it easy for everybody that's interested to target it. But it's really exciting to

get started with our first creators on it.

Very exciting. Everything that Unity and Apple are doing in this collaboration. We want to end with just

one last question. We've talked a lot about gaming. Unity has exposure to non gaming segments. It's

being used in many different industries. How do you think that this is going to continue to evolve? How

do you think Unity, especially with AI now, is going to be used in some of these different industries

that you're currently being used in?

Frankly, one of the main reasons that I joined was because of the massive opportunity and the

continued growth of real time and 3D in non gaming use cases and industrial use cases and digital

twins. It's a, I believe deeply that essentially every industry is going to be rebuilt around real

time and 3D. And for a variety of really important reasons, we are 3D and real time people. And so

whether it's medical or manufacturing or the way that I interact with my house or devices or

anything else like that, 3D is a richer, deeper, more information full way for me to do some

interaction that I'm trying to interact. And we're very, very focused on how do you make sure these

tools that were pioneered in gaming and continue to be extremely robust in gaming are as useful and

that we expand through all the different types of use cases. AI, really, really critical. Obviously,

inside of it, you go through that content creation problem we talked about. It's even more so outside

of gaming where there's not maybe the same concentration of artists and 3D technical creators in a

particular industry or a particular enterprise that they have to go through. So we're working very,

very hard to make it so that it's easy for everyone to pick up and use. And frankly, it's the fastest

growing part. You know, it continues to see extremely robust growth, whether it's in manufacturing,

commerce, medical, construction, you know, sort of end to end. The last thing I'll just leave you

with on that one, since you asked the question. One of the things I think is the most exciting

opportunity in a place that Unity plays specifically well, we're not trying to go... Well, actually,

let me step back. Every industry has a whole bunch of 3D data. Like a manufacturing company builds

their stuff in 3D CAD tools. They have tons and tons of 3D data. We're not trying to change that. We're

not trying to say, oh, you shouldn't use this super high-end CAD tool that you've always used to create

the very specific stuff that you need to do before you go to tooling and manufacturing. What we're

trying to do is free that 3D data, because today that 3D data is captive on high-end desktops in the R&D

department of one part of the company. And yet the use cases, if you take that 3D data and you say,

okay, well, actually, what I really want to do is I want to be able to do an unlimited amount of

photos so that I can do for commerce or for any other thing that I need to do it. Well, the way that

works today is you wait to get a real object and you actually have to go create photos of it and all

that sort of stuff. There's no reason for that. You should be able to literally have not the R&D

department, but the marketing department have access to the same 3D data, transformed to be real

time and interactive, and transformed so that it can run on every device so that it can get both

throughout the enterprise, but also to all of the customers and partners of a particular company.

And so that's our goal. Our goal is to actually take the 3D that they already have, but truly free it

so that it adds value across everything that they're trying to do in their enterprise from, you

know, core sales to support to, frankly, even internal logistics and operations. And that's why we

see such an extraordinary pull from all of these industries into Unity.

Amazing. Really exciting times ahead.

Awesome. Yeah, this is great. It's really, really great speaking with you guys. So yeah, I've enjoyed

it. Well, thank you so much for your time, Mark. And for all our developer listeners out there, make

sure to check out Muse, Sentis, and the Unity VisionOS Open Beta. And also make sure to check out

Arc's blog on generative AI's impact on gaming. Thank you.

Unity gets a nice shout out in that.

Thank you. All right. See you guys.

Arc believes that the information presented is accurate and was obtained from sources that Arc

believes to be reliable. However, Arc does not guarantee the accuracy or completeness of any

information. And such information may be subject to change without notice from Arc. Historical

results are not indications of future results. Certain of the statements contained in this

podcast may be statements of future expectations and other forward looking statements that are

based on Arc's current views and assumptions and involve known and unknown risks and

uncertainties that could cause actual results, performance or events to differ materially from

those expressed or implied in such statements.

Machine-generated transcript that may contain inaccuracies.

Hosts Andrew Kim and Nicholas Grous venture into the world of artificial intelligence (AI) in the gaming sector with Marc Whitten, the President of Create at Unity. They discuss AI’s role in creating vivid, immersive 3D experiences. Marc gives them a behind-the-scenes look at the complexities and opportunities involved in deploying AI for content creation in games. They also explore Unity’s mission to make game design accessible to all. The conversation doesn’t stop at gaming; they also cast a wider look at how real-time 3D technology is making waves across various industries. So join Andrew, Nicholas, and Marc on this insightful journey where they blend technology, creativity, and gaming into a captivating exploration.https://ark-invest.com/podcast/unlocking-the-power-of-ai-in-gaming/

" target="blank">
“I believe that we’re going to enter this golden age of inference based techniques used at Runtime and many of those are going to be very small models.” @MarcWhitten





Key Points From This Episode:

Mark Whitten’s background and role as President of Create at Unity
Gaming industry competitiveness and use of AI
AI targeting specific players and improving ad monetization
Focus on building and maintaining gaming communities
AI at Runtime for immersive and dynamic game worlds
Non-player character (NPCs) having meaningful conversations and shaping game experiences
Implications and challenges of generative AI in game development
Unity’s use of AI tools and collaboration with Apple
Apple’s approach to spatial computing and Vision OS
Orchestration of AI techniques through Muse platform
Solving content creation crisis in the industry
Productivity gains and increased number of creators with AI tools
Challenges and benefits of using neural network-based features in games
Cintus platform for local model execution and improved performance
Designing games for different platforms and input methods
Real-time and 3D technology in non-gaming industries
Unity’s goal of expanding applications and accessibility of tools
AI’s role in content creation across various industries
Unity’s focus on real-time and 3D technology in different sectors
Partnerships with various platforms and potential for creativity