Lex Fridman Podcast: #398 – Mark Zuckerberg: First Interview in the Metaverse

00:00:00 - 00:30:00

In this episode of the Lex Fridman Podcast, Mark Zuckerberg discusses the potential of the Metaverse and its impact on human connection. They explore the use of photorealistic avatars and spatial audio to enhance communication. The conversation also touches on the technical complexities of creating immersive experiences and the future vision of making the scanning process more accessible. The podcast hosts and guest express excitement about the future of virtual reality and mixed reality technology in remote meetings and social interactions.

Show more

00:30:00 - 01:00:00

The guest envisions a future where holograms and realistic avatars are common, allowing people to interact in virtual meetings and social hangouts. They discuss the potential benefits and challenges of photo-realistic avatars compared to expressive ones. The podcast also explores the ethical considerations of interacting with deceased loved ones in the metaverse and creating AI versions of oneself. The hosts express excitement about the possibilities of the metaverse and the blending of physical and digital realities.

Show more

01:00:00 - 01:12:05

The speaker discusses the release and reception of Llama 2, an open-source model for AI development. They mention the priority of integrating Llama 2 into consumer products and the ongoing work on future foundation models. Mark Zuckerberg discusses the concept of the metaverse and the potential for profound changes in communication and human experience. The development of the metaverse is seen as the next frontier for social networks and requires new computing platforms and software stacks.

Show more

The following is a conversation with Mark Zuckerberg inside the Metaverse.

Mark and I are hundreds of miles apart from each other in physical space,

but it feels like we're in the same room because we appear to each other as

photorealistic Kodak avatars in 3D with spatial audio.

This technology is incredible and I think it's the future of how

human beings connect to each other in a deeply meaningful way on the internet.

These avatars can capture many of the nuances

of facial expressions that we humans use to communicate emotion to

each other. Now I just need to work on upgrading

my emotion expressing capabilities of the underlying human.

And now a quick few second mention of a sponsor. Check them out in the description.

It's the best way to support this podcast. We've got element for

delicious electrolytes, inside tracker for biological data,

8-sleep for delicious naps, AG1 for delicious health, and Netsuite

for probably delicious but I haven't uh tasted it

uh business management software. Choosewise and my friends. Also if you want to

work with our team where I was hiring go to lexfreeman.com slash hiring

and now on to the full ad reads. As always no ads in the middle.

I try to make this interesting but if you must skip them

friends please do check out the sponsors. I enjoy their stuff.

Maybe you will too. You never know until you try.

This episode is brought to you by the thing I consume

many times a day. The thing I'm consuming currently as I am speaking the words

I'm speaking. Element electrolyte drink spelled L M N T.

I do a low carb diet sometimes a very low carb diet

and I usually eat only once a day so that's fasting and low carbs and for

that you have to get your electrolytes right.

Electrolytes include sodium, potassium, magnesium, the proportions are really

important. The deliciousness of the proportions won't

combine with water. It's also important it turns out.

It's also just important to consume water and making water a little bit more

delicious which is what element does

is a win-win. That's what we call in the business a win-win friends.

Get a simple pack for free with any purchase.

Try it. Join me and try it at drinkelement.com slash flex. Get a simple pack

for free with any purchase at drinkelement.com slash flex.

This show is also brought to you by Inside Tracker, a service I use

to listen to my body through the power of data science.

So for my body comes signals through blood tests that give information about

whatever's going on in my body and that data is used with machine learning

algorithms to make suggestions about what you should do in your life.

Now in this conversation about the metaverse where Mark and I are incredibly

in a whole other plane of existence we have teleported into this place

while we're miles apart from each other in physical space

in the virtual space we're right there together shrouded in darkness.

I wonder if those entities that are driven by the content of our minds

if they have biological like signals if they need to make lifestyle and diet

decisions. I wonder one day we should be able to

ask them. Actually that's one of the things that

I'm about to announce which is really cool which are the AI

personalities and I do think from just from a technical

perspective is super exciting and difficult to encapsulate a particular

personality like Snoop Dogg in an AI agent

where it reflects all the quirks the weirdnesses but also the beauty of the

character of that particular human being. I love

conversation that task doing that thing technically

super exciting but I think super difficult so I'm glad they're taking it

on. Anyway get special savings for a limited

time when you go to insettracker.com slash lex.

This episode is also brought to you by 8Sleep and it's pod 3 mattress. It can

cool down or heat up the two sides of the bed separately.

This actually makes me think of that meme where the two kids are scared in the

corner and they're looking at a bunny and

I am those two scared kids in the corner thinking about the bunny

where the bunny represents the people that want their bed heated up.

I don't know who does this but I want to meet you. Maybe you have an 8Sleep like

in the Antarctic or somewhere in northern Canada

but you also don't have heat and maybe you're out

camping. I can't possibly construct a situation which you would use heat.

I'm only half kidding of course. Anyway I

prefer to cool down the bed and it's an incredible experience. A cold bed with a

warm blanket. It's another place I teleport to in my

mind when I go to the nap. It's not a metaverse. It's a

nappaverse and there I find a deep piece from the chaos of life

and I return refreshed ready to take the chaos on

once again. Anyway check it out and get special savings when you go to

asleep.com. This show is also brought to you by AG1.

Another delicious thing I consume every day. I just drink it like an hour ago.

It's delicious. It's green and makes me feel like Popeye.

I grew up I don't know how many years ago watching Popeye.

There was a stretch of time where I wanted to be

Popeye. I think I admired Popeye for being strong.

Maybe because I wanted to be strong. I always thought

a man is supposed to be strong and so if I just eat my spinach like Popeye did

I'll be strong so hence the color green is forever associated

with strength and I'm also playing Diablo now and that's one of the things

you can increase is strength, dexterity, willpower,

intelligence, all that kind of stuff but strength still has that

sexy sound to it and so every time I play Diablo actually

and I drink AG1 and it just makes me feel stronger and better and like I

got my nutritional bases covered. Anyway they'll give you one month supply

of fish oil when you sign up at drinkag1.com slash Lex.

This show is also brought to you by Netsuite an all-in-one cloud business

management system. I still hold in me a little bit of a

flame that turns hopefully into a fire which is a desire to

contribute by building stuff whether it's

working at a company or starting a company

and I'm very cognizant of how difficult it is to run a company

successfully whether you're looking at a huge company like Metta

or a tiny startup all of it and actually

yes all of it is a source of challenge and complexities and

and fear and anxiety and uncertainty all of that

but ultimately the camaraderie of the people working together

that's a deeply fulfilling deeply meaningful pursuit

especially when there's a big vision that you're reaching for so

I love people at companies working at a thing

chasing that big impossible vision actually the metaverse is one such thing

just the technical complexity of everything involved it's just

really incredible from the headset software and the hardware

the cameras the scanning for the average all of it is just incredible

anyway if you want to figure out how to run a business

small or large successfully you should be using the best tools like Netsuite

you can start now with no payment or interest for six months

go to netsuite.com slash lex to access their one-of-a-kind financing program

that's netsuite.com slash lex

this is the Lex Friedman podcast and now dear friends

here's Mark Zuckerberg

this is so great

lighting change wow yeah we can put the light anywhere

and it doesn't feel awkward to be really close to you no it does I actually

moved you back a few feet before you got into the headset you were like right here

I don't know if people can see this but this is

incredible the realism here is just incredible

where am I where are you Mark where where are we

you're in Austin right no I mean this place

we're shrouded by darkness with ultra realistic

face and just feels like we're in the same room

this is really the most incredible thing I've ever seen

and sorry to be in your personal space I mean we have done jujitsu before

yeah no I was commenting to the team before that

even that I feel like we've choked each other from further distances than it

feels like we are right now I mean this is just really incredible I don't know

how to describe it with words it really feels like

it feels like we're in the same room yeah feels like the future

this is truly truly incredible I just wanted to take it in

I'm still getting used to it's like it's you it's really you

but you're not here with me right you're there wearing a headset not wearing a

headset it's it's really really incredible so

what can you describe what it takes currently

for us to appear so photorealistic to each other

yeah so I mean for background we both did these scans

for this research project that that we have at meta called codec avatars

and the idea is that instead of actually

instead of our avatars being cartoony and instead of actually transmitting a

video what it does is we've sort of scanned

ourselves and a lot of different expressions and

we've built a computer model of sort of each of our

faces and bodies and the different expressions that we make

and collapse that into a codec that then when you have the headset on your head

it can it sees your face it sees your expression

and it can basically send an encoded version of what you're supposed to look

like over the wire so in addition to being photorealistic

it's also actually much more bandwidth efficient

than transmitting a full video or especially a

3d immersive video of a whole scene like this

and it captures everything like the flaws

like to me the subtleties of the human face like even the flaws

that's like that's all amazing it makes you uh

it makes it so much more immersive it makes you realize that like perfection

isn't the thing that leads to immersion it's like the little subtle flaws like

freckles and like variations in color and just yeah

wrinkles all stuff about noses yeah asymmetry

and just the different like the corners of the eyes like what your eyes do when

you smile all that kind of stuff yeah eyes are a huge part of it

yeah i mean there's all the studies that most of communication even when

people are speaking is not actually the words that they're saying right it's

kind of the expression and all that so and we try to capture that with the

kind of classical expressive avatar system that we have that's the kind of

more cartoon designed one you can you can kind of put those kind of

expressions on those faces as well but there's obviously a certain realism

that comes with delivering kind of this photorealistic experience that

i don't know i just think it's really magical i mean this gets to

kind of the core of what the vision around virtual and augmented reality is

of like delivering a sense of presence as if you're

there together no matter where you actually are in the world and

i mean this this uh experience i think is a good embodiment of that where it's

like i mean we're in two completely different states halfway across the

country and it just like you know looks like

you're just sitting right in front of me it's uh it's pretty wild

yeah yeah i can't it's almost almost getting emotional it's like it feels

like a totally it's a fundamentally new experience

like for me to have this kind of conversations with loved ones

it would just change everything maybe just to elaborate

so the i went to pittsburgh and went through the whole scanning procedure

which has so much incredible uh technology so

software and hardware going on but it is a lengthy process so what's your

vision for the future of this uh in terms of making this more

accessible to people you know it starts off

with a small number of people doing these um very detailed scans right which

is this that's the version that you did and

that i did and you know before there are a lot of people who

who've done this kind of a scan for we probably need to kind of over collect

expressions um when we're doing the scanning because we haven't figured out

how much we can reduce that down to a really streamlined process

um and extrapolate from the the the scans that have already been done

but you know the goal and we have a project that's working on this already

is just to do a very quick scan with your cell phone

where you just take your phone kind of wave it in front of your face for a

couple of minutes um you know say a few sentences make a

bunch of expressions but overall have the whole

process just be two to three minutes and then produce something that's of the

quality of what we have right now so i think that that's one of the big

challenges that remains and right now we have the ability to do the scans if you

you know have hours to sit for one and with today's technology i mean you're

using a meta headset that exists it's a product that's

kind of for sale now you can drive these with that

um but the production of of these scans in a very efficient way

is one of the last pieces that we still need to really nail

and then obviously there's all the experiences around it i mean right now

we're kind of sitting in a dark room which um you know is is you know

familiar for for your podcast but i think part of the

vision for this over time is um is you know not just having this be like a

video call i mean that's fine it's it's cool or it feels

like it's immersive but um you know you can you can do a

video call on your phone the thing that you can do in the metaverse

that is different from what you can do on a phone is like doing stuff where

you're physically there together and and participating in things together and we

could play games like this um we could have meetings uh like this

in the future once you mix um once you get mixed reality and augmented

reality we could have codec avatars like this

and go into a meeting and have some people physically there and have some

people show up in this photorealistic form uh super imposed on the on the

physical environment i think that stuff like that is going to be super

powerful so we got to still build out all those

kind of applications and the use cases around it but i don't know i think it's

going to be a pretty wild uh next few years around this

i mean i just i i'm actually almost at a loss of words this is just so incredible

this is truly incredible i i hope that people like watching this can

get a glimpse of like how incredible it is it really feels like we're in the same

room like there is that um i guess there's an uncanny valley

that seems to have been crossed here like it looks like

you yeah i mean i think there's still a bunch of tuning

that i think we'll want to do where different people

emote to different extents right so i think one of the big questions

is you know like when you smile how wide is your smile and how wide do you want

your smile to be um and i think getting that to

be tuned on a per person basis is um is going to be one of the things

that we that we're going to need to figure out

um you know it's like to what extent do you want to give people control over

that um you know some people might try to

you know might might prefer a version of themselves that's more emotive in their

avatar than their actual faces you know so for example you know i i always get a

lot of um critique and and shit for um for for having

like a relatively stiff uh expression but you know i mean i might i might feel

pretty happy but just make a pretty small smile so i mean maybe you know for

me i would it's actually you know it's like i'd want to have my avatar really

be able to better express um like how i'm feeling

than than what than how i can do physically so i think that there's a

question about how you want to tune that but uh but overall yeah i mean you

we want to start from the baseline of capturing how people actually emote and

express themselves and i mean i think the the initial version of this is

has been pretty impressive and like you said um i do think we're we're kind of

beyond the the uncanny valley here where it and it does feel like you it doesn't

feel uh it doesn't feel weird or anything like that

i mean that's going to be the meme that the two most monotone people are in a

metaverse together but i think that actually makes it

more difficult like the the the amazing thing here is that the subtleties of

the expression of the eyes you know people say i'm

monotone and emotionalist but i'm not it's just this maybe my expression of

emotion is more subtle usually like with the eyes

and that's one of the things i've noticed is just how expressive the subtle

movement of the corners of the eyes are in terms of displaying happiness or

boredom or all that kind of stuff i am curious to see

just i've never done one of these before i've never done a podcast as is one of

these kodak avatars um and i'm curious to see what

what how what people think of it because you know one of the issues that

we've had in some of the vr and and mixed reality work is

it tends to feel a lot more profound when you're in it

than the 2d videos capturing the experience so i think that this one

because it's photorealistic um may look kind of as amazing

in 2d for people watching it as it as it feels i think to be in it

but we've certainly had this this um this issue where a lot of the other

things just it's like you feel the sense of immersion when you're in it that

that doesn't quite translate to a 2d screen but i don't care i'm curious to

see to see what people think yeah i'm curious to see if people could see

that um like my heart is actually beating fast now

this is super interesting like the that such

intimacy of conversation could be achieved remotely

this been that you know i don't do remote podcast for this reason

and this is like breaks all that this feels like just an incredible

transition to something else to the different kind of communication

breaks all barriers like geographic physical barriers

uh what you mentioned do you have a sense of timeline in terms of how many

difficult things have to be solved um make this more accessible

to like scanning with a smartphone yeah i mean i think we'll probably roll

this out progressively over time so it's not going to be like we rolled out one

day everyone has a codec avatar um we want to get more people scanned and

into the system and then we want to start uh integrating it

into each one of our apps right making it so that you know i think that

for a lot of the work style things productivity i think that this is going

to make a ton of sense in a lot of game environments i mean this could be fine

but games tend to have their own style right where you almost want to fit more

with the aesthetic style of the of the game um but i think for doing meetings

and one of the things that we get a lot of feedback on work rooms

where you know people are pretty blown away by the experience and this feeling

that you can like be remote but feel like you're physically

there around a table with people but then you know we get some feedback that

people have a hard time with the fact that the avatars are so expressive and

and don't feel you know as as realistic in that environment so i think something

like this um could make a very big difference for those remote meetings

and especially with quest 3 coming out which is going to be the first mainstream

mixed reality product right where you're really taking digital um you know

expressions of either a person or or objects and overlaying them on the

physical world um i think the ability to do

kind of remote meetings and and things like that where you're like

just remote hang sessions with friends i mean i i think that that's going to be

very exciting so yeah rolling it out over the next

over the next few years it's not ready to be like a kind of mainstream product

yet but um we just want to we'll keep tuning in keep

getting more scans in there and keep you know and kind of rolling it out into

more of the features but yeah i mean definitely in the next in the next um

few years it will you'll be seeing a bunch more experiences like this

yeah i would love to see some celebrities scanned and some non-celebrities

i just just more people to experience this i would love to see that this is

something i mean on my mind is but i'm literally at a loss

because it's very difficult to just convey how incredible this is how like

how i feel the emotion how i feel the presence how i feel like the subtleties

of the emotion in terms of like work meetings or any kind of in

terms of podcasts this is like this is awesome i don't even need your arms or

legs is that well we got we got we got to get that

i mean that's that's its own challenge and part of the question is also so you

have the scan then it takes a certain amount of

compute to go drive that both for the sensors on the headset and

and then rendering it so one of the things that we're working through

is what is the level of fidelity that is optimal right you could do the full body

in in kind of a codec and that can be quite intensive

but but one of the things that we're we're thinking about is like all right

maybe you can kind of stitch a somewhat lower fidelity version of your body

but still still have the main kind of the major movements

but but your face is really the thing that we have the most resolution on

right in terms of being able to read and express emotions i mean like you said

if you move your you know eyebrows like a millimeter i mean that really changes

the expression what you're you're emoting whereas

you know i mean moving your your arm like a

an inch probably doesn't matter quite as much so so yeah so i think that will

we do want to get all of that into here and and that'll be some of the work

over the next period as well so you mentioned quest three

that's coming out i've gotten a chance to try that too that's awesome

so the how'd you pull off the mix so it's not just virtual reality it's mixed

reality yeah i mean i think it's going to be it's

going to be the first mainstream um mixed reality device i mean obviously we

shipped quest pro um last year but it was $1,500

um and well part of what i'm super proud of is you know we try to innovate not

just on pushing the state of the art and

delivering new capabilities but making it so it can be available to

everyone and you know we have this and it's

coming out it's $500 and um in in some ways i think the mixed reality is

actually better in quest three than it was um

than than what we're using right now in quest pro so and i'm really proud of the

team for being able to deliver that kind of an innovation and get it out but

you know some of this is just um software you tune over time and get to be

better part of it is you put together a product and you figure out

what are the bottlenecks in terms of making it a good experience so we got

the resolution for the mixed reality cameras and

sensors to be multiple times better in quest three

and we just figured that that that made a very big difference when we saw the

experience that we were able to put together for quest pro um and part of

it is also that you know qualcomm just came out with their next generation

chipset for for vr and mr that we worked with them on a kind of custom

version of it um but that was available this year for quest three and it wasn't

available in quest pro so you know in a way i'm quest three even though it's not

you know the pro product um actually has a stronger chipset in it than the pro line

at a third of the cost so um so i'm i'm really excited to get this in people's

hands it um it does all the vr stuff that that quest two and the others are done

too it does it better because the display is

better um and and the chip is is better so you'll

get better graphics uh it's 40 thinner so it's um

so just more comfortable uh as well but but the mr is really the big capability

shift and i'm part of what's exciting about the whole space right now is you

know this isn't like smartphones where you know companies put out a new

smartphone every year and you can almost barely tell the difference between

that and the the one year before it now for this each time we put out a new

headset it has like a major new capability and and the big one now um is is mixed

reality the ability to basically take digital representations of people um or

objects and and superimpose them on the world and basically you know i mean

there's a one version of this is you're gonna kind of have these augments or

or holograms and and experiences that you can kind of bring into your

living room or a meeting space or office um

another thing that i just think is going to be a much

kind of simpler innovation is that there are a lot of VR experiences today that

don't need to be fully immersive and you know if you're playing a shooter game

or you're doing a fitness experience and sometimes people get worried about

swinging their arms around like am i going to hit a lamp or or something you

know it's and and am i going to run into something

so having that in mixed reality actually is just a lot more comfortable for

people right you you kind of still get the immersion in the 3d experience um

and you can you can have an experience that just wouldn't be possible in the

physical world alone but by being anchored to and being able to see the

physical world around you it's like it just feels so much safer and

more secure and i think a lot of people are really going to enjoy that too so

yeah i'm really excited to see how people use it but yeah quest three coming

out um later this fall yeah and i got to experience it with

other people sitting around and there's a lot of furniture and so you get to see

that furniture you get to see those people and you get to see those people

like enjoy the ridiculousness of you like swinging your arms i mean

presumably they're friends of yours even if they make fun of you they uh

there's a lot of love behind that and that you guys got to experience that so

that's a really fundamentally different experience than just pure vr with like

with zombies coming out of walls and yeah it's like someone

shooting at you and you hide behind your real couch in order to duck the fire

it's incredible how it's all integrated but also like subtle stuff like

in a room with no windows you can add windows to it and you can look outside

as the zombies run towards you but like it's still nice view outside

you know yeah it's it's really and so that's pulled off by having cameras on

the outside of the headset that do the pass through

uh that technology is incredible to do that on a small headset

yeah it's not just the cameras you basically need to you need multiple

cameras to capture the different angles and and sort of the

three-dimensional space and then it's a pretty complex

compute problem an ai problem to map that to your perspective right because the

cameras aren't exactly where your eyes are because no two people's eyes are

you know going to be in exactly the same place you kind of need to

to to get that to to to line up um and then do that basically in real time

and then generate something that looks that kind of feels natural um

and then superimpose whatever digital objects you want to put there so it's

yeah it's a it's a very interesting technical

challenge and um i think we'll continue tuning this for for the years to come

as well but uh but i'm pretty excited to to get

this out because i think quest three is going to be the first device like this

with that millions of people are going to get that's a mixed reality and it's

only when you have millions of people using something that you start getting

the whole developer community really starting to experiment and build stuff

because now there are going to be people who actually use it um

so i think we'll get you know we got some of that flywheel going with quest pro

but i think it'll really get accelerated once quest three gets out there

so yeah i'm i'm pretty excited about this one plus there's hand tracking

without so you don't need to have a control so this camera the cameras

aren't just doing the pass through uh of the entire physical reality around

you it's also tracking the details of your hands in

order to use that for like gesture recognition this kind of stuff

yeah we've been able to get way further on hand recognition

in a shorter period of time than i expected so that's been pretty cool i

don't know did you see the the demo um experience that we built around

um piano like yeah the piano learning to play piano

yeah it's incredible you're basically playing piano on a table and it's that's

without any controller and like how well it matches

physical reality with no latency and it's tracking your hands with no

latency and it's tracking all the people around you with no latency

integrating physical reality and digital reality obviously that connects

exactly to this uh kodak avatar which is in parallel

allows us to have ultra realistic copies of ourselves in this mixed reality

it's uh so like it's all converging towards like an incredible digital

experience in the metaverse to me obviously i love the intimacy of

conversations so even this is awesome but do you have other

ideas of what this unlocks of like something like kodak avatar unlocks

in terms of applications in terms of uh things we're able to do

well there's what you can do with avatars overall in terms of super

imposing digital objects on the physical world

um and then there's kind of psychologically what is having photo

realistic do um you know so i i think we're moving towards a world where

you know we're gonna have something that looks like normal glasses

where you can just see you see the physical world be able to see holograms

and in that world i think that they're gonna be

you know not too far off you know maybe you know by the end of this decade

we'll be living in a world where there are kind of as many holograms when you

walk into a room as there are physical objects

and it really raises this interesting question about

what are um about you know a lot of people have this

phrase where they they call the physical world the real world

and you know i kind of think increasingly and the physical world is

super important but i actually think the the real world is the combination of

the physical world and the digital worlds coming together but until this

technology they were sort of separate right it's like you

access the digital world through a screen right and you know maybe it's a

small screen that you carry around or it's a bigger screen when you

sit down at your desk and you strap in for a long session but

um but they're they're kind of fundamentally divorced and disconnected

and i think part of what this technology is going to do

is bring those together into a single coherent experience of what the modern

real world is which is and it's got to be physical because that we're physical

beings so the physical world is is always going to be super

important but but increasingly i think a lot of the things that we

kind of think of um can be digital holograms i mean any

screen that you have can be a hologram um you know any

media um in any book art um you know it can

basically be just as effective as a hologram as a physical object any game

um that you're playing a board game or um or any kind of physical game cards

you know ping pong things like that they're they're often a lot better as

holograms because you could just kind of snap your fingers and instantiate them

and have them um show up you know it's like you

have a ping pong table show up in your living room but then you can snap your

fingers and have it be gone um so that's super powerful um

so i think that it's actually an amazing thought experiment of like

how many physical things we have today that could actually be better as

interactive holograms but then beyond that i think the

the most important thing obviously is people so the ability to

you know have these mixed hangouts whether they're social

or meetings where you show up to a conference room

you're wearing glasses um or a headset in the very near term but you know

hopefully by you know for the next five years glasses or so

and um and you know you're there physically some people are there

physically um but other people are just there as

holograms and it feels like it's them um who are right there and

also by the way another thing that i think is going to be fascinating about

being able to blend together the digital and physical worlds in this way

is we're also going to be able to embody um

ai's as well so i think you'll also have meetings in the future

where you're basically you know maybe you're sitting there physically

and then you have you know a couple of other people who are there as holograms

and then you have like bob the ai who's an engineer on your team who's helping

with things and he can now be embodied as a

you know as a realistic avatar as well and just join the meeting

in that way so i think that that that's going to be pretty compelling

as well so then okay so what can you do with photo realistic avatars

compared to kind of the more expressive ones that we have today

well i think a lot of this actually comes down to

acceptance of the technology um and because all of the stuff that we're

doing i mean the the motion of your eyebrows the

motion of your eyes the cheeks and and all of that

there's actually no reason why you couldn't do that on an expressive avatar

too i mean it wouldn't look exactly like you but i mean you can make a cartoon

version of yourself and still have it be um almost as expressive

but i i do think that there's this bridge between

the current state of most of our interactions in the physical world

and where we're getting in the future with this kind of hybrid

physical and digital world where i think it's going to be a lot easier for

people to kind of take some of these experiences

seriously with the photo realistic avatars to start and then

i'm actually really curious to see where it goes longer term i could see a

world where people stick to the photo realistic and maybe they

modify them to make them a little bit more interesting but

maybe fundamentally we like photo realistic things um

but i can also see a world that once people get used to the photo the

photo realistic avatars and they get used to these experiences

that i i actually think that there could be a world where people actually

prefer um being able to express themselves in kind of non

you know ways that aren't so tied to their physical reality and

so i that's one of the things that i'm really curious about and

i don't know in a bunch of our internal experiments on this one of the things

that has i thought was psychologically pretty

interesting is people have no issues blending photo

realistic stuff and not so you know we could have a you know

for this specific scene that we're in now we

we happen to sort of be in a dark room um i think

part of that aesthetic decision i think was based on the way you like to do

your podcast but we've we've done experiences like this

um where you have like a cartoony background

but photo realistic people who you're talking to

and we seem to like people just seem to just think that that is completely

normal right it doesn't bother you it doesn't feel like it's weird

another thing that that we have experienced with is um

is basically you have a photo realistic avatar that you're talking to

and then right next to them you have an expressive kind of cartoon avatar

and that actually is pretty normal too right it's it's like

it's not that weird right to to basically being

interacting with with different people in different modes like that so

i'm not sure i think it'll be an interesting question to what extent

these photo realistic avatars are like a key part of

just transitioning from being comfortable in the physical world to this

kind of new modern real world that that kind of includes both the digital and

physical or if this is like the long term way that it

stays um that's that's a i mean i think that they're going to be

useless for both the expressive and the photo realistic over time i just don't

know what the balance is going to be yeah it's a really good

interesting philosophical question but to me in the short term the photo

realistic is amazing to where i would prefer

like you said the work room but like on a beach with a beer

just to see a buddy of mine remotely on a chair next to me

drinking a beer i mean that as realistic as possible is an incredible

experience so i don't want any fake hats on him i don't want any

just chilling with it with a friend drinking beer looking at the ocean

while not being in the same place together i mean that

yeah that experience is just it's a fundamentally uh

it's just a high quality experience a friendship whatever we seek in friendship

it seems to be present there in the same kind of realism i'm seeing right now

this is this is totally a game changer so to me this is i can see myself

sticking with this for a long time yeah and and i mean it's also its novel

and it's also a technological feat right it's like being able to pull this off

is like it's a it's like a pretty impressive and i think to some degree

it's just this kind of like awesome experience yeah um but i'm already

sorry to interrupt i'm already forgetting that you're not real

like this really so i am novel it's it's just a

an avatar version of me but it's a deep philosophical question yes

but i mean but here's some of the so i put this on this morning and

i was like all right like it's like okay so this my hair is a little shorter in

this than my physical hair is right now i probably need to go get a haircut

um and like and i actually i did happen to shave this morning but

but if i hadn't you know i could still have this photorealistic avatar that is

that is more cleanly shaven right even if i'm

you know a few days in um physically so i do think that they're gonna start to be

these subtle questions that seep in where the the avatar is realistic

um in in the sense of this is kind of what you looked like at the time of

capture but it's not necessarily temporarily accurate

to exactly what you look like in this moment and

i think that we're gonna end up being um a bunch of questions

that come from that over time that i think are gonna be fascinating too

you mean just like the nature of identity of who we are are we the

people you know how people do like like summer beach body

with people be for the scan they'll try to lose some weight and look their best

and sexiest with the nice hair and everything like that

i mean it does um it does raise the question of

you know if a lot of people interacting with the digital version of ourselves

who are we really are we the the entity driving the avatar or are we the avatar

well i mean i think our physical bodies also fluctuate and change over time too

so i think there's a similar question of like which version of that are we

right there's there's like the i mean it's and it's interesting identity

question because all right it's like i don't know it's like

weight fluctuates or things like that it's like i i think most people don't

tend to think of themselves as the uh well i don't know it's it's an

interesting psychological question some maybe some people

maybe a lot of people do think about themselves as the kind of worst version

um but you know but i think a lot of people probably think about themselves

the best version and and i and then it's like what you are

on a day-to-day basis doesn't necessarily map to to um

to either of those so i think that's yeah there will definitely be a bunch of

a bunch of social scientists and and folks will have to

you know and psychologists are really there's going to be a lot to

understand about how our perception of ourselves and others

um shifted from this well this might be a bit of a

complicated and a dark question but one of the first feelings i had

experience in this is i would love to talk to loved ones

and the next question i have is i would love to talk to people who are not

longer here that are loved ones so like if you look

into the future is that something you think about who people pass away

but they can still exist in the metaverse you can still have

you know talk to your father talk to your grandfather and grandmother and

a mother once they pass away the power of that

experience is one of the first things my mind jumped to because it's like

this is so real yeah i think that there are a lot of norms and

things that people have to figure out around that

there's probably some balance where you know if someone is has lost

a loved one and is grieving there there may be

ways in which you know being able to interact or relive certain memories

could be helpful but then there's also probably an extent to which it

could become unhealthy and i mean i'm not an expert in that so i

think we'd have to study that and understand it in more detail

we have you know a fair amount of experience

with how to handle death and identity and people's digital content through

social media already unfortunately right with her

you know there's you know unfortunately you know people

who use our services die every day and their families

you know often want to have access to their profiles and we have whole

protocols that we go through where you know there's certain parts of it that

that we try to memorialize so that way the the family can

get access to it so that way the account doesn't just go away immediately

but then there are other things that are you know important kind of private

things that that person has like we're not going to give the family access to

someone's messages you know for example so

so yeah i think that there's there's some best practices i think from

the current digital world that will carry over but

but yeah i think that this will enable some different things

another version of this is is how this intersects with ai's

right because and one of the the things that

that we're really focused on is you know we we want there to

we want the world to evolve in a way where there isn't like a single ai

superintelligence but where you know a lot of people are empowered by having

ai tools to to do their jobs and you know make their lives better

and if you're a creator right and if you run a

you know podcast like you do then you have a big community of people who

are super interested to talk to you i know you'd love to you know cultivate

that community and you interact with them online outside of the

the podcast as well but i mean there's way more demand both to interact with

you and i'm sure you'd love to interact with

the community more but you just are limited by the number of hours in the

day so and with some point i think making it so that

you could build an ai version of yourself that could interact with people

you know not after you die but but while you're here to help

you know help help people kind of fulfill this desire to interact with you

and your desire to build the community and there's a a lot of

interesting questions around that and you know that's obviously it's not just in

in the metaverse i think you know we we'd want to make that work

you know across all the messaging platforms you know whatsapp and

messenger and instagram direct but you know there's certainly you know a

version of that where if you could have an avatar

version of yourself in the metaverse that people can interact with and you

could define that sort of an ai version where you know people

know that they're interacting with an ai that it's not you know the the kind of

physical version of you but maybe that ai even if they

know it's an ai is the next best thing because they're probably not going to

you know necessarily all get to interact with you directly

i i think that that could be a really compelling experience there's a lot of

things that we need to get right about it

that you know it's we're not ready to release the the version that a creator

can can kind of build a version of themselves yet but we're

starting to experiment with it in terms of releasing

a number of ai's that people can interact with in different ways

and i i think that that is is also just going to be a very powerful

you know set of capabilities that people have over time

so you've made major strides in developing these

early ai personalities um with the idea where you can talk to them across the

meta apps and have like interesting unique kind of

conversations what can you describe your vision there and these early strides

and what are some technical challenges there

yeah so i mean a lot of the vision comes from

this idea that yeah i i don't i don't think we necessarily want there to be

like one big superintelligence we want to empower

everyone to both you know have more fun accomplish their business

goals you know just everything that that they're

trying to do and you know we don't tend to have you know one person that we

work with on everything and i don't think in the future we're going to have

you know one ai that we work with i think you're going to want a

variety of these um so there are a bunch of different uses

um if some will be kind of more assistant oriented there's a

sort of the kind of plain and simple one that we that we're building is called

just meta ai it's simple it you can chat with it in any of your threads

it doesn't have a face right it's it's just it's

it's just kind of more vanilla and and neutral and kind of factual

but it can help you with a bunch of stuff then there are a bunch of cases

that are more kind of business oriented so let's say you want to contact a

a small business um you know similarly you know that

business probably doesn't want to have to staff someone to man the phones

and you probably don't want to wait on the phone to talk to someone but you're

having someone who you can just like talk to in a natural way who can

you know help you if you're having an issue with a product or if you want to

make a reservation or if you want to you want to buy something online

um having the ability to to do that and have a natural conversation

rather than navigate some website or have to call someone and wait on hold

um things will be really good both for the businesses and for for

normal people who want to interact with businesses so i think stuff like that

makes sense um then there are going to be a bunch of

use cases that i think are just fun right so i think people are going to

i think that there will be ai's that i can tell jokes so you can put them into

chat thread with friends i mean i think a lot of this because we're like a

social company right i mean we're you know fundamentally around

helping people connect in different ways and part of what i'm what i'm excited

about is how do you enable these kind of ai's to

facilitate connection between two people or more

you know put them in a group chat you know make the group chat more interesting

um around whatever your interests are sports fashion

um trivia video games i love the idea of

of playing i think you mentioned balder's gate an incredible game

just having an ai that you played together with

i mean that could that seems like a small thing but it could

deeply enrich the like gaming experience i do think that ai's will be

will make the npc is a lot better in games too so that's a

a separate thing that i'm pretty excited about but um

but yeah i mean one of the one of the ai's that we've built that

just in our internal testing people have loved the most is like a

like a adventure text-based um like a dungeon master

yeah nice and and i i think um you know part of what what has been

fun and we talked about this a bit but we've gotten some like real kind of

cultural figures to play a bunch of these folks and be the

embodiment and the avatar of them so um so snoop dog is the dungeon master

which i think is just hilarious in terms of the next steps of

you know if you mentioned you mentioned snoop to create a snoop ai

so basically ai personality replica a copy

or not a copy maybe um inspired by snoop what are the some of the technical

challenges of that what does that experience look like for snoop

to be able to yeah so that so starting off creating new personas

is easier because it doesn't need to stick exactly to what you know that

physical person would want how they'd want to be represented right it's like

it's just a new character that we created so even though

there's a snoop in that case is you know he's um you know he's basically an

actor right he's playing the the dungeon master but it's not snoop dog

right it's it's um you know whoever the the dungeon master is

um if you want to actually make it so that you have an ai embodying

a real creator there's a whole set of things that you need to do to make sure

that that ai is not gonna say things that the

creator doesn't want right and um and that the ai is gonna you know

know things and be able to represent things in the way that the creator would

want um the way that the creator would know um

so i think that it's less of a it's less of a question around like

having the avatar express them i mean that that i think we're you know it's

like well we have our kind of v1 of that that will

release soon um is after connect but you know that'll get better over time

but a lot of this is really just about continuing to make the the models for

these ai is it's that they're just more and more

i don't know you could say like reliable or predictable in terms of what

they'll communicate it's that way you know when you want to create the lex

assistant ai that that your community can talk to

you can you know it's you don't program them like normal computers you're

training them their ai models not not not um kind of normal

computer programs but um but you want to get it to be predictable enough

so that way you can set some parameters for it and

even if it isn't perfect all the time um you want to generally be able to stay

within those bounds so that's a lot of um what what i think we need to

nail for for the creators and that's why that one's

actually a much harder problem i think than starting with

with uh with new characters that you're creating from scratch so

that one i think will probably um start releasing sometime next year

not this year but experimenting with existing characters and the assistant

and games and a bunch of different personalities and

experimenting with some small businesses um i think that that stuff will be

ready to do this year and we're rolling it out you know basically right after

connect yeah i'm deeply entertained by

the possibility of me sitting down with myself and saying hey man

like you need to stop the dad jokes or whatever i think the idea of a podcast

between you and ai assistant lex podcast

i mean there is uh just even the experience of a kodak avatar

being able to freeze yourself like basically first mimic yourself so

everything you do you get to see yourself do it that's a surreal experience

that feels like uh if i was like an ape looking in a mirror for the first time

realizing like oh that's you but then freezing that

and being able to look around like i'm looking at you

it's uh i don't know how to put it into words but it just feels like a

fundamentally new experience like i'm seeing maybe color for the first time

seeing i'm experiencing a new

way of seeing the world for the first time because it's physical reality

but it's digital like and realizing that that's possible is just it's

it's blowing my mind this is really exciting because i live most of my life

you know before the internet and and and experiencing the internet

experiencing voice communication video communication

you think like well there's a ceiling to this but this is making me feel like

there might not be there might be that blend of physical reality and

digital reality it's actually what the future is

yeah i think it's a weird experience it's a it feels like the early days of

of like a totally new way of living and like there's a lot of people that kind

of complain well you know the internet is not that's not

reality you need to turn all that off and go you know in nature but this

feels like this will make those people happy

i feel like because it feels real the flaws and everything

yeah well i mean a big part of how we're trying to design these

these new computing products is that they should be physical

right i think part that's a big part of the issue with computers and

tv's and even phones is like yeah i mean maybe you can interact with them in

different places but they're they're fundamentally like you're sitting you're

you're still and i mean people are just not meant to be that

way i mean i think you and i have this shared

passion for sports and martial arts and doing stuff like that we were just

moving around it's like so much of what makes us people is like

you know you move around you're not we're not just like a brain in a tank

right it's the where you know the human experience is a physical one

and so it's it's not just about having the immersive

expression of the digital world it's about being able to really natively

bring that together and and i do really think that the

the real world is this mix of the physical and the digital right the

digital is there's too much digital at this point for it to just be solid to a

small screen but the physical is too important so you

don't want to just sit down all day long um at a desk

so i i think that this is uh yeah i do think that this is the future this is i

think the kind of philosophical way that i would

want the world to work in the future is a much more coherently blended

physical and digital world there may be some

difficult philosophical and unethical questions we have to figure out as a

society uh maybe you can comment on this

so the the metaverse seems to enable sort of unlock a lot of

experiences that we don't have in the physical world

and the question is like what is and isn't allowed in the metaverse

you know in video games we allow uh all kinds of crazy stuff and

in physical reality you know a lot of that is illegal

so where's that line where's that gray area between video game and physical

reality do you have a sense of that well i think i mean there

there are content policies and things like that right and in terms of what

what people are allowed to create but i mean a lot of the rules around physical

i think we try to have a society that is as free as possible

meaning that people can do as much of what they want

unless you're going to do damage to other people

and and infringe on on their rights and the idea of damage is somewhat

different in a in a digital environment i mean when i get into

you know uh some world with my friends the the first thing we start doing is

shooting each other which obviously we would not do in the physical world

because you'd need to hurt each other but in in a game

that's like just it's almost you know it's like just fun and

um i'm even in like the lobby of a game right it's like it just it's not even

bearing on the game it's just kind of like a funny

um sort of humorous thing to do so it's like is that is that problematic i

don't think so because it's it's fundamentally it's not you're not

causing harm in that world so i think that the um part of the

question that i think we need to figure out is what are the ways

where things could have been harmful in the physical world that we will now be

freed from that and therefore there should be fewer restrictions in the

digital world um and then there might be new ways in which

there could be harm in the digital world that there weren't the case before

so there's more anonymity right it's um you know when you when you show up to a

you know a restaurant or something it's like all the norms where you pay the

bill at the end it's because you know you you

you have one identity and you know the you know if you if you

stiff them then like you know life is a repeat game and

that's not going to work out well for you but you know in a digital world where

you can be anonymous and show up in different ways um i think the incentive

to act like a good citizen can be a lot less and that causes a lot of issues

and toxic behavior so that that needs to get sorted out um so i think in terms

of what is allowed i think you want to just look at what what what are the

the damages but then there's also other things that are not

related to kind of harm you know less about what should be allowed

and more about what will be possible that are more about the laws of physics

so right it's like if you wanted to travel uh to see me in person

you'd have to get on a plane and and that would like

you know take a few hours to get here whereas you know we could just jump in

a conference room and you know put on these headsets and

we're basically teleported into a space where we're you know it feels like

we're together so that's a very novel experience

that um that it breaks down some things that previously would have

defied the laws of physics for what it would take to get together

and i think that that will create a lot of new opportunities right so

um and one of the things that i'm curious about is you know there are all

these debates right now about you know remote work or people being together

and you know i think this gets us a lot closer to being able to work physically

in different places but actually have it feel like we're together

so you know i think that the dream is that is that people will one day be able

to just work wherever they want but we'll have all the same opportunities

because you'll be able to feel like you're physically together i think

we're not there today with with um with just video conferencing and the basic

technologies that we have but i think part of the idea is that with something

like this over time you can get closer to that and that would open up a lot of

opportunities right because then people could live physically

where they want while still being able to get the benefits of being physically

or kind of feeling like you're together with people at work all the

ways that that helps to build more culture and build better relationships and

build trust which are the real issues that if you're not seeing people

you know in person ever so yeah i don't know i think it's going to be

it's very hard from first principles to think about all the implications

of um of a technology like this and you know all the good and and and the

things that you need to mitigate so you try to do your best to kind of envision

what things are going to be like and accentuate the things that they're

going to be awesome and hopefully mitigate some of the the downside things

but i you know it's the reality is that we're going to be building this out

one year at a time it's going to take a while um so we're going to just get to

see how how how it evolves and and what developers and different folks do with it

uh if you could comment this might be a a bit of a very specific technical

question but llama 2 is incredible it's the you've released it recently

um there's already been a lot of exciting developments around it

is there what what's your sense about its release and

is there a llama 3 uh in the future yeah i mean i think on the last podcast

that we did together we were talking about the debate that we were having

around open sourcing llama 2 and i'm i'm i'm glad that we did um you know i

think at this point there's the the value of open sourcing

a foundation model like llama 2 is significantly greater than um than the

than the risks and in my view i mean we did we spent a lot of time

took a very rigorous assessment of that and red teaming it um but i'm i'm very

glad that we released llama 2 i think the reception has been

um it's it's just been really exciting to see how excited people have

have been uh about it and it's gotten way more

you know downloads and usage than i than i i would have even expected and i was

pretty optimistic about it um so that's that's been great

um llama 3 uh i mean there's always another model

that we're training so i mean it's you know for right now

you know we built we trained llama 2 and we released it as an open source model

and right now the priority is building that into a bunch of the consumer

products all the different ai's and um and a bunch of

different um products that that we're basically building as

consumer products because llama 2 by itself it's not a consumer product right

it's more of a piece of infrastructure that people could

could build things with so that's been the big priority is kind of continuing

to fine tune and um and kind of just get llama 2 and

it's um and it's little the branches that we built off of it

ready for consumer products that hopefully you know hundreds of millions

of people will will um enjoy using those those products and

billions one day but yeah i mean we're also working on

on the future foundation models and um and i don't have anything

new or news news on that i don't know and i don't know exactly when it's going

to be ready um i think just like we had a

debate around llama 2 and open sourcing it um

i think we'll need to have a similar debate and process to red team this and

make sure that this is safe but and my hope is that we'll be able to

to open source this next version when it's ready to but um but that's not that

we're not we're not you know close to doing that this month i mean this is um

that's just it's a thing that we're we're still somewhat early in working on

well in general thank you so much for open sourcing llama 2 and for being

transparent about all the exciting developments around AI

i feel like that's contributing to a really awesome conversation about

where we go with AI and obviously it's really

interesting to see all the same kind of technology integrated into

these personalized AI systems with the AI personas

which i think when you put in people's hands and they get to have

conversations with these AI personas you get to see like

interesting failure cases like where the things are dumb or they go into

weird directions or and we get to learn as a society together what's

what's too far what's interesting what's fun how much

personalization is good how much generic is good and we get to learn all of this

and you probably don't know this yourself like we have to all figure it out by

using it right yeah i mean part of what we're

trying to do with the initial AI's launch is um having a diversity of

different use cases just so that people can try different

things because i don't know what's going to work i mean are people going to like

playing in the text-based adventure games are they

going to you know like having a comedian who

who can add jokes to threads or they can want to interact with

historical figures you know we made we made one of Jane Austin and one of

Marcus Aurelius and i'm curious to see how that goes

i'm excited for both yeah because i'm a big fan i'm excited for both

i have conversations with them i mean yeah that's yeah you know and

and i'm also excited to see you know the internet i don't know if you heard can

get kind of weird um and i applaud them for it so i get that yeah

yeah so it'd be it'd be nice to see how weird they take it what kind of memes

i generated from this and i think all of it is a especially in these early stages

of development as we progress towards agi it's good to learn

by playing with those systems and interacting with them at like a large

scale like you said yeah totally i mean that's why

well so we're starting out with a set and then um

we're also working on this platform that we call AI studio

that's going to make it so that you know over time anyone will be able to create

if one of these ai is almost like they create any other UGC

content across the platform so i'm excited about that i think that to some

degree we're not going to see the full potential of this until

then you just have the full creativity of the whole community being able to

build stuff but there's a lot of a lot of stuff

that we need to get right so um so i'm excited to take this in stages i don't

i don't think anyone out there is really

doing what we're doing here i think that there are there are people who are

who are doing kind of like fictional or consumer oriented character type stuff

but the extent to which we're building it out with

the um you know avatars and expressiveness and

and making it so that they can interact across um you know all the different

apps and um they'll have profiles and you know we'll be

able to engage people on instagram and facebook i i think it's it's just it's

it's going to be really fun well i'm still so we're talking about ai but i'm

still blowing away this entire time that i'm talking to mark zuckerberg

and you're not here but you feel like you're here i've done

quite a few intimate conversations with people alone in a room and this feels

like that so i keep forgetting for long stretches of time

that like we're not in the same room and for me to imagine a future

where i can with a snap of a finger do that with anyone in my life

the way we can just call right now and have this kind of

shallow 2d experience uh to have this experience like we're sitting next to

each other it's like i don't i don't think i can

i don't think we can even imagine what how that changes things

where you can immediately have intimate one-on-one conversations with anyone

that's that might like in a way we might not even predict

changed civilization well i mean this is a lot of the thesis behind the whole

metaverse is giving people the ability to feel like you're

present with someone i mean this is like the main thing i talk about all the

time but i do think that there's a lot to to process about it i mean

from my perspective i mean i i'm definitely here we're just not

we're we're not physically in the same place it's not like you're you know

you're not talking to an ai right here you know this is

um so i think the the thing that's novel is the ability to convey through

technology a sense of almost physical presence

um so the the thing that is not physically real

is um is us being in the same physical place

but uh but but kind of everything else is and i think that that gets to this

somewhat philosophical question about what is the nature of kind of the

modern real world and i i just think that that's it really is this

combination of a physical world and the presence that we feel

but also being able to combine that with this increasingly rich and powerful

and capable digital world that we have and and all of the

the innovation that's getting created there so i i think it's super exciting

because i mean the the digital world is just is just increasing

um in its capability and our ability to do awesome things

but the physical world is so profound and that's a lot of what makes us human is

is that we're we're physical beings so i don't think we want to run away from

that and just spend all day on a screen and that's like you know it's one of

the reasons why i care so much about about helping to shape and

accelerate the these future computing platforms i just think this is so

powerful and it's it's you know even though the

current version of this is like you're wearing a headset

um i just think this is going to be by far the most human

and social computing platform that has ever existed

and i know that that's what what makes me excited

yeah i i i think just to linger on this kind of changing nature of reality like

of what is real maybe shifting it towards the sort of consciousness so

what is real is the subjective experience of a thing that makes it feel

real versus necessarily being in the same physical

space because it feels like we're in the same physical

space yeah and that the conscious experience of it

that's probably what is real not like that the space time

like the physics of it like you're basically breaking physics

and focusing on the consciousness that's what's real

just whatever is going on inside my head but there were a lot of social and

psychological things that go along with that

experience that was previously only physical presence right i think that

there's like an intimacy a trust you know there's a

level of communication because so much of communication is non-verbal and is

based on expressions that you're kind of you know you're you're

sharing with with someone when you're in this kind of environment

and before those things would have only been possible

you know had you know i got on a plane and and flown to austin and

sat you know physically with you in the same place so

i think we're we're basically shortcutting those laws of physics

and delivering the social and psychological benefits of being able to

be present and and feel like you're there with another person which has good

real benefits to anyone in the world and i think

that that like you said i mean i think that is going to be a very profound

thing and that a lot of that is you know that's the

promise of of the metaverse and what you know why

you know i i just why i think that that's the next frontier for

for what we're working on you know i started working on social networks when

they were primarily text or the first version of facebook

your profile you know you had one photo and the rest of it was like lists of

things that you were interested in and and then we kind of went through the

period we were doing photos and you know now we're kind of in the

period where most of the content is video

but there's a clear trend where you know over time

the way that we want to express ourselves and and kind of

get insight and content about the world around us

gets increasingly just richer and more vivid

and i think the ability to be immersed and feel present with the people around

you or the people who you care about is from my

perspective clearly the next frontier it just so happens

that it's incredibly technologically difficult

right it requires building up these new computing platforms and completely new

software stacks to deliver that but i mean i kind of feel like that's what

we're here to do as a company well i really love the connection you

have through the conversation and so for me this photo realism

is really really exciting i'm really excited for this future

and thank you for building it thanks to you and thanks to the amazing meta teams

that i've met the engineers and just everybody i've

met here thank you for helping to build this future and

thank you mark for talking to me inside the metaverse this is blowing my mind i

can't quite express i would love to measure my heart rate this whole time

would be hilarious if you're actually like sitting

on a beach right now i'm not i'm in a conference room

okay well i'm at a beach and if i'm not wearing any pants i'm really sorry

about that for anyone else who's watching me in physical space

anyway thank you so much for talking today this is this this really blew my

mind it's one of the most incredible experiences in my life so thank you for

giving that to me awesome awesome glad you got to check it out and

it's always fun to talk all right i'll catch you soon see you later

this is so so amazing man this is so amazing

Machine-generated transcript that may contain inaccuracies.

Keywords

Mark Zuckerberg, Meta, Quest 3, AI assistants, AI models, physical and digital realities, virtual meetings, AI entities, deceased loved ones, AI versions, Llama 2, consumer products, photorealistic avatars, spatial audio, hand tracking, human perception

People

Lex Fridman, Mark Zuckerberg

Companies

Facebook, Meta

Organizations and Institutions

Metta, Lex Fridman Podcast

Mark Zuckerberg is CEO of Meta. Please support this podcast by checking out our sponsors:

– LMNT: https://drinkLMNT.com/lex to get free sample pack

– InsideTracker: https://insidetracker.com/lex to get 20% off

– Eight Sleep: https://www.eightsleep.com/lex to get special savings

– AG1: https://drinkag1.com/lex to get 1 month supply of fish oil

– NetSuite: http://netsuite.com/lex to get free product tour

Transcript: https://lexfridman.com/mark-zuckerberg-3-transcript

EPISODE LINKS:

Mark’s Facebook: https://facebook.com/zuck

Mark’s Instagram: https://instagram.com/zuck

Mark’s Threads: https://threads.net/@zuck

Meta AI: https://ai.meta.com/

Meta Quest: https://www.meta.com/quest/

Meta Connect 2023: https://www.metaconnect.com

PODCAST INFO:

Podcast website: https://lexfridman.com/podcast

Apple Podcasts: https://apple.co/2lwqZIr

Spotify: https://spoti.fi/2nEwCF8

RSS: https://lexfridman.com/feed/podcast/

YouTube Full Episodes: https://youtube.com/lexfridman

YouTube Clips: https://youtube.com/lexclips

SUPPORT & CONNECT:

– Check out the sponsors above, it’s the best way to support this podcast

– Support on Patreon: https://www.patreon.com/lexfridman

– Twitter: https://twitter.com/lexfridman

– Instagram: https://www.instagram.com/lexfridman

– LinkedIn: https://www.linkedin.com/in/lexfridman

– Facebook: https://www.facebook.com/lexfridman

– Medium: https://medium.com/@lexfridman

OUTLINE:

Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.

(00:00) – Introduction

(08:38) – Metaverse

(23:01) – Quest 3

(37:50) – Nature of reality

(42:28) – AI in the Metaverse

(59:26) – Large language models

(1:05:23) – Future of humanity