[bearjaws]

  “The experiences we’re building for the metaverse require   
  enormous compute power…and RSC will enable new AI models 
  that can learn from trillions of examples, understand
  hundreds of languages, and more,” Meta CEO Mark Zuckerberg 

I don't really understand how AI processing is going to make the 'experiences' any better? This seems to me like investor fluff, saying they have some insane capability that other 'VR providers' don't have...

[rococode]

I think there are plenty of possibilities:

- 3d worlds with style transfer on the textures, like maybe there's a cafe with the visual style of Starry Night or something

- NPCs with conversation models that are finetuned for each NPC's personality and saves some history for each person it talks to for continuity

- Game-playing AI on NPCs that make them go around doing actual things or playing minigames with players

- The usual user tracking models, figuring out what people like to do in the metaverse and giving them more of that

- All the lower-level stuff that AI can do better - user inputs, rendering, etc.

Whether or not they can pull it off is a separate question - I think the tech is close but not quite there yet - but there's no doubt that the metaverse concept of "an expansive virtual world with lots of fun things to do" has many ways to use huge amounts of computation.

[Bombthecat]

Loot boxes with special designs just for you, modeled, picked, designed and coloured after your taste.

[mark_l_watson]

I am not sure either, but I worked on “game AI” over 20 years ago for Nintendo and Disney, and I am 100% sure that I could have used deep learning to good effect if it had been available. In the past seven years, I have been using mostly LSTM and GAN, and recommender models, BTW.

[forgotmyoldacc]

Example 1: GPT-3 is a decent chatbot. Training a similar model so you can have a conversation with AIs in the "metaverse" (god, that word is terrible) could be fun / useful.

Example 2: Using AI upscaling (like Nvidia) to improve visual fidelity in games.

Example 3: Hand/body tracking for avatars.

The more AI compute, the more experimentation researchers can do.

[nerdponx]

I guess "AI in the metaverse" sounds a lot better than "machine learning in Meta Inc's new VR platform".

[giantrobot]

VR advertising and tracking platform.

[d1sxeyes]

One particularly interesting piece of tech I've seen is Nvidia's AI Video 'Compression'.

In summary, rather than actually streaming video to the person you're chatting with, you send a keyframe, and then 'compressed' video is sent over the wire, and 'decompressed' at the receiver end.

I'm putting 'compression' in quotations because to me I'm not sure I'm comfortable calling it compression. Basically, you're remotely controlling an avatar of yourself.

While the obvious usage of this is reducing bandwidth used (in their example, an h264 stream at ~100KB/frame can be compressed to 0.1KB/frame, literally a thousandth of the bandwidth), it opens up some VERY interesting possibilities for a company like Meta (check from about 1:55 onwards in the video below).

You can view someone's face from any angle, not just the angle they're speaking from (as you might in a VR world), or you can even map the key points onto a completely different keyframe, allowing for hyper-realistic avatars or next-level virtual backgrounds (imagine: you send a keyframe of you sitting at your desk and hop on a video conference from the beach, and no-one's any the wiser as long as the sea is quiet enough)

https://developer.nvidia.com/ai-video-compression

[sbierwagen]

A Fire Upon The Deep (1992):

>Fleet Central refused the full video link coming from the Out of Band … Kjet had to settle for a combat link: The screen showed a color image with high resolution. Looking at it carefully, one realized the thing was a poor evocation…. Kjet recognized Owner Limmende and Jan Skrits, her chief of staff, but they looked several years out of style: old video matched with the transmitted animation cues. The actual communication channel was less than four thousand bits per second

>The picture was crisp and clear, but when the figures moved it was with cartoonlike awkwardness. And some of the faces belonged to people Kjet knew had been transferred before the fall of Sjandra Kei. The processors here on the Ølvira were taking the narrowband signal from Fleet Central, fleshing it out with detailed (and out of date) background and evoking the image shown. No more evocations after this, Svensndot promised himself, at least while we’re down here.

[smoldesu]

I'm hardly an advocate for any of Facebook/Meta's actions over the past... however long, but a lot of people forget that "Metaverse" doesn't just mean "virtual reality". VR could be a large component here, but the biggest goal of the Metaverse is really to map physical things into a digital world. That data can be used in any number of ways, not just VR/AR; it could be used to provide 3D models for common shopping goods in the Walmart app, give meteorologists an interactive forecast maps, map GitHub repositories to technology that you use every day... the list goes on. The real goal is ripping digital metadata from real-world objects, which could indeed be inferenced like an AI model for any number of uses.

[rytill]

For one, real-time avatars make heavy use of “AI processing”. https://research.facebook.com/videos/audio-and-gaze-driven-f...

[jayd16]

If computer vision falls under AI then its pretty obvious why it would help with AR and world sensing.

[some_furry]

Any Virtual Reality where I don't have the option of being a talking blue anthropomorphic dhole (i.e. my fursona) isn't one that I'll ever choose to adopt. Calling it a "metaverse" doesn't affect my decision here.

[cm2012]

Zuck has vision, especially for what people will want to use. I am looking forward to what FB comes up with here.

[52-6F-62]

But his vision, in summary, is to dictate a world view. I am not looking forward to what they come up with.

[giantrobot]

Do you get paid in MetaBucks or a real currency? What are the hours and benefits like? Does Zuck wave out a window at you in lieu of cash bonuses?

[cm2012]

I get that it's a very unpopular opinion on HN. A bit of why I post comments like these is to have at least one counterpoint in HN threads that are generally anti Facebook. I am genuinely bullish on the prospects of Facebook as a company based on everything I've seen.

[dvsfish]

I'd rather these absurd empire businesses to at least do something interesting with their empires instead of just make it a bit easier to consolidate money. Zuck is uninspiring and hard to like, but at least this move is somewhat visionary.

[tikimcfee]

It won’t make it better - it makes it more cost efficient to throw random numbers at a random number optimizer to increase the number of times they can report someone clicked or saw an ad. That’s it, end of story.

The value ad is that the engineering community that they employ has a job, the stock stays higher because of their perceived value add to the tech, and the push to control data continues unburdened by something as trivial as a lack of compute power. Hooray. Progress.

[munchbunny]

In practice it'll probably look like indirect improvements in the infrastructure that devs get for computer vision/natural language and other miscellaneous model training stuff.

A lot of this stuff is like trickling tech from F1 teams down into consumer cars. Some of the tech will likely end up in commodity datacenter/cloud stuff.

[macrolocal]

Hm, maybe they're optimizing DLRM models, since inefficient HW communication sometimes bottlenecks Facebook's data center performance for them [1]. The improvements would be better personalization, ie. monetization.

[1] https://arxiv.org/pdf/2104.05158.pdf

[plafl]

There may be some possibilities. I'm not sure if it counts as AI but nevertheless a nice video:

https://m.youtube.com/watch?v=BTETsm79D3A

There is never enough compute power. Dwarf Fortress on a supercomputer?

[c7DJTLrn]

In the past they've used machine learning to do a kind of blurring I can't remember the name of to make scenes in VR look more realistic. They are building a lot of ML models. I think the future of VR is almost intertwined with machine learning.

[rNULLED]

Consider reading https://ai.facebook.com/blog/ai-rsc

[impulser_]

Creating a 3D world with limited amount of data.

[maydup-nem]

> understand hundreds of languages

> understand

I know this is CEO-talk, but I sometimes wonder if these pricks really think they are inventing AI.

[zwaps]

ît is not about making experiences better, it's about modeling behavior as to sell stuff

[varelse]

Those models are surprisingly tractable. You're nowhere near as interesting and unique as you might think you are at scale.

Evidence: actual work experience at building latent representations to characterize customer behavior at FAANG. It's hard to come up with something that really gets you, but it's not hard to come up with something likely to make you spend more. You're surprisingly predictable on that axis and even if you aren't because you put the hours into being a crazy outlier, almost everyone else is, and you don't matter.

[convolvatron]

I wouldn't think it would take putting in any hours to be a crazy outlier - just a markedly different value system.

[varelse]

Which itself requires hours of listening to alternative influences in order to develop, no? The pressure to conform within 2 sigma is strong in almost any society IMO.

What people don't seem to grasp is that most of the supposed alternatives to mainstream are pretty mainstream too or we wouldn't have stores like Hot Topic in the first place.