[dleslie]

This is why I drifted towards game development for most of my career. Consoles, until the penultimate (antipenultimate?) generation, ran software bare or nearly bare on the host machine.

I also spent time in integrate display controller development and such; it was all very similar.

Nowadays it feels like everything rides on top some ugly and opaque stack.

[dijit]

For context of what this guy is saying: the modern Xbox's (after 360) are actually running VMs for each game. This is part of why despite the hardware being technically (marginally) superior: Xbox tends to have lower graphical fidelity.

[mywittyname]

The 360 has a Type 1 (bare metal) hypervisor. So there's not much, if any, performance impact to having it since the software runs natively on the hardware.

Microsoft used a hypervisor primarily for security. They wanted to ensure that only signed code could be executed and wanted to prevent an exploit in a game from allowing the execution of unsigned code with kernel privileges on the system itself.

Every ounce of performance lost to the hypervisor is money Microsoft wasted in hardware costs. So they had an incentive to make the HyperV as performant as possible.

[dijit]

the 360 has no hypervisor.

The CPU had an emulator that you could run on x86 Windows, but it was not itself a hypervisor.

The hypervisor in the XB1 served a more important purpose: to provide developers a way of shipping the custom SDK to clients, and not forcing them to update it. This was quite important for software stability and in fact we made a few patches to MS's XB1 SDK (Durango) to optimise it for our games.

VM's are VM's, there are performance trade-offs.

I know this because I worked on AAA games before in this area, do you also work in games and are repeating something you think. you heard?

[mywittyname]

I don't work in the industry. But just because you "worked on AA games" before doesn't make you correct.

This detailed architectural overview of the 360 discusses the hypervisor:

https://www.copetti.org/writings/consoles/xbox-360/

This YouTuber, who is an industry vet, and has done several xbox ports claims the XB360 has a hypervisor:

https://www.youtube.com/watch?v=Vq1lxeg_gNs

And there entries in the CVE database for the XB360 which describe the ability to run code in "hypervisor mode":

https://www.cvedetails.com/cve/CVE-2007-1221/

This detailed article on the above exploit goes into detail on how the memory model works on the XB360, including how main memory addressing works differently in hypervisor mode than in real mode:

https://www.360-hq.com/article1435.html

That's a whole lot of really smart people discussing a topic that you claim doesn't exist.

[dijit]

Appreciate the detailed reply!

> This detailed architectural overview of the 360 discusses the hypervisor:

> https://www.copetti.org/writings/consoles/xbox-360/

Yes, the 128KB of key storage and W^X. That's not a hypervisor in the sense that the XB1/HyperV or VMWare have a hypervisor, they shouldn't even share a name it's not the same thing at all.

It's like calling the JVM is a virtual machine in the same way QEMU is.

The 360 "Hypervisor" is more akin to a software T2 chip than anything that actually virtualises.

[robocat]

I don’t think you are showing respect when you simplistically repeat your assertion without effort, after two people expended their precious time to tell you in detail that you are wrong with examples. I don’t know anything, but a few minutes following the provided links and I find https://cxsecurity.com/issue/WLB-2007030065 which says:

  The Xbox 360 security system is designed around a hypervisor concept. All games and other applications, which must be cryptographically signed with Microsoft's private key, run in non-privileged mode, while only a small hypervisor runs in privileged ("hypervisor") mode. The hypervisor controls access to memory and provides encryption and decryption services.

  The policy implemented in the hypervisor forces all executable code to be read-only and encrypted. Therefore, unprivileged code cannot change executable code. A physical memory attack could modify code; however, code memory is encrypted with a unique per-session key, making meaningful modification of code memory in a broadly distributable fashion difficult. In addition, the stack and heap are always marked as non-executable, and therefore data loaded there can never be jumped to by unpriviledged code.

  Unprivileged code interacts with the hypervisor via the "sc" ("syscall") instruction, which causes the machine to enter hypervisor mode.

You can argue your own definition of what a hypervisor is, but I suspect you won’t get any respect for doing so.

[black3r]

360 indeed uses hypervisor [0], but uses it only for security, to make the app signature verification run on a higher level.

Windows on PCs also runs under hypervisor if you enable some security features (e.g. VBS/HVCI which are on by default since Windows 11 2022 update, or Windows Sandbox, or WDAG) or enable Hyper-V itself (e.g. to use WSL2/Docker).

The performance losses are indeed there, but by purely running the hypervisor you lose just around 1% [1], because the only overhead is added latency due to accessing memory through SLAT and accessing devices through IOMMU...

I'd imagine XB1 is running with all the security stuff enabled though, which demands additional performance losses [2]..

[0]: https://www.engadget.com/2005-11-29-the-hypervisor-and-its-i...

[1]: https://linustechtips.com/topic/1022616-the-real-world-impac...

[2]: https://www.tomshardware.com/news/windows-11-gaming-benchmar...

[ComputerGuru]

There’s no reason a pass through GPU configuration in a vm would have lower graphical fidelity.

[AstralStorm]

There is a reason but it would only harm particularly poorly written games and even then a single digit percentage.

To excercise that you need a lot of separate memory transfers. Tiny ones. Games tend to run bulky transfers instead of many megabytes.

Memory bandwidth and command pipe should not see an effect even with the minimally increased latency, on any HVM.

[ComputerGuru]

Again with the caveat that this is specific to dom0 virtualization taking advantage of full hardware acceleration VT-d/VT-x, etc, what you say isn’t even necessarily the case.

With modern virtualization tech, the hypervisor mainly sets things up then steps out of the way. It doesn’t have to involve itself at all in the servicing of memory requests (or mapped memory requests) because the cpu does the mapping and knows what accesses are allowed or aren’t. The overhead you’re talking about is basically traversing one extra level in a page table noticeable only on micro benchmarks when filling the tlb or similar - this is a change in performance you might encounter a micro-regression in (without any virtualization to speak of) even when going from one generation of cpu architecture to the next.

Theoretically, the only time you’ll have any overhead is on faults (and even then, not all of them).

Of course I guess you could design a game to fault on every memory request or whatever, but that would be a very intentionally contrived scenario (vs just plain “bad” code).

[dijit]

Hello ComputerGuru,

As you may understand: there's more to graphical fidelity than just the GPU itself.

CPU<->GPU bandwidth (and GPU memory bandwidth) are also important.

There is a small but not insignificant overhead to these things with virtualisation: VMs don't come for free.

[marshray]

"Pass through GPU configuration" means that GPU memory is mapped directly into guest address space in hardware.

Bandwidth from a VM partition should be identical to that from the root partition.

[dijit]

I don’t understand what you’re trying to imply here.

Are you seriously suggesting that I chose to downgrade the graphics on the XB1 because I felt like it, and that dozens of other AAA game studios did the same thing?

Our engine was Microsoft native, by all rights it should have performed much better than PS4.

If you’re going to argue you’ll have to do a lot better than that since I have many years of lived experience with these platforms.

[marshray]

OK, you have a technical disagreement. No need to take it personally.

You may be right - you probably have more experience with this particular thing than I do.

I can't answer for the performance of the XB1, but I am curious what % reduction in GPU memory bandwidth you observed due to virtualization.

Did you have a non-virtualized environment on the same hardware to use for comparison?

[p1necone]

Lower graphical fidelity than what? PlayStation?

[dijit]

> Marginally superior to what? PlayStation?

Precisely

Both the original Xbox One and the Xbox One S have a custom, 1.75GHz AMD 8-core CPU, while the Xbox One X bumps that up to a 2.3GHz 8-core chip. The base PS4 CPU remained clocked at 1.6GHz and contains a similar custom AMD 8-core CPU with x86-based architecture, while the PS4 Pro bumps that clock speed up to 2.13GHz.

EDIT: you’ve edited your comment, but also yes.

[p1necone]

The CPU isn't particularly relevant is it (although the CPUs in the PS4/XBone generation were exceptionally terrible compared to what was standard on PCs at the time)? Graphical fidelity is going to depend much more on the GPU (although the CPU is going to bottleneck framerate if it's not powerful enough).

In the current generation the Series X has a more powerful GPU than the PS5, which tends to mean a lot of games run at higher resolutions on the system, although there's some games that run slightly better on PS5 (I think the Call of Duty games might be in that category?). And a lot (most?) are basically the same across both systems - probably because devs aren't bothering to tweak cross platform games separately for the two different consoles.