[asveikau]

You can't imagine a patch format that avoids this? It sounds like you lack some creativity.

If you need to lookup patches by pre-patched hash of the source file, you could, for example, precompute hashes and store them on the disc.

[lazide]

It’s an info theory thing. But nice try. Why don’t you propose something better? Besides what I already proposed anyway.

The ways to optimize/‘solve it’ all require degrees of rigor in information control and tracking that aren’t realistic given the market conditions and supply chains.

At least unless people stop being okay with paying for $40+ games that download 40G patches anyway. Which would require severe changes in trajectory of bandwidth availability, which isn’t likely.

[asveikau]

No sir, excuse my bluntness but you are full of shit trying to claim that information theory blocks this from being practical. I worked on stuff like this. I worked on the format that Windows setup uses for installation media for example. It has delta patches too. I think the same exact idea used there would work. It has a hash of every file precomputed and only stores what's unique.

[lazide]

HN rated limiting sucks. And client crashes suck more.

You’re not reading my comment, or thinking about the game distribution problem.

MS can get benefit from reading all the files, verifying hashes, etc.

And in a typical OS update scenario, MS can trust that a files contents haven’t been updated since the hash was checked. Which reading from a DVD/BluRAY, scratches are a problem and it isn’t that simple.

Games typically don’t, especially those on a DVD or BluRAY. Because they are slow, and have terrible seek times.

So, like I pointed out - it doesn’t make sense to do the work you need to do in info theory to actually apply a delta patch in these scenarios.

[asveikau]

The file format I was describing is used in the Windows DVD. I'm sure you know more about it than I do.

[lazide]

And as noted, windows has a lot of use cases games don’t. Which is why essentially nobody does what it does.

Also, windows updates take forever - as you’ve probably noticed.

[mafuyu]

I mean, zsync? Performing simple hashes of blocks of data isn’t exactly hard for the console. On the CDN side, just add a caching layer for the resulting chunks and it should sort itself out, since there are only so many variants of the source disc. It won’t get you the best compression ratios, but it’s flexible. We were considering this for firmware updates of an IoT product. It’s not like differential updates are unheard of.

[lazide]

For that sync to work, you have to read the whole disk, do hashes, then compare to hashes for a given other version. Which requires reading the DVD/BluRAY (slow) and comparing to the versions on the server.

And preferably, if you’re just reading hashes off the disk, that none of the actual data on the disk is corrupted or doesn’t match the hashes.

And then, due to the reality of the way games are distributed, downloading 90%+ of the game anyway.

Or just download the full version, which is simpler and likely the same amount of bandwidth, and faster since you’re not having to read/check the slow disks for more than basic ‘is it this game’ checking.

[mafuyu]

I understand the types of considerations that may make console developers not bother, but also it is a bit ridiculous on its face when they are selling physical media. But for patch-heavy games like CoD, maybe it’s a lost battle anyways, archival be damned.

But I will push back a little and say that a zsync-like differential update scheme would still be totally feasible. BD read speeds are going to be in the 100s of megabits per second, and the compute for the hash is free (ie. It’s I/O bound). You can parallelize and start downloading blocks before you’re done hashing every block on the disc. It seems likely to me that you’d still end up better off with this scheme if you have slower than gigabit download speeds (which is true for the vast majority of the US). Zsync is fairly coarse and flexible, and essentially looks like BitTorrent between two peers over HTTP if you squint. If you assume the download speed and network reliability is the bottleneck that outweighs things like disc I/O and compute, it essentially degrades gracefully to just downloading the entire update in the case that there are zero matching blocks.

Edit: I should mention that another key aspect of the setup is that there are a small number of printed disc revisions, and a small number of target download (most people will get the same game files for a given region). This means that a CDN cache will quickly find the hot blocks to serve, even without any precomputation of the diff between source and target.

[lazide]

Most games, near as I can tell, the version on the DVD/BluRAY for the initial release is pretty much never finished - often barely playable. Even at 'release' date, the initial update is often at least as large or larger than the data on the DVD/BluRAY.

So possible? Sure, almost anything is possible with enough work and tradeoffs. It just isn't economical or likely actually faster given current bandwidth constraints and how they're distributed.

Especially if you consider that if there is network play, they'll have to be up to date anyway or most games won't let them connect, so 'offline' play is going to be a relatively rare situation. So why optimize for it?