[zbobet2012]

As a streaming industry software engineer: File based streaming is done not for DRM but because of distribution cost and user experience. A segmented ABR video delivery massively decreases CDN costs (which is why Youtube, which is drm free does it). Video startup times, seek times, thumbnail scrubbing, fast forward, clip previews, ad insertion and many other things don't work in a file based experience.

In addition subtitles, secondary audio, descriptive video, and multi-view video etc. are all things which we mandate by law which do not work well in a file base expierenced.

Peer to peer as a distribution method is not only know, but there are plenty of providers that use peer to peer like streaming setups (see https://streamroot.io/streamroot-dna/). You may be using something like BitTorrent and _not even know it_.

Distribution is 1/10th the story and DRM is only a small part of it as well. BitTorrent does nothing to solve the other 9/10ths and removing DRM doesn't either.

[mdorazio]

> Video startup times, seek times, thumbnail scrubbing, fast forward, clip previews, ad insertion and many other things don't work in a file based experience.

Not a single one of these things is true. In fact, file-based delivery offers a superior experience in several of them with proper implementation. Streaming is popular because it lowers distribution costs, decreases piracy, and allows rights holders to pull content whenever they want.

Personally, I hate streaming. Buffering sucks. Bitrates suck. Audio quality sucks. Never knowing how long something I like will actually be available sucks. It's just an all-around bad experience if you care about your media.

[zbobet2012]

Video startup times are kept low in modern clients by starting at low bitrates and then seamless adaptation up. This is very important for advertising companies, social media, live streams, etc. Every second of startup time cost massive user loss in live streaming (this is well researched and documented). Additionally streaming protocols (HLS/DASH) force files to be encoded such that a full download is not required for a decode to happen. This is not a requirement (and pretty rare unless you know what you are doing with an encoder) for file based workflows.

Fast forward for low power clients relies on an "iframe only" track as there decoders can't do many x realtime decode. This is not present in modern file based work flows.

Seeks use the same startup/segmentation requirements as video startup.

Thumbnail scrubbing requires a thumbnail track and is not supported by file based workflows at all.

Clip previews on sites load at low resolution/bitrates and can be poped up to "full resolution" via ABR. Doing a preview of 10x 4k streams without abr wouldn't work even on a modern gaming rig.

I'd love to hear how you'd do SSAI on top of a file based workflow.

You could of course build a "file based" implementation that had all of these features, but you would just be rebuilding DASH or HLS.

[michaelt]

  I'd love to hear how you'd do SSAI
  on top of a file based workflow.

Given that this is a discussion of Netflix and Bittorrent, and the popularity of ad blockers among HN readers, I think the answer to "How would you do server-side ad insertion" is "I would accept losing that feature"

[ryantriangles]

Popular video file players have supported generating iframe tracks for fast forwarding and thumbnail scrubbing for many years now, is there something about the way eg MPCBE and PotPlayer operate that is somehow unreasonable?

[zbobet2012]

It's terrible performance on a power limited devices. It either burns up your battery or is unreasonably slow. Particularly when the source stream is high resolution or complexity.

[Rychard]

Sure, but if you have the file, you could generate these ahead of time on a more powerful machine.

At any rate, I don't think users would balk at being given a choice.

[afiori]

> You could of course build a "file based" implementation that had all of these features, but you would just be rebuilding DASH or HLS.

[auggierose]

Yeah, but would they pay for it?

[baybal2]

HLS and DASH, or any other protocol along their lines (like what youtube used long ago for own HTTP streaming) are still completely inferior to proper streaming protocol on latency and such. I'd say plain "seeking over http" with range requests does often work with lower latency than best HLS/DASH servers on SSDs.

A proper udp based streaming of course tears HLS to pieces

[londons_explore]

HLS was just a stopgap so providers could use cheap http cdn's.

It worked well enough nobody can be bothered with real streaming now...

[spronkey]

SSAI - you write your own player and do it that way?

[zbobet2012]

Also most streaming services offer an offline playback option today.

[zeptomu]

> In addition subtitles, secondary audio, descriptive video, and multi-view video etc. are all things which we mandate by law which do not work well in a file base expierenced.

That's just not true - VLC has many more features than all the web-based (or "app"-based) popular streaming players. Granted, I checked and my VLC does not have "thumbnail scrubbing", but although that is nice, I don't think this is a big deal and if it's really popular, it will be added to VLC (or it may already be there, I did not check).

[JohnBooty]

I'm a big fan of torrents and would prefer them to streams! I'm a bit of a data hoarder. But there are clear reasons why Netflix et al don't simply have you downloading massive files. It would definitely be simpler, but that doesn't mean it's an adequate engineering choice. Lot of smart people working there and they don't jump through all those extra hoops just for fun.

1. You can't play a torrented file until the download is 100% complete. (Well, maybe 95% complete, depending on how tolerant you and your software are when it comes to malformed files)

That's the way it was designed to work. Otherwise, the bits at the beginning of the file would be much more common amongst your non-seeding peers and the bits at the end would be much rarer amongst your non-seeding peers. Your downloads would start crazy fast and would get progressively slower.

Some clients let you abuse the protocol and "stream" torrents sequentially, but if significant percentages of torrenters (such as literally every Netflix user, in an imaginary world where Netflix simply served everything via torrents) did that it would be an issue.

That's a great way to deliver big files in their entirety via P2P, but that is pretty much the pathological opposite of what is requred for streaming.

For streaming you obviously need that sequential access. You need to optimize for the beginning of the file, since people want to start watching right away. People do not want to wait 60 or 30 or even 10 seconds for a YouTube vid to start streaming.

2. Think about your actual viewing habits. How many times have you watched the first 1% of a video on YouTube or Netflix, and then decided to watch something else instead? Even if we (probably generously) assume most users watch an average of 50% of a given video (I suspect it's much less) that's still 2x wasted bandwidth per video, on average, if video providers blasted out entire files. Netflix's infrastructure (or rather, their AWS monthly bill) is massive already. Imagine it being asked to blast out 2 or 5 or 10x as much data per video as it's already doing.

3. There's also the rather important matter of many (most?) playback devices not having huge gobs of local storage with which to hold complete movie downloads. I've got probably 20TB of hard drives scattered around this place, but that's not the norm.

4. Can't adapt to changing bandwidth conditions by scaling video quality up and down midstream. Personally, I wish I had more control over this as a customer, as sometimes I'd rather just wait than watch a compromised stream, but Average Joe is not going to want to muck around with that sort of choice, or even understand what it means.

[butteroverflow]

Of course you can. Every modern BitTorrent client allows you to download file pieces by their order. IIRC it has been a solved problem for at least 8 years.

[JohnBooty]

That's abuse of the protocol. If a small number of people do it, no problem.

If everybody did it, that would be a big problem.

More to the original poster's point, my post answer's the posters question of "why doesn't YouTube just offer file downloads like Bittorrent?"

There are many clear reasons why; my post gave one of the more prominent ones.

[rakoo]

That is just not true, ever since PopcornTime has shown us that getting pieces in order is a viable strategy, especially at relatively large scale (for the BitTorrent network). There hasn't been any significant shortage of availability linked specifically to that way of downloading.

[cyberbanjo]

Why is that abuse of the protocol?

[JohnBooty]

Imagine a file split into 100 blocks.

You're the seeder. Initially you're the only one with all 100. And everybody's gonna download things sequentially.

Imagine 300 peers grab block 1 from you. Gonna be pretty slow, each peer gets 1/300th of your bandwidth.

Imagine those 300 peers finish block 1 and move onto block 2. Same bandwidth crunch. Although new peers can at least get block 1 from that initial cohort of 300 peers. But nobody can blocks 2 through 100 from anybody but you. Not ideal.

Now repeat the process for 3 through 100. You're gonna be the bottleneck for a loooooooong time for those remaining pieces.

...

...

...now imagine we do it differently. Those 300 peers each grab a random block. This part's just as slow.

But once they have their initial randomly-chosen blocks, our bandwidth explodes. Each of those 100 blocks is now available from ~4 sources (you, and roughly 3 others). And you could even log off at this point, since there is a complete (distributed) copy of your file out there.

As more blocks are exchanged, the effective aggregate bandwidth rapidly increases even farther.

[Rychard]

Bittorrent tries to maximize the benefit of its P2P nature by not sending the same blocks to more than one peer.

i.e. I'll send block 1 to one peer, block 2 to another peer. Then I can send blocks 3 and 4 to them (respectively), while they each send each other blocks 1 and 2.

[zzo38computer]

If the protocol does not require it to be downloaded in order (I don't know much about the protocol), then presumably you should also be allowed to disable that feature if you do not want to play back the file immediately.

[paulddraper]

> You can't play a torrented file until the download is 100% complete.

That's just not true.

I've used uTorrent, and I recall deciding whether I wanted to stream or not. It worked well too.

[afiori]

This is addressed as "abusing the protocol"

[paulddraper]

And given that most popular clients allow you to, this because worthy of the "95% theoretical problem" title.

[m-p-3]

How is it abusing the protocol if it allows you to do it?

[zzo38computer]

"You can't play a torrented file until the download is 100% complete." Doesn't it depend on the file format in use? Some file formats are streamable, and in that case if the file is being downloaded in order then presumably it can be played (assuming the torrent software stores the file in a streamable way, and the playback software is capable of working with a file that is not yet complete but will get filled up over time).

(In the case of ZIP archives, 7-Zip can't open partial files, but bsdtar can. Although in my case it was from a damaged floppy disk rather than torrent, but the same thing would do, if you are downloading ZIP files from torrent. But if you are downloading music, then presumably bsdtar is irrelevant.)

[jayjader]

> 1. You can't play a torrented file until the download is 100% complete. (Well, maybe 95% complete, depending on how tolerant you and your software are when it comes to malformed files)

Theoretically, yes. But in practice, especially with "modern"/high-speed internet, I can open my torrent in VLC after the first few MB are present on my computer and read the entire file as it comes in, without me the user encountering stuttering or buffering of any kind.

In effect, from the user's perspective, you're streaming the torrent - without running into the often present decrease in quality from the "traditional" stream's encoding.

[DevoidSimo]

Why do you say subtitles and secondary audio (I assume you mean multiple language tracks) don't work well? I've downloaded files with multiple subtitles and audio tracks, which VLC makes easy to switch between. Admittedly, many/most torrents don't offer these but that doesn't mean they can't.

[developer2]

Subtitles are not a burden; downloading a single torrent with a dozen subtitle tracks adds negligible bytes to the overall download size. Audio tracks are a tad rougher; the good torrents come with multiple tracks, but you are downloading all audio tracks even if you only play a single language–and the size of those audio tracks is not as negligible. Most people with large or unlimited bandwidth simply don't care about "paying the price" for multiple audio tracks; it takes a few extra minutes, at $0 additional financial cost.

The rest is your parent justifying their job/industry as if they're a Godsend–perhaps as compared to cable? It's an indefensible position thus far. The streaming services are all trash with pathetic bitrates; all streams average the same ridiculously low bitrate over the entire stream, without accounting for dark scenes that require orders of magnitude more bandwidth. Every Netflix (and competitors') 1080p streams with dark scenes are unwatchable. It's highway robbery to deliver what looks like 180p frames for a 1080p stream. We're supposed to be at 4K these days, yet they can't even deliver acceptable 1080p. Until the streaming services are willing to stop compressing everything far beyond watchable levels, they don't deserve anyone's business. Netflix, I believe, averages ~3-5 GB for a 90-120 minute movie? That number should be, at least via opt-in to those with the bandwidth for it, 20-40+ GB.

I don't necessarily expect fully uncompressed Blu-ray quality, but the standard should be to deliver something watchable, without banding artifacts. At a minimum that would mean massively variable bitrates, where the highest bitrate of a stream should be allowed to be 10-20x+ its minimum bitrate. Compressing a 60+ GB original file into a download of than 5 GB is flat out unacceptable and unjustifiable.

[zbobet2012]

Netflix and other (similar) streaming services which user per-title encoding technologies deliver a harmonic mean PSNR of greater than 45 and greater than 94 VMAF on their highest rendition, which most well connected users pull. This is visually indistinguishable from a lossless encode.

Complaining about the _size_ or _bitrate_ of encoded file isn't worthwhile. Encoding technology has advanced substantially since blue-ray days and we simply no longer needs the 40mbps bitrate.

Indeed HEVC can offer a mathematically lossless encode at around 90mbps for most content.

[krageon]

I'm sure you're very knowledgeable and correct about a lot of things, but where your argumentation falls down most is user experience.

A concrete example: You say here that what Netflix provides is "visually indistinguishable from a lossless encode". Try actually doing this with an open mind and then make this claim again. It was definitely not the case when I last tried it (very recently).

Argumentation of this nature is sort of endemic in the tech industry: "You think you see or experience this, but I know what you see and experience and you are wrong". It happens in discussions about battery life, visual appeal of different encoding options, basically anything which is at some point subjective and while the people claiming to see a difference could be (factually) wrong in a lot of cases (I often cannot tell), it is very arrogant to sit/stand there and tell them that you know better than them what they see.

[dual_basis]

But what about when it has been tested and proven that people cannot discern the difference? I don't know if this is what was done with the per-title encoding that the parent comment was referring to, but I know it is the situation with audio. So many audiophiles swear they can hear a difference between FLAC and high quality VBR MP3, but it has been repeatedly demonstrated that, when presented with both, humans cannot distinguish them better than chance.

[krageon]

> But what about when it has been tested and proven that people cannot discern the difference?

The way people perceive (and how to measure it) is in fact not a solved problem. There are studies done and we can often say with a high degree of certainty that something is probably not visible/distinguishable, but claiming something like that for everyone is too strong and also misrepresenting the state of our learning.

[developer2]

Huh? Open Netflix and play any recently added 1080p stream that contains dark scenes. Those scenes are literally unwatchable. I don't mean "not perfect". I mean they look like they were encoded with a 256 color palette. Bitrate is what matters–even with current codecs used by streaming services–and they do not provide something watchable.

[zbobet2012]

VLC works for the subset of subtitles you, an English speaking user, uses. VLC subtitle support does not support things like ASA or EBU subtitles which are used in other parts of the world. In addition rendering of those subtitles is very constrained. These can be added or configured of course.

Secondary audio is not only additional languages but things like descriptive video services (narration for the blind). VLC does not generally support having multiple active audio tracks without flagging things on the command line.

[mijamo]

Arabic/chinese/japanse subtitles work perfectly in VLC work perfectly and look much nicer tahn on Netflix for instance.

I have never seen descriptive video services in any streaming service so I very much doubt it is a major issue, and even in that case why wouldn't you be able to mix it with the main track in a separate channel (which is what is done on TV, as TV does not support multiple active tracks either) ?

[zzo38computer]

Even so, being able to mix the premixed descriptive video track with the normal audio track may still be helpful to users who want different mixing levels than what they used (because sometimes the music is too quiet, for example). (I don't like descriptive video so much myself, but some people do use it, and so may wish to alter the setting.)

[afiori]

> I have never seen descriptive video services in any streaming service

netflix has them

[fooker]

What if some particular video doesn't? If I have a file I can download a .srt and have the video player display it.

[imtringued]

How are they constrained? I remember this HN submission: https://news.ycombinator.com/item?id=19150484 Basically the entire video is contained in the subtitles.

[zzo38computer]

Subtitles, secondary audio, etc can work in files if those things are encoded in the file. Something like the television captions could be work and then the playback software can be configured what fonts and colours to use to display them, or to not display them at all, and possibly implement stuff such as caption scrollback that the provider did not put in, even.

Still, there is many reason for having the streaming, although it should be a open protocol and not too complicated. And then, if you write a client software and program it to save the file to disk, it can do that too, possibly during playback, and you can program in other stuff too if you want to do.

I thought of idea of Live Audio Video Protocol (but have not actually written any part of it yet, but have some ideas about it), that you can use Ogg streams and you can initiate with file selection, format/quality negotiation, etc, and then receive the stream but you can also send commands to select a channel or subchannel, pause, seek, request data, rate of keyframes (if the server supports changing this, which it might not), and change other settings.

(Actually, I find a few problems with Ogg, and everything else is too complicated, so I made up something which is slightly more complicated than Ogg but not too much, which is called GLOGG; it is mostly like Ogg, but uses UUIDs to identify codecs, allows identifying how individual streams are related (the relation codes are specific to the codec, and are not defined by the container format; you can also specify if a program that does not understand it should ignore it), and a few other things, but still simple compared to most other container formats.)

And about ad insertion, they do that on television without any problem, so I don't know how that is a problem any differences whether it is a stream or file. (Although sometimes they put stuff within the show itself and damaging the picture, they should not do, but that is only my opinion and is unrelated to the protocols.)

[mises]

> In addition, subtitles, secondary audio, descriptive video, and multi-view video etc. are all things which we mandate by law which do not work well in a file base expierenced.

Since when are subtitles mandated by law? "Multi-view video"? Maybe I'm just missing something, but I've never heard of any law mandating any of the things you listed above (except maybe from a government-funded project, but I'm not sure either, that just seems more likely to require it).

[dylz]

There's been stuff like this: https://www.insidehighered.com/news/2017/03/06/u-california-...

[ngcc_hk]

P2P solves it. Just not the payment side. Only if one can pay.

I do not even mind it is drm as long as I can get a good quality one in 10 minutes. Even if I contribute bandwidth (as long as controller and during my download) and paid.

There is always someone not paid. But it is those who can and willingly do which oddly leave out. Like Mp3 in 1990s.

[TheOtherHobbes]

P2P could have totally transformed the industry if it had worked out a way to pay creators - especially niche creators and beginners.

It would have owned discovery, which would have badly damaged the big studios. And Netflix would probably have never happened.

It would have been a similar story in music.

But P2P was always about people who think a file is a file is a file, and creation costs are an externality, which can be ignored as irrelevant.

So now we have TPB limping along, and new creators are mostly owned by YT - which is legendary for draconian application of copyright in the favour of fellow corporates.

[feanaro]

> Video startup times, seek times, thumbnail scrubbing, fast forward, clip previews, ad insertion and many other things don't work in a file based experience.

One of these things is not like the others.