Hacker News new | ask | show | jobs
The Virtual Blender Camera (srlt.ca)
89 points by fn42 1043 days ago
10 comments

See also previous discussion at https://news.ycombinator.com/item?id=36562757

Unbelievably creative and impressive project, it's so much fun to see :) It's just cool that raytracing is accurate enough to make this possible.

I thought the virtual handles to control the aperture and focus were a nice touch
Agreed, it's a fun throwback to skewmorphic design trends.
The video on the website there is absolutely amazing. I can recommend watching it
Phenomenal work

It makes me wonder though: what's missing here that still puts it in the uncanny valley?

Do you need to deliberately deform these stock 3D models with small defects/"entropy"? Or is it simply not enough detail for the base models?

I am foemer VFX freelancer who worked for cinema production and was specialized on "invisible" VFX.

Making things non-uncanny is usually about two things:

1. The amount of work/time you can put into a single thing

2. Your ability to see

The latter is basically the same reason why people are unable to draw things realistically, that they see on a daily basis. Your brain is very good at giving tou just a conceptual picture of a much more complex reality. Artists for whom realism is important have to have the ability to look at things as they hit the retina and not get blinded by that abstraction in the head.

This is the foundation of achieving any kind of realism (and there is more than one realism, you can also take photographs that look uncanny).

In this case I think it is about materiality. This is typically about a lack of imperfections or a lack of taking the way things are manufactured in the physical world into account: The motar in a brick wall isn't made up of perfect uniform lines, concrete is casted, walls painted in certain ways, outside objects are exposed to the weather, but not uniformally so, etc. This is very often about very small things like subtle variation in the albedo/roughness etc.

Well, real cameras photograph the real world, virtual cameras photograph the virtual world, so yeah, hugely more detailed and complex virtual worlds to photograph, but perhaps the author doesn't have access to the level of virtualized reality to get past the point of uncanniness. But perhaps nobody really does.
Let's look at this from a perspective of Shannon's Information Theory. Cinema is a double tranmissive system. First, the world has things & shapes: it is information. It transmit / sends information about itself via light, which bounces off it and scatters or bounces. This travels through first an air/liquid/vacuum medium (distorting in some cases) and then the lens's optical medium. Then it impacts either a shutter (blocking the light) or if the shutter is open a frame of film, which is actually a lot of independent little film grains on a transmissive medium. Ok, we have now received the information, and the shutter closes and advances to the next frame, to repeat another reception.

Film is kind of interesting because the process of getting the information isn't done there. We also have to re-broadcast the film out, but honestly, that part is kind of boring: shine light through the developed film and it attenuates some parts of the light more than others, reproducing the information encoded on developed film quite directly & without loss.

So far, this has all been modelled pretty well by this project. We have fancy lens optics, reproducing the light-capture system of a camera. What's missing / un-canny valley so far is that the virtual world is usually a fairly poor facimile of the real world. The modelling straight up isn't as good. How things animate and move lack a subtlty of complex motion that real bodies in motion carry. There's a host of small issues around how light interacts/bounces off subjects that we don't model well in Blender or most systems: subsurface scattering effects aren't as fancy as they could be, the physical based rendering models aren't complex enough, the air itself as as a medium isn't well modelled. There's a huge combo of things the virtual worlds aren't as good at as the real world, and there's so many behaviors and nuances of things in the real world that virtual worlds usually don't capture as well. This largely defines the uncanny valley.

But, just to throw a little more fuel on the fire: this project also is missing another step in cinema that I skipped above. I don't think this is where the uncanny valley problem is, but I think it's a pretty sizable difference between film and digital cinema. Film has another tranmission process that I didn't describe above!

So, we've shot our movie. Now what? Well, we develop the film. What is developing? Well, we emerse the film in an activation bath to develop the exposed silver-halide crystals better known as film grains. There's information trapped in these crystals, they're at a certain state, and we have a chemical process which sends this information out, through a medium. The medium is the chemical developer, which turns the exposure into developed film grain, which is the received information from this system.

One of the really crazy things to me is that developing film is not at all like reading exposure values off a digital sensor. Because the process happens over time chemically, and the process itself is actively consuming the film developer as it works, which creates little local pockets where there's less developer. The process is non-linear. A heavily exposed scene will consume the developer and reduce further development speed not just for that film grain, but for the area around it.

Again, this isn't the uncanny valley problem. But it's still something missing from digital cinema, from this effort, that makes it substantially different from film cinema. There's projects like Filmulator https://filmulator.org/ that I love and adore which can simulate chemical development of film from RAW images.

I'd love to see Virtual Blender Camera team up with efforts like these, to create a more genuine film-cinema feel, that models more than just the optical capture systems. That way we can accurately model both of the Shannon Information Theory processes of information transmission that make up cinema. But we'll still need much better virtual worlds & much more believable & nuanced virtual actors within these virtual worlds to overcome uncanny valley problems.

There was a similar effort in 2018 using Indigo Renderer (greetz to all who know it!), which can more efficiently render these kinds of situations using bidirectional path tracing and Metropolis-Hastings sampling: https://youtu.be/y8mKtNCq5CI?t=1712
I thought this was going to just be some sorta custom renderer that did some sort of physical simulation. But it was literally modeling a "pinpoint camera" in Blender, then modeling lens shapes....amazing!
I'm not a 3D modeler (aside from CAD), photographer, or artist but that must be one of the most impressive things I've seen lately. Can anyone comment on how CPU and/or GPU intensive this is.
Probably very intense. Anything involving material transparency in blender is a real hog.
Since modern 3D renderers are already designed to render in a photorealistic way using fundamental principles of light transport, what does this technique actually offer that the renderer itself does not offer? Path tracing follows the basic laws of global illumination, after all, and the path tracing algorithm offers simulation of all different kinds of lens types, etc.
Partly this is a fun exploration of creating art in an unusual medium, much like a photographer might experiment with antique lenses, unusual film formats and suchlike.

But also, although a lot of modern 3D renders are highly realistic there are certain levels of realism they don't bother to pursue, for both computational efficiency reasons and the fact a lot of users don't want them.

This chap had to simulate a lot of light rays, because only a small fraction of them ended up on the imaging plane. You can achieve effects like bokeh adequately with much less render time by using a conventional blender camera and a blur proportional to the Z buffer.

Meanwhile things like film grain, the corners of the image being darker, spherical aberration, chromatic aberration, parts of the image out of focus, lens flares etc aren't what everyone wants for their image.

My understanding is that modern renderers render in a "physically accurate" way. However, cameras throughout history has never captured images in a "physically accurate" way but rather through a set of imperfect techniques (lenses, film etc) which produces a certain look. When rendering, it's hard to make things seem realistic as they often look too perfect but something like this gets closer to how a photo would look.
Path tracing isn’t really physically accurate, as it also models how cameras work. Path tracing is considered “photorealistic “. I’ll have to dig into this project more, but all pathtracing algorithms start with the fundamental principles of optics in the lens, and changing the field of view of that lens affects the image and the perspective distortion, and you also have effects like the depth of field, and other aspects we associate with cameras. Depth of field, after all, is not a physically accurate phenomenon but rather based on how cameras work, and yet that is also how 3D renderers interpret the environment.
But those algorithms generally implement only the most basic aspects of a lens.

No one normally implements a model of a cheap lens that has severe chromatic aberration due to poor focusing of different wavelengths in its poorly designed glass stack.

Depth of field also exists in eyeballs too
It’s not quite the same effect as a camera. Eyes also do things that cameras do not.
This is an art project. A breathtakingly technical art project.

Did you expect this to be practical? It's a fantastic hack.

Very cool, now I just need to learn blender.
I can highly recommend two channels on YouTube above all others for learning Blender. Seek out videos from Blender Guru, and CGCookie. Both have some great beginner playlists to get you the basic skills and then when you're ready they also have advanced techniques and tips / tricks for more "expert" projects.
In theory you should be able to just put a pinhole in front of this virtual camera, to create nested virtual cameras.
Would it be feasible to create a videogame that uses these techniques or would it be too slow to render?