Hacker News new | ask | show | jobs
by ponker 2069 days ago
Imagine if Apple or Google shipped a digital SLR with all of their software magic onboard (or executed on a paired phone).
6 comments

I think there was a thread here on the disadvantages of all the software processing in phone cameras today. I had a similar experience trying to capture the orange skies when the wildfires were raging here in California. My iPhone corrected every single image to remove the orange and replace it with a dark gray sky. It looked more overcast than apocalyptic. But I'm not fully against the idea of shipping some of those algorithms in a DSLR, especially the Pixel's low light photography.
What you're describing is the problem with auto white balance, which affects phone cameras and webcams equally. Often it works well, but sometimes (like in your case) it fails disastrously.

Unfortunately the iPhone (and Mac's built-in FaceTime) cameras have no option to manually set the white balance. You need to use third-party apps/webcams to handle it.

> Unfortunately the iPhone (and Mac's built-in FaceTime) cameras have no option to manually set the white balance.

I think the new iPhones ship with native ProRAW support, which might mean that the built-in Photos app would be able to access the same amount of imaging data that a specialised RAW editor could in prior versions.

I would be surprised if any other auto white balance camera (basically any digital camera on the market) didn't do the same in that situation.
Apple ProRaw should allow best of both worlds:

https://www.techradar.com/news/what-is-apple-proraw-the-new-...

Does iPhone not have a pro mode? On my Samsung phone I put it into pro, adjusted the white-balance inside and then pointed my camera outside.
Nope, all you get is exposure controls. Third party apps like Halide fill in the gaps here quite well, but sadly you can't make them your primary camera.
It will be dismissed as a toy by your typical dpreview crowd because “software magic” is the antithesis of specs.
Well, yeah, because the software magic is a pure attempt to make up for the lack of specs. Sensors the size that fit in phones will never be able to create shallow depth of field naturally, so fix it in post. Sensors the size that fit in a phone will never have the same light sensitivity, so fix it in post with AI (even better?!).
I don't really see the point.

Things like low-light photography has always been available on DSLR's, like SLR's long before it, since you've always been able to control shutter speed, aperture, and ISO.

And all the extra software magic can already be done on your laptop on RAW files, where you've got enough space on your screen for all the sliders -- and a large enough screen to see the difference it makes in the details.

These days, pretty much the only people shooting with DSLR's are the people who want/need that fine-grained manual control anyways.

The big issue that's missing are stacked exposure RAW files with gyroscope and accelerometer data.

When that becomes available, then yes the possible processing on a computer will match and exceed what can be done on a phone.

If any camera would be able to do it, I'd bet on the Sony A9iii or A7iv.

Ah, that's a great point.

It does seem like it's high time for there to be some kind of RAW-over-time standard, e.g. a single file with 2 hours of exposure that Photoshop would have built-in tools to manage -- and even the ability to handle matrix transformation movement over time, e.g. to photograph stars over 5 minutes.

I wonder how useful it would be to embed gyroscope/accelerometer data with it though? Everything I imagine having to do with long/stacked exposures involves a tripod.

Motion data actually seems like it would be more useful to embed in regular-exposure RAWs, since it could help build a deconvolution kernel to undo motion blur.

The idea would be that all stills, instead of being just a single still, are actually multiple stills that can be stacked on top of one another in order to reduce noise and reduce motion blur, as well as increase dynamic range.

Of course, this would need a camera with much faster sensor readouts, which should be possible with Sony's next generation of stacked sensors.

I would love that. I'm incredibly curious to see where things are headed, because it feels like this has to be the way forwards.

Of course, storing 100 or 10,000 stills would up storage requirements dramatically -- but then lossless compression should bring that back down to entirely reasonable levels.

Because all you really have to do is compute the "average" image across the entire exposure (or at various "keyframes" if there's significant motion) and then encode the differences from those, which is trivial to compress.

I really hope this is something we see in the next 5 years, as opposed to 20 years.

What about multi focal aspects of smartphone triple/quad cameras? Aren’t they supposed to add some sort of data points?
> These days, pretty much the only people shooting with DSLR's are the people who want/need that fine-grained manual control anyways.

Or people who shoot journalism, sports, conferences or other live events.

I don't care how damn good smartphones are (or will become). DSLRs will forever rule the roost in fast paced environments, smartphones are just too fiddly and there's nothing the manufacturers can do, the hinderance is not software, its form-factor.

But what computational photography aspects that only Apple/Google provide are going to be useful for journalists or sports photographers?

That was the original question, and I'm not seeing it.

Obviously they're using DSLR's for the manual controls and lenses. But my point stands: if they need advanced image processing for something they're going to publish... they need to adjust it on a large screen. Not on-camera.

> And all the extra software magic can already be done on your laptop on RAW files, where you've got enough space on your screen for all the sliders -- and a large enough screen to see the difference it makes in the details.

Not entirely true: DSLRs wouldn't be able to capture depth data like the dual/triple-camera phones can, for "computational photography" magic (or simple masking).

The point is, the sensor size, lens variety, and ergonomics of a full size camera with the astonishingly good computational photography software of Apple or Google. You can’t do it with a standard camera because the software needs sensor data (gyro, accelerometer) and fast control (video frames, changing exposure) of the camera unit.
the software magic "just" takes raw images and make something with it. photographers prefer retaining the raws and do the something themselves on a fast computer.

Now, there's some good use cases for it but I guess the tradeoff hasn't been worth it so far. E.g. having some processing options in camera can speed up your workflow (but then again all the "toys" like this are removed from pro cameras because most people didn't use them). Also, the same magic Apple or Google use doesn't really exist outside of their phones (yet at least, I assume Adobe is working on this).

There's also some obstacles that I know of:

- bigger sensor, slower readout, slower sensor processing

- much bigger raw, needs big expensive CPU, heating problems

- hiring engineers that can do this (it took a long time for Apple to catch up to Google f.e.)

> executed on a paired phone

One constraint around this is the sheer bandwidth required to do any sort of live processing (either for viewfinding purposes or autofocus) — the latency (and even bandwidth) of anything less than a direct raw feed into the processor isn't gonna be good enough.

The ability to acquire a raw image over USB from a real camera would be amazing.
The "real" camera would need a lot of sensors in it, because I think the software works by combining the image data with the motion data of the camera to stabilize it.
How do you mean? You can already shoot tethered on most (all?) high-end cameras.
With raw image, you do not mean setting the DSLR to shoot raw? Because you can get those files obviously.
gphoto is a linux utility that allows you to do just that :)