[delta_p_delta_x]

I would like to see computational photography applied to raw images from DSLRs and MILCs with APS-C and larger sensors. Perhaps Canon, Nikon, Sony, and Fujifilm could have built-in options in their cameras for ‘social media mode’, with a modicum of noise reduction (honestly unnecessary at ISOs lower than about 1600 for modern cameras), but drastically improved HDR and white balance.

Many of these cameras are able to take bracketed[1] exposures, and the SNR in even just one image from such sensors is immense compared to the tiny sensors in phones. Surely with this much more data to work with, HDR is much nicer and without the edge brightening typically seen in phone HDR images.

[1]: https://www.nikonusa.com/en/learn-and-explore/a/tips-and-tec...

[dusted]

They already are. However, most photographers do not appreciate this type of distortion being applied to their images.

At a glance, my samsung note 22 ultra takes better picture than my nikon d7500. At a glance.

However, as soon as you want to actually DO anything to it, like, view it in any real detail, or on anything but a tiny screen, reality returns.. While the phone is absolutely fantastic for a quick snapshot, it just does not come close to the definition of the older camera with the bigger sensor.

[packetlost]

The workflows for someone using a DSLR/mirrorless are entirely different too. Most people edit their photos on a computer in "post production" (this isn't always true, I myself use Fujifilm's built-in JPEG processing features heavily), whereas phones are typically doing very light edits if anything.

Even if SNR isn't really the issue with the tiny sensors (it is), the minuscule lenses used by phones almost certainly don't come anywhere close to the detail fidelity of a lens 300x the size.

[lambdasquirrel]

Not to mention that those giant honking lenses are where half the fun is.

[psychphysic]

Similarly sometimes I just want to apply whatever ai magic elixir my phone does to the raw image from my DSLR.

Messing around in lightroom etc is just not worth it for hundreds of shots I take per day sometimes.

[throwanem]

If you're making the same or very similar edits to a lot of shots by hand, you don't have to do that. You can create a preset from one shot's edits, then apply it to as many others as you like in a single step.

[lambdasquirrel]

Yeah, Photoshop has actions too. See, photographers have been doing computational photography too, this whole time. It’s just that we prefer to put it together with a general purpose tool, rather than what formula Apple or Samsung has decided would be best.

[throwanem]

Which is the point I failed to touch upon in my smartassed earlier comment about the macro rig. We use most of (maybe all) the same transformations in a tool like Lightroom, but prefer to compose them by hand precisely because the assumptions made by phone camera ISPs are not at all guaranteed to hold over the range of raw images we use our cameras to produce. (They don't even hold over the range of raw images people use phone cameras to produce!)

[matwood]

So true. Not only that, but pulling out my phone and quickly snapping a shot is often better than quickly pulling out my non-phone cameras - especially in challenging lighting conditions.

I absolutely can get better shots with my non-phone cameras, but they take more work.

[prmoustache]

I understand carrying and taking shot with a DSLR is much more involved than pulling out a phone from a pocket, but have you ever used compact cameras with 1" censor like a Sony RX100 or Canon G9x ?

They both fit in your pocket and startup as fast as you can activate the camera on your phone. Out of habit I usually reach the power button inconsciently when grabbing it from my pocket and the lense is usually extending while my arm is moving in position so it is usually immediately ready to snap a picture.

If you are in a setting where you are almost certain to take picture, a micro four third format also works great and depending on the lense are usually small enough if you are the kind of guy who wears small shoulder or sling bags.

[matwood]

Well now we're getting into the old adage, whatever camera I have with me. I always have my phone and many times have my Z5 or d7100. There isn't much space to add another camera.

[steveBK123]

The trick is to not take 100s of shots per day, and failing that, filtered out 70-95% of what you shoot, before editing.

[SAI_Peregrinus]

That somewhat depends on three main things: your photography style, your equipment quality, and your skill. If you shoot landscape photos, you almost certainly don't want to be taking 100s of shots per day and should spend more time and effort on composition. If you shoot wildlife action (e.g. birds in flight, especially fast ones like swallows) you'll be taking hundreds to thousands of shots per minute of action (30 shots/second from a high-end body like a Sony a1 or Cannon R5 means 1800 shots per minute, even mid-range bodies take 10 shots/second). If you've got good enough equipment and skill that most of those shots are usable, then there's going to be a lot of sorting to find the ones worth editing.

[Retric]

I find the best photo before editing is often different than the best photo after editing. Granted that’s probably a skill issue on my part.

[steveBK123]

you can bulk edit with your adjustments taken from one image and apply to a set of images taken in same lighting, etc

[dwighttk]

I don’t use it, so I don’t know, but doesn’t Google Images offer do this to any photo you upload?

[dimatura]

Yep, I forgot the name of the settings but my Sony mirrorless has some modes which are in the same spirit of the typical smartphone camera - sort of like an enhanced Auto mode. Though In my experience, definitely less aggressive than what my pixel does.

Camera manufacturers definitely are at a disadvantage when it comes to AI/ML expertise, infrastructure and data compared to Google or Apple. So I do think these big cos. do a better job on average from a purely technical viewpoint. On the other hand, there's no accounting for taste - stylistically, I do agree with many others that they are often too heavy-handed.

[prox]

This! My close to 10 year old DSLR still outperforms the latest and greatest on any phone for this reason.

[causi]

I recently started collecting a few of the digital cameras I lusted after as a kid. It made me realize there's a huge population of cameras out there that would produce incredible photos if not for small details. Details like not having readily available batteries, or an odd cable or memory card limitation. The biggest one for me personally is CCD degradation in cameras that don't have a built in pixel mapping function, so every photo they produce has the exact same set of bad pixels. Sadly as far as I know there's no way to set up automated photo editing where you can just select all the files and hit "fix defect set for Camera A".

[corysama]

I don’t know what I’m talking about. But, I recall the Unity3D folks talking about a camera calibration process that involved photographing a dozen full frame images of a flat neutral gray surface, averaging all those images to average out the sensor noise and being left with an image containing the per-pixel bias of that particular sensor.

[buildbot]

This is called flat frame calibration, basically all cameras sold today will have that auto applied to your raw data even. I had to implement it here for processing raws from a very old camera: https://blog.maxg.io/reverse-engineering-the-sinar-ia-raw-fi...

[dwringer]

I have used a program called Pixel Fixer which can learn a profile of each of your cameras and directly process out the dead pixels in the RAW files in a batch operation.

It hasn't been updated in about 8 years so it doesn't necessarily support anything too recent, and is not open source, but I had some success with using it.

[causi]

Does it support jpegs? Most of the cameras in question don't support RAW output but the problems are individual stuck pixels that are identical between every shot.

[dwringer]

Unfortunately it looks like the author only listed the possibility of future JPEG support, but it was not implemented.

[causi]

Dang. The only thing I've found that comes close is a program called PixelZap, but it hasn't been updated since 2002 and the functionality to purchase the paid version probably just sends your credit info to Russia.

[mtrycz2]

My old DSLR has a RAW+JPEG setting which will do both, I use it all the time.

[hef19898]

My favorite setting as well, I can edit the best shots from RAW, but can share any shot as a JPEG. And those are usually not too bad anyway, sometimes I just stick with the camera JPEG. Only exception, and rarely enough, continous high speed shooting. Even in JPEG Fine, the buffer never runs full. In RAW it does so after appr. 10-13 shots.

[dusted]

Yea, I also do RAW+JPEG, Some times the JPEG turns out just fine and I won't bother with darktable, other times I use the JPEG as a sort of "what would the camera have done" reference, like, for inspiration for the direction I could develop it.

[buildbot]

There is no reason to do this. If you want to apply the algorithms and Iphone uses, simply take a burst and post process later for HDR or whatever. I remember being in middle school and messing around with Hugin and my Minolta bridge camera…

People value quick shots/edits and don’t care about quality or editing things later don’t mind an iPhone doing all this behind the scene - but it is irreversible. The sort of error in the article would drive myself and other photographers up the wall.

Also, an Iphone has a CPU and ISP that outclass desktops from only a few years ago - camera manufacturers simply don’t have the same compute available.

On the other hand, some brands do provide interesting computational photography in their cameras at the very high end. Panasonic mirrorless full frame cameras have a pixel shift mode for super res/no bayer interpolation, with some ability to fix motion between steps. Phase One has frame averaging and dual exposure in their IQ4 digital backs, for sequential capture into a single frame and super high dynamic range respectively.

[astrange]

There is no consumer software that can do what the iPhone does with its sensors.

iPhone HDR produces an HDR image file. Consumer HDR apps do the opposite - they take an HDR raw and tone map an sRGB JPEG out of it.

The only portable format that really supports HDR images is EXR, so if you're not generating that you're not getting it.

(I don't think there's anything that can do deep fusion either, though obviously you need it a lot less.)

[_aavaa_]

I don't think that's true. Capture One introduced HDR merging in one of their recent releases [0].

It does raw in "raw" out. You get a DNG out which is demosaiced. I have not yet looked to see what it's doing under the hood, and how the result compares to EXR. (But in my experience, it seems to struggle with large exposure differences, even when on a tripod).

[0]: https://support.captureone.com/hc/en-us/articles/44100147302...

[astrange]

I mean, sure, producing another raw is one way to do it, but I wouldn't really call that an image format.

[_aavaa_]

I have no idea why you wouldn’t call it an imagine format.

But either way that’s changing the goal posts. Here is a consumer software doing what that iPhone does.

If that’s not to your liking, there is other software which produces 32bit float tiff images, those will support an even higher dynamic without the pain of EXR.

[buildbot]

You are all over this thread trying to prove that iPhones somehow have a unique setup in the industry that no other camera can match, and people keep telling you how wrong you are. DNG is 100% and image format, and can be “raw” or completely processed depending on the layers internally. Educate yourself before making such blanket statements.

[astrange]

I know what a DNG is. It's not a final delivery format; it's not compressed and I don't think eg gaming content pipelines are going to accept it for ingest either.

(Also, macOS won't display them in HDR, but will display EXR files in HDR. Some trivia for you.)

[sorenjan]

Hugin produces exr files from image stacks.

[buildbot]

Sure, there’s not a photoshop button, but does github count https://github.com/timothybrooks/hdr-plus

It’s just an algorithm.

[illiac786]

> iPhone HDR produces an HDR image file

What file format is this? iPhone produces tone mapped HEIC, yes, but that’s not HDR as you pointed out.

Or do you mean HDR video?

[astrange]

No, the HEIC has a proprietary attachment with the HDR data as well. You can see the iPhone display adapting when you view it in Photos.

[_aavaa_]

Do you have any info on how that attachment is stored/accesssed?

[illiac786]

I think I found it:

https://github.com/m13253/heif-hdrgainmap-decode

[aoeusnth1]

Capture One creates .dng files, which are HDR raw equivalents when combining hdr from bracketed RAWs.

[pizza]

How about JPEG XL?

[SSLy]

>camera manufacturers simply don’t have the same compute available.

what forbids them from buying the competitive SoCs from Qualcomm? Pride?

[scriptproof]

They would have to rewrite entirely their software. There had been some attemps to use these Qualcomm processors in cameras (the Yi M1 for example) and the result was terrible, the autofocus specially.

[astrange]

The much larger sensors on a dedicated camera take longer to read out / have physical shutters / are very power hungry, so doing bursts is just a worse tradeoff on a dedicated camera than on a phone.

Also, the camera market is much smaller than the phone market. Also, most camera companies are Japanese and Japanese companies are generally speaking not as good at software as they could be. (Though this is getting less true.)

[zirgs]

Power consumption is the main issue.

Software bloat is another. Current digital cameras start up and are ready to shoot almost instantly.

But it takes a while for a phone to boot up.

Photographers don't want that.

[joking]

They don't really even have to do that, they could use the iPhone soc, but the pairing between the phone and the cameras it's not optimal.

[buildbot]

Huge amounts of power and data - the power cost of sending 48MP over a PCB vs. over wifi is 12 orders of magnitude. 40pJ Vs . 2.5J !!!!

[_visgean]

i think the battery consumption is also quite bad.

[jlarocco]

I'm not exactly sure what you're asking for here. In my experience (and as seen in the article) the image processing in most digital cameras will already blow an iPhone out of the water.

As far as I know, iPhone and Android aren't doing anything that isn't already done by digital cameras. They ramp up the settings on things like noise reduction and sharpness to balance out their tiny sensors, but it's more or less the same algorithms that the cameras are using.

Good cameras even allow you to tweak the settings and control RAW conversion right on the camera. The author could have botched the noise reduction on his Fujifilm to match the iPhone if he wanted to. [0]

[0] https://www.jmpeltier.com/fujifilm-in-camera-raw-converter/

[xgbi]

I think you’re missing what makes iPhone photos look so good; they do much more than just ramping up the sliders on sharpness.

When you take sunlit panoramas for instance, the iPhone will auto bracket and perform hdr treatment, it’s fantastic : you can see both the ground and the blue sky and the clouds. You can’t do that easily with a dslr, certainly not like a phone is doing right now, which is integrating the last x frames and modulating the digital shutter to capture multiple exposures.

Same when you take night shots, where the iPhone is integrating 1-2 s of sensor frames and compensating for movement with the accelerometer. You can take handheld shots of the sky, which is completely out of question with a standard dslr. Maybe Sony can do that with a very fast lens, in body stabilization and a very high iso, but this is 10k worth of equipment right there.

I’m still liking my dslr « real » bokeh and photos, but some of the innovation in digital photos should spill into dslr/mirrorless.

[morsch]

> You can’t do that easily with a dslr, certainly not like a phone is doing right now, which is integrating the last x frames and modulating the digital shutter to capture multiple exposures.

Standalone cameras have been doing this for at least a decade. The same is true for nighttime exposures. That's not to say that the iPhone doesn't do it better, maybe it does, it certainly can throw more processing power at it.

https://www.imaging-resource.com/PRODS/sony-a3000/sony-a3000...

[matwood]

If you're talking about auto-bracketing (AB), yes. But phones are doing more of an AB+. Are stand alone cameras aligning all the pictures in camera? Are they constantly reading the sensor so when you hit the button it takes the last series of shots, aligns them, and merges them?

I've been using AB for 20ish years, but it has never been as automatic as on a phone. It could certainly be that I haven't bought the right stand alone camera though.

[godsfshrmn]

My mirror less Olympus does this as well. Set number of f stops, exposures etc all in-camera

[CWuestefeld]

Yup. I was recently doing some night sky shooting with my E-M1.3. Not only does it do what you say, but when using its "super exposure" it'll take multiple shots moving its sensor (using its in-body stabilization) by sub-pixel amounts. This produces from a 15Mp sensor, a 50Mp or 80Mp result - and as a side-effect, the stack of images results in weighted averaging of the pixels, thus doing a fantastic job of noise suppression. And yes, it auto-aligns, at least in the 50Mp mode.

[fsloth]

It composes the hdr as well?

[leviathant]

The Canon T2i that I bought in 2010 sure did.

[prmoustache]

Most cameras have an hdr mode nowadays.

[astrange]

It's a lie though; it produces an SDR image.

Cameras have HDR video capability, but not HDR stills except for the very latest models.

[jack1243star]

> As far as I know, iPhone and Android aren't doing anything that isn't already done by digital cameras.

It's the opposite, most digital cameras aren't doing enough compared to smartphones, which have minimum control and look very good at 2M pixels. A proper camera with default settings looks bland in the hands of an amateur (too little saturation) and handles dynamic range poorly (without bracketing or post-processing). Plus it's never going to beat smartphones at connectivity (SNS), or portability.

Most people aren't going to spend time processing RAW files, or try to get in-camera picture control/film simulation etc. to look pleasing like photographers do.

[jlarocco]

I don't buy it.

Which cameras have worse "Auto" settings than a phone? I have two Fujifilms, a Panasonic, and a Nikon, and they all take better photos on "Auto" than my iPhone or any of my friends' Androids.

And who buys a camera in this day and age but doesn't want to play with the settings? Point and shoot users stick with their phones now.

[jsmith99]

It's not about better or worse, it's about trade offs. At small social media sizes pictures will look better from a phone than from an enthusiast camera.

Sharpening is a destructive process and the appropriate amount depends on how and what size you are using the image. It can also be easily applied anytime later. So enthusiast camera apply minimal sharpening and look under sharpened at small sizes, and phone cameras apply heavy sharpening but have artefacts if you zoom into them.

Same with saturation and 'ai' tricks like skin smoothing: users of enthusiast cameras often have different preferences.

[astrange]

> Same with saturation and 'ai' tricks like skin smoothing: users of enthusiast cameras often have different preferences.

Note, phone cameras don't do skin smoothing, people just think they do. (Unless you get a beauty camera app.)

The issue is that almost any processing has the effect of removing noise, which is the same as skin smoothing. If you're taking 2-3 images and merging them, it'll remove the noise even if you do something smarter than averaging the pixels - so you actually have to add steps to measure the noise level and add it back.

The same issue comes up in video compression, where there's dedicated film grain metadata since almost any compression kills the real thing.

[leokennis]

> Which cameras have worse "Auto" settings than a phone? I have two Fujifilms, a Panasonic, and a Nikon, and they all take better photos on "Auto" than my iPhone or any of my friends' Androids.

Better or worse is of course entirely subjective. Maybe your real cameras take more neutral pictures, but people these days might expect less neutral pictures: more saturated, more HDR, more contrast.

The goal of photography is often not to look "real" or "neutral". For example, Ansel Adams took beautiful photos...but very stylized.

> And who buys a camera in this day and age but doesn't want to play with the settings? Point and shoot users stick with their phones now.

This point I tend to agree with. Imagine a device that combines the camera/photos interface of a smartphone (full screen viewfinder, touch controls, camera apps, on-device editing etc.) with the hardware of a camera (larger body allowing for bigger sensors, more lenses etc.). Basically the specs of a DSLR with an interface for the masses. Who would carry this with them? It sounds a bit like the novelty "dumb phones" that exist nowadays: very interesting, but a way too small market.

[rimliu]

Pretty much all of them are worse. Theres is no magic wrt to dynamic range on the DSLR (or mirrorless) so if camera does not HDR processing the result will be worse than smartphone's.

[astrange]

DSLR/mirrorless sensors have several stops more dynamic range than a phone (don't know exact numbers), so that's not necessarily true. The phone trick of taking multiple images doesn't always work if there's eg rapid motion.

[buildbot]

Dead wrong. There are cameras that have 15 stops of native dynamic range.

[stephencanon]

Our Fuji’s defaults are pretty good, but iPhone’s exposure and white balance on auto is light-years ahead of our Nikon gear.

[astrange]

Fujifilm is deservedly known for being very good at color.

https://www.thephoblographer.com/2018/01/04/film-emulsion-re...

Although their digital cameras' emulation modes of their old film stock are kinda lame. They also make those weird non-Bayer-format cameras which I'm not sure are worth the price.

[ghaff]

I know it was the fave of a lot of nature photographers and the like, but I never liked the greens in Velvia in particular. If I were going to use a slow film, I'd use Kodachrome 25--which had about the same working ISO. (That said, at some point, Kodak improved the look of 100 ISO Ektachrome and that's pretty much all I used from then on.)

[cycomanic]

> I'm not exactly sure what you're asking for here. In my experience (and as seen in the article) the image processing in most digital cameras will already blow an iPhone out of the water.

> As far as I know, iPhone and Android aren't doing anything that isn't already done by digital cameras. They ramp up the settings on things like noise reduction and sharpness to balance out their tiny sensors, but it's more or less the same algorithms that the cameras are using.

That is not true AFAIK, phone cameras use things like "AI" enhancement amongst other things. The fake bokeh is one example of this, but there are others, like "AI sharpening" etc. . Also the amount of NR is typically much higher in a phone camera.

You are correct they typically also employ a lot more range compression and contrast enhancement to make pictures "pop" more, but you can often achieve similar things by adjusting camera settings especially in lower grade DSLRs.

[prmoustache]

What they typically do is create a fake ugly bokeh and oversaturaate pictures. That doesn't make better photos in general.

That makes mostly better thumbnails. Perfect for instagram but shitty for everything else.

[rocqua]

Not OP, but the ability to very quickly bracket photos. Say at 30fps for 10 frames, and then use software to process that data like how a smartphone does, but with my own control.

Let me figure out when a scene is static enough to combine multiple shifted pictures for better snr. Ideally, let me enable that feature on my camera. Combine that feature with image stabilization, and let me tweak the conversion from multiple images after the fact.

[wrldos]

Yep. I’ve got an iPhone 13 Pro. Sometimes it absolutely mutates images to the point they are unusable.

Bought a Nikon z50 because this was annoying me. Shooting JPEGs and it does a vastly better job and it will still go in my pocket with the 28mm f/2.8 or the 16-50 kit lens. The iPhone is embarrassingly bad in comparison.

[cultofmetatron]

Its definitely done! and whats better, you can do far more with a proper raw file from a large sensor. the trick is the workflows aren't as simple as having it happen automatically.

in my workflow I use either dx0 photolab which has excellent facilities for really bringing out a single image.

sometimes I wanna go hardcore with a landscape and take multiple shots manuallly and blend them together using software like aurora HDR (example: https://www.flickr.com/photos/193526747@N04/52219385902/ ) That image is 5 stacked images combined together using a bit of computational photography and adjusted for saturation.

if you want somethign taht will get you descent results fast and work with raws. you can also go with luminar https://skylum.com/luminar

an iphone image looks better at first snap but my z5's images blow them out of the water once I give them some love in the edit room.

[mikewarot]

Long ago I did some experiments with SuperResolution[1] which is a technique of aligning images to sub-pixel resolution and stacking them to increase the amount of information with the square root of the number of exposures.

I was using Hugin to align the images from my Nikon DSLR. I found that you can get to at least double the resolution in both dimensions fairly quickly, but you'll never get to "enhance" like in TV shows.

[1] https://en.wikipedia.org/wiki/Super-resolution_imaging

[kuschku]

Modern cameras use the optical image stabilization for pixel shift photography, where you can get up to 9× the normal sensor resolution (by moving the sensor slightly).

[astrange]

Of course, for this to work nothing else about the camera or the scene can move, or else you need a lot of smarts to realign the image and ignore the parts of the image that aren't registering anymore. This is typically pretty difficult for an outdoor photo.

[pkulak]

I've got an Olympus M43 camera, which has an image sensor that's both significantly smaller than full frame, and significantly larger than a phone. I've found that I can do absolutely amazing things to the RAW image later. There are these processes now that will use ML to remove noise, and it blows my mind every time. I can shoot at ISO 6400 now without even thinking about it. I used to cringe when I had to hit 1600.

Olympus (OM Systems?) should build this stuff into their cameras. I used to have a bit of inkling to some day "upgrade" to full frame, but not any more.

[Tarrosion]

I'm also shooting on an Olympus M43, several years old. Any suggestions for software tools you've found helpful?

[pkulak]

Yeah, didn't want to sound like a shill, but Dx0 is so good I run it in a virtual machine on my Linux box. It's "Deep Prime" noise reduction thing is what I was talking about. There are others, even some stand alone, but it's nice having it all in one place, and Deep Prime does seem to be the best.

[irthomasthomas]

Most already offer that in their various automatic modes. We don't like it because whatever can be done computationally on a camera, can be done 10x better on a real computer with more hardware and finer controls.

[NegativeK]

It's going to take ILCs switching to sensors like the Z9, without a physical shutter, doing very high frame rates at full resolution.

I'd be interested in computational photography on ILCs if they allowed tuning it -- with phones, it comes with a bunch of other stylistic choices, and I want control over that stuff.

[haswell]

I’d love to see this on “adventures” cameras like the Olympus OM-D line.

This would also pair well with Fujifilm’s lineup which already includes camera features focused on in-camera processing.

[rimliu]

I think Olympus was a pioneer at least in some aspects in computation photography. At least "live composite" and "focus stacking" were not very common at the time Olympus introduced them.

[account42]

Pretty much all RAW processing includes computational photography - at the very least debayering, but likely more.

[astrange]

Taking the RAW involves it too - autofocus and auto ISO.

And developing the raw involves picking a white balance and several kinds of lens correction.

[throwanem]

> I would like to see computational photography applied to raw images from DSLRs and MILCs with APS-C and larger sensors.

Wash your mouth out with soap! I did not spend five thousand dollars on a D850-based macro rig to have it produce results no better in quality than what I can get from my phone.