What ImageIO gives you free, and what it costs to leave

What I learned

• Find out what a library does for you before you have to leave it. For me, that meant writing the replacement and timing both. Reading the docs wouldn't have told me.
• Relying on a platform library has a price, so write it down. Speed, file formats, running on other systems, and bugs you can't fix yourself. I guessed three of those four wrong.
• Plan for the reference being wrong. A test image I could work out by hand ended up being the better check.
• Check the order of your processing steps. Shrinking the image before cleaning up noise, instead of after, saved 1.1 seconds per photo with a two-line change.

LightPhotos started out Mac-only. When I moved it to other platforms, I expected the hard parts to be the window, the graphics and the file dialogs. They weren't. The hard part was something I'd stopped thinking about years ago: reading an image file.

The one call

ImageIO, Apple's built-in image library, has a function that means "give me this image, already shrunk to about this size". You don't load the full image and resize it yourself. The size is part of the request.

For a JPEG that's handy. For a camera RAW file it makes a huge difference. A RAW file is the raw data from the camera's sensor, not a finished picture, and turning it into full colour takes a lot of work.

Almost every RAW file also carries a small JPEG preview, which the camera writes so its own screen has something to show. If you ask ImageIO for a small version, it finds that preview and skips the sensor processing entirely. On Sony ARW files I measured 11 ms instead of 170 to 210.

I didn't write any of that. I got it from one function argument.

No other platform has that function

Building it yourself means learning that most RAW files are really TIFF files underneath, that TIFF files are organised as a list of labelled entries (tags), and that the preview sits at a known spot in that list. I know all of that now, and honestly I'd rather not have needed to.

Two steps in the wrong order

That 680 ms started out as 1.80 seconds. My code was cleaning up noise on the full-size image from the sensor and only shrinking it afterwards.

Swapping those two steps was a two-line change, and it saved 1.1 seconds per photo.

The other costs

Speed was the first cost. File formats were the second. ImageIO reads HEIC, the format iPhones use, and nobody has written a HEIC reader in pure Rust (the language LightPhotos is written in). My only option would be to hook into a C library, which would mean extra build tools on every platform I was trying to keep simple to build. So HEIC still only works on the Mac, and that's a difference you can see as a user.

The third cost was a library that wouldn't run in a browser. The RAW library checks the system clock to log how long each photo took to load. In the browser (under WebAssembly) there's no clock behind that call, so it crashes. My build now downloads the library, applies a four-line patch and builds from that copy. Rust's build tool, Cargo, can't apply a patch for just the browser build, so every build uses the patched copy.

Making the other platforms look like the Mac

Opening a RAW file on Linux, Windows and in the browser was only half the job. The photo also had to look the same as it does on the Mac, and that turned out to be the hardest part of the whole move.

Bigger photo projects have camera profiles, tuned for each camera model, that describe how that camera sees colour. I don't have those. All I have is the basic colour information the open source RAW library I use (rawler) keeps for each camera. My first version on the other platforms looked flat, and about 1.45 times darker than the same photo on the Mac.

Here's what I had to do by hand to close that gap.

• Use what the camera recorded. I apply the white balance the camera saved when you took the shot, then convert the camera's colours into normal screen colours using the numbers rawler has for that camera.
• Handle the brightest parts the way Apple does. Colours too bright for the screen get pulled back gently instead of being cut off, and the very brightest tones go to pure white, which is what ImageIO does.
• Trim the edges. A sensor has a border of pixels that aren't part of the picture. I wasn't trimming all of them, and those dark edges made the browser version noticeably darker than the Mac.
• Turn the photo the right way up. The RAW library always reported photos as upright, so I read the rotation from the photo's details myself. Until I did, some previews came out sideways.
• Reduce noise. ImageIO quietly cleans up noise when it opens a RAW file. I added a fixed amount of noise reduction to stand in for it.

Brightness was the biggest piece. At first I tuned it by eye, with a brightness and contrast tweak that got close but never looked quite right. So I wrote a small tool that opens the same RAW file twice, once with ImageIO and once with my own code, and compares how bright the two results are. Then I used it to fit a brightness curve. Across 148 Sony files, the tool lined my output up with Apple's at 17 points from black to white, and I kept the middle value at each point.

On the ten files I checked, my version had been 1.34 to 1.51 times darker than the Mac. With the curve, it's within 2% either way.

It still isn't a perfect match, and I want to be honest about that. I only fitted the curve on Sony files, because that's what I shoot with, so photos from other cameras may not match as closely. The curve fixes brightness and contrast, not colour, so a shade can still come out a little different. Fujifilm cameras with X-Trans sensors also get a softer result than they would on the Mac.

Apple's decoder isn't always right

Part of my plan was an accuracy check: open a test file both ways, confirm they match, and treat Apple's result as the correct one.

Apple's decoder couldn't open the file. It can't read Linear DNG files at all.

So I rebuilt the test around an answer I worked out myself. I made an artificial DNG file holding a known mathematical gradient and calculated every pixel by hand. The Rust decoder matched exactly, with zero mismatches across 147,456 samples.

If two decoders agree, that only tells me they agree. When one matches numbers I worked out myself, I know it's right.

Would I use it again?

Yes, and I still do on the Mac. It gives you a lot: decades of support for different file formats, a fast shortcut I'd never have thought to build, and correct results on formats that are mostly reverse-engineered and full of quirks from each camera maker.

Having rebuilt a small piece of it myself, I have a lot more respect for how good ImageIO is. Apple has been doing this for years, across a huge list of cameras, and on the Mac I get all of it from one call. Thank you, Apple.

The price is that it only works on Apple's platforms, and some bugs are out of your hands. When Apple's decoder can't open a Linear DNG, I can't patch it, and the only workaround is shipping a second decoder. I don't control that code. I just live with whatever it does.