How a photo gets on screen

What I learned

• Show something cheap while the real work runs. Showing the camera's own preview first turned a 250 ms wait into a 35 ms one, even though nothing actually got faster.
• Base the next step on what came back. I first checked "is this a RAW file?", which seemed sensible but would have led to a growing list of special cases for each format. Checking "is what I got back big enough?" is one rule that covers every format.
• Send the photo to the graphics card once. Anything that happens 60 times a second, like zooming or dragging a slider, should only change a few numbers and never redo the expensive decode.
• Keep platform differences in one place. The four builds only differ in how they read files. Everything after that is shared, so it can't drift apart between platforms.

Three different jobs

I want LightPhotos to feel instant. You open a folder of holiday photos, the thumbnails fill in, you double-click one and it opens full size, you scroll the wheel and it zooms. None of that should surprise anyone. That's what I'm aiming for.

A photo takes one of three paths through the app, and I treat each one as its own problem.

• Opening. You clicked, and you want the photo large on screen right now.
• Browsing. The grid needs a small version of every photo you can currently see.
• Exporting. You want a JPEG on disk with your edits applied.

When you open a photo, what matters is the gap between your click and something appearing. Browsing means doing one small job a hundred times without slowing down. Exporting is about getting the pixels exactly right, and it can take as long as it needs, because you've already pressed the button and walked away.

Opening a photo

A camera RAW file isn't really an image yet. It's the raw readout from the sensor, and turning it into something you can look at takes real work. That step is called demosaicing, and on my machine doing it for a Sony ARW file at screen size takes about 250 ms.

Almost every RAW file comes with a shortcut, though. Your camera already saved a small JPEG preview inside it so it could show you the shot on its own screen. On that Sony the preview is 1616 pixels on the long side, and pulling it out takes about 35 ms.

So the first pass always does the same thing. It gets the cheapest decode it can near the size I asked for, using the embedded preview if there's one worth using. Then the app checks one thing.

Did that come back as big as the window actually needs?

If it did, the app stops there, and you're already looking at the final result after one decode. That's what happens with every JPEG, because a JPEG has no embedded preview and the first pass just decodes it at the requested size. It also happens with a RAW file in a smaller window. If the loupe (the single-photo view) only needs 1400 pixels, the camera's 1616-pixel preview is already bigger than that, so the expensive decode never runs.

If it came back too small, the app queues a second decode at the full target size and swaps it in when it's done. That's the case I hit most days, because my window asks for around 2560 pixels and the camera's preview is 1616.

I like this rule much better than the one I started with, which checked whether the file was a RAW. Checking the size that came back is one rule that covers every format, including ones I haven't added yet.

The third pass is for zooming. Full-resolution decodes take seconds and hundreds of megabytes of memory, so that one only runs when you zoom in past the point where the screen-sized version still has enough detail.

There's one more detail I'm pleased with. When you're arrowing through a shoot, the app loads the photos on either side of you ahead of time, but only with the cheap first pass. If it also ran the full decode for a photo you haven't looked at yet, that work would compete with the photo you're actually looking at. As soon as you move onto it, it gets the full decode like any other photo.

After that, it all happens on the graphics card

Once a photo is decoded, it's sent to the graphics card (the GPU) once. Zooming and panning never touch the image data again. They just change a handful of numbers that get sent to the card each frame.

The editing sliders work the same way. When you drag Exposure, the new value goes to the GPU, and a shader, which is a small program running on the card, redraws the picture on the fly. Nothing gets decoded again, and the main processor never loops over the pixels. You're looking at the original decode with some math applied on its way to the screen.

Two ways of drawing

The buttons, grid, filmstrip and sliders are drawn by egui, a Rust interface library that redraws the whole interface every frame. The photo itself is drawn by a few hundred lines of hand-written GPU code, underneath a transparent panel.

Splitting it this way lets me write the interface in the loose, convenient style egui encourages, while keeping the pixel-heavy part small enough to tune by hand. With a general-purpose toolkit, the photo would have been a widget, and it's hard to get a widget to take a 24-megapixel image with a custom shader.

Where the four builds meet

The macOS, Linux, Windows and browser builds differ in only one place: how they turn a file into pixels. After that, they all use the same code.

The GPU tone shader is a direct translation of that one edit function. The function that bakes your edits into finished pixels is used in exactly two places: the exporter and the thumbnail you're looking at.

That's why an exported JPEG matches what you saw on screen. I never built that as a separate feature. It comes from there being only one code path, and I'd much rather get that for free than have to keep it true by hand.

If you want to see how this feels, there are builds for all four platforms on the downloads page.