Your data lives where you are allowed to write

What I learned

• Where you keep data decides what's easy. Keeping it next to the photos makes moving, copying and sharing free, and makes searching across folders slow. A central database is the other way round.
• The most limited platform taught me the most. I didn't choose to put the cache next to the photos. The browser's permission rules didn't let me put it anywhere else.
• Design for the strictest platform early. The cache in the system folder worked fine and nobody was complaining, so I'd never have found the better design on my own.
• Write down the downside where someone will run into it. Keying records by filename alone makes late background results risky, so that warning belongs on the code that has to deal with it.

LightPhotos has no database. When you rate a photo four stars, it writes a small JSON file called IMG_4471.jpg.xmp into a hidden .lightphotos folder, right next to IMG_4471.jpg. That's all there is to how it saves your work.

The first version didn't work this way, and I wouldn't have picked this design if you'd asked me up front. I ended up here because I was pushed into it twice.

My first attempt

I started with one catalog for the whole app, which recorded each photo by its full path on disk and lived in the usual folder where Mac apps keep their data. It's the standard approach, and it works fine for a while.

But it breaks in exactly the way people use a photo tool, because people move folders. You import a card into ~/Desktop/import, go through it picking keepers, then drag the folder onto an external drive. As soon as you do that, every path in the catalog points at nothing.

Putting it next to the photos

Now if you move the folder, your ratings and edits move with it. The same goes for copying it to a friend or backing it up. I didn't write any code for that. It just works because your work and your photos are in the same place.

A few details make it pleasant to live with.

• The folder is only created when you save something. Just looking through someone else's photos leaves nothing behind.
• Files are written to a temporary copy, then renamed. On any common disk format a rename happens all at once, so if the app crashes partway through, you still have the previous version instead of a half-written file.
• Your original photo is never touched. This app is for deciding which photos to keep, so it shouldn't change the photos you might keep.

Then the browser made the decision for me

Thumbnails take a while to make, so the app saves them to disk. When LightPhotos only ran on the Mac, that cache lived in ~/Library/Caches, which is where Apple tells you to put it. Then I made the app run in a browser tab.

The system cache folder simply isn't reachable from a browser tab. So I moved the thumbnail cache into .lightphotos, next to the ratings that were already there. I only did it because the browser left me no choice, but it ended up being better on the Mac too.

What I got out of it

Moving a folder now brings its thumbnails along with its ratings. Plug the drive into another computer and the grid fills instantly, instead of spending a minute rebuilding every thumbnail.

The Mac and browser versions also share one cache, which needed one deliberate compromise. The app recognises a cached thumbnail by the photo's file size and the time it was last changed, rounded to the millisecond, because milliseconds are all a browser will tell you. The Mac reports that time down to the nanosecond, so without rounding the two versions would label the same untouched photo differently and never find each other's thumbnails.

There's one cached thumbnail per photo, so a folder's cache can never grow bigger than the number of photos in it, and I don't need rules for throwing old entries out. I just clear out leftovers for deleted photos when a folder opens. If the app can't write to a folder, it makes fresh thumbnails every time and doesn't bother you about it. Losing the cache there is a small slowdown, and I didn't think it was worth an error message.

What it cost me

Everything is saved under the photo's filename alone, because the folder is already known from where the file sits. That makes work that spans folders risky. If you switch folders while a background job is running, a result that arrives late could get saved against a photo with the same name in the new folder.

You can change folders at any time, even with work still running. Here's what happens to each kind of job when you do.

• Exports keep running. Each export already knows the photo's full location and its edits, so it finishes no matter which folder you're looking at.
• Edits you were making get saved first. If you moved a slider and haven't left the photo yet, I save that change before the new folder opens, so you don't lose it.
• Thumbnails keep running, and late results are thrown away. Anything that finishes for the old folder after you've left it gets dropped, so it can't land on a photo with the same name.
• Auto Tone stops. Photos it already finished keep their changes, and the rest are skipped.
• Bulk delete stops. It finishes the photo it's moving to the trash right now, then stops.

Those last two are on my to-do list. It's frustrating to start Auto Tone on a big shoot, or clear out a batch of rejects, and then have to sit and wait before you can look at another folder. I'd like both to keep running the way exports do, by having each job carry the full location of the photo it's working on, so it no longer matters which folder is open when it finishes.

There's also no quick way to ask a question across every folder you've ever opened. "Show me everything I rated five stars this year" would mean going through all of them. A central database could answer that in a millisecond. If that kind of search were central to what LightPhotos does, this would be the wrong design.

The downside that worries me most is renaming. The small file that holds your ratings and edits (a sidecar file) only finds its photo by filename. If you rename IMG_4471.jpg to iceland-sunset.jpg in Finder, the sidecar is still called IMG_4471.jpg.xmp, and LightPhotos no longer knows the two belong together. Your rating and edits for that photo seem to vanish. Renaming files is a normal thing to do, so this is fragile.

I haven't fixed this yet, but I know how I'd like to. Each sidecar would also store a fingerprint of the original photo, a short code worked out from the photo's contents (a hash). When LightPhotos opens a folder and finds sidecars that don't match any photo, it would work out the fingerprints of the photos in that same folder. If one matches, the photo was just renamed, so the app renames the sidecar to match the photo's new name, and your ratings and edits come back.