Never load the whole folder

What I learned

• Look for loops over everything the user gave you. Loops that handle one request are usually fine. The ones that grow with how many photos someone owns are the ones that bite.
• Size memory by what has to be kept at once. I kept asking how many photos were in a job. The better question was how long each thumbnail actually needs to stay in memory.
• Write the calculation next to the number. Whoever reads a bare 32 later, including me, needs to know what would make it wrong.
• Recheck old numbers when things change. Three of these four bugs were correct when I wrote them. Something changed later and I didn't go back.

Imagine a wedding photographer handing you a card with 8,889 photos on it. You open the folder in LightPhotos, and the app dies.

That was a real bug. I didn't expect to fix the same bug three more times, in places that looked nothing like each other.

The crash

The grid shows a small preview (a thumbnail) for every photo in the folder. My code for filling it came down to this:

for each photo in folder:
    request a thumbnail
    upload it to the GPU as a texture

That loop is fine if you have twenty photos. With 8,889 it asks the graphics card to hold 8,889 images (textures), about 1.3 GB of video memory, which pushes the app's total memory use to around 1.9 GB. Then the operating system kills the app.

The loop itself does what it says. It just assumes a folder only has a few photos in it, and I never noticed I was assuming that.

The fix, the first time

On a large monitor you can see about forty thumbnails at once. The other 8,849 are off screen and nobody needs them yet.

So now the grid only draws the rows you've scrolled to, and asks for thumbnails for those rows plus a few extra above and below so scrolling stays smooth. Nothing else gets loaded or kept around. Web developers who've built long scrolling tables will know this trick; it's usually called list virtualisation. I'd have stopped there, but the same problem kept coming back.

The fix, the second time

Once a thumbnail is loaded, I keep it in a memory cache that holds up to a fixed number of them. I picked that number a long time ago, when thumbnails were small. Later I made thumbnails bigger, a fixed 512 pixels on the long side and about 700 KB each, roughly seven times what each one used to cost.

Nothing crashed or complained. The limit was still a valid number, but it meant something very different from what I had in mind when I wrote it.

So I worked the limit out again from what it should have depended on in the first place: how many thumbnails need to be in memory at the same time. That's the visible grid plus three extra rows above and below, around 180 cells, so the limit is now 256. If I set it lower, the cache throws thumbnails away right after loading them, which is worse than having no cache.

The fix, the third time

If you select a few thousand photos and hit Auto Tone, LightPhotos looks at each one to work out its adjustments. My first version asked for a thumbnail for every selected photo up front, since it would need all of them eventually. With 20,000 selected, at roughly 0.4 MB each, I was asking the cache to hold about 8 GB inside a browser tab that's limited to 4 GB and never gives memory back.

I'd been asking how many photos were in the job. What mattered was how long each thumbnail needed to stay in memory.

I didn't pick 32 by feel. Analysing one photo takes about 8 ms, and it has to run on the app's main thread, so I can't split it across CPU cores. Loading a thumbnail from a RAW file that isn't cached yet takes 137 ms in the slow cases (95 out of 100 loads are faster). To keep the loading ahead of the analysis even in that worst case, the window is the slowest load divided by one analysis.

Before I measured, I assumed loading the thumbnails was what held the batch back. I was wrong. The analysis takes about fifty-five times longer in real time, because it can't run in parallel. If I'd spent my time speeding up the loading, it wouldn't have helped at all.

The fix, the fourth time

When you open a photo large, I load it at a size that fits the window. In principle, resizing the window by one pixel would mean loading it again. Now the requested size rounds up to a multiple of 512 pixels, so dragging a window edge only triggers a handful of reloads instead of one for every frame.

It's the same idea as the other three, applied to something you can change smoothly (window size) instead of a list. If an expensive cache depends on a value like that, I round the value off.