Finding Instagram reels in years of video, fully local on a Mac

I have a few years of swimming video in Google Photos and I want to cut Instagram reels from it. Watching it all is not an option. So today I built reels-finder, a Claude Code skill that indexes the archive and lets me ask for moments like "open water, people talking, 2026".

Everything runs on the Mac. No cloud API, no upload.

The pipeline

The skill is one Python script with a SQLite database and a few commands:

• reels ingest unzips the Takeout, restores dates and GPS from the JSON sidecars, skips duplicates by a partial hash and files the videos by year.
• reels scan finds scene cuts with ffmpeg and saves three frames per scene.
• reels transcribe runs MLX Whisper (large-v3-turbo) on the audio.
• reels describe sends the three frames to a vision model in LM Studio and gets back JSON: description, tags, setting, activity and a reel score from 1 to 5.
• reels search queries a full text index and writes an HTML gallery. I tick clips there, copy the IDs, and reels export builds a 9:16 FCPXML timeline for DaVinci Resolve.

Getting the videos out was the first decision. As far as I know, the Google Photos API now only lets an app read media it uploaded itself, so Takeout is the practical way in.

The first test

I made an album with 33 iPhone videos (2.4 GB) and downloaded it by hand. On my M1 Pro with 16 GB of RAM, scan found 52 scenes and transcribe took about three and a half minutes for all of them.

The real speech came out well: a coach talking about a relaxed arm in front, two of us on the beach planning a swim to a rock 600 metres away. But the list also had lines nobody said:

Thank you.
Gracias.
Dzięki za oglądanie!
Субтитры создавал DimaTorzok
Продолжение следует...

This is a known Whisper habit. It was trained on a lot of subtitled video, so on silence or water noise it writes what subtitles often end with. In a search index this is poison, because every quiet clip now "contains" the word thank.

The fix

Two simple rules removed all of it. Keep only the languages I actually speak in my videos, and drop a short list of typical endings:

def clean_segments(res, cfg):
    if res.get("language") not in cfg["languages"]:  # ["pl", "en"]
        return []
    return [
        s for s in res.get("segments", [])
        if s.get("no_speech_prob", 0) < 0.6
        and s.get("avg_logprob", 0) > -1.0
        and not WHISPER_JUNK.search(s["text"])
    ]

The language rule did most of the work. The Russian and Spanish lines were all detected as Russian or Spanish, which is wrong for every clip I own.

Takeout has its own traps

When the real Takeout arrived, it came as two zip parts with 31 videos and their JSON files. Two things came up:

• A video can sit in one part and its JSON in the other. Matching only inside one zip loses the date and the GPS. The skill now indexes sidecars by album folder across all parts.
• All 31 were already in the library from the manual download. Duplicate detection caught 31 of 31, but the extracted copies stayed in the staging folder and used 2.3 GB. Now they are deleted on the spot.

Dates were good. 26 matched the video metadata to the second, 3 were one second apart, and 2 were off by a few hours. For those two I trust Google, because the time inside the file was probably an edit or export. 15 videos also got GPS, which only comes from the JSON.

What it took

One afternoon, mostly spent on data, not on AI. The scene cuts and the vision model are the easy, fun part. The work was in dates, duplicates and a speech model that is too polite. Next time I would read the transcripts of a small sample on day one, and test with a two-part Takeout from the start instead of a manual download.