Find duplicate documents in a folder.

What it does

Duplicate Finder scans a folder and identifies documents you have more than once. It works in two modes:

Mode 1 — Exact duplicates (free). Uses file hashing to find byte-for-byte identical files, even when the filenames differ. No API key needed, no cost, works on any file type.

Mode 2 — Exact + near-duplicates (AI). Adds Claude AI to catch documents that are almost the same — the signed version versus the draft, the letter with a changed date, the memo saved twice with small edits. Requires an Anthropic API key.

Either way, you get an Excel report where duplicate groups are highlighted — red for exact duplicates, yellow for near-duplicates — so you can decide what to cull from the review set.

Setup

1. Install Python 3 (once)

   Download it free from python.org/downloads and run the installer. On Windows, check the box that says "Add Python to PATH" during installation.

2. Install the libraries for the mode you'll use

   For exact duplicates only (free), one library is enough:

   Mac

   Copy/usr/local/bin/pip3 install openpyxl

   Windows

   Copypip install openpyxl

   For AI near-duplicate mode, install all three:

   Mac

   Copy/usr/local/bin/pip3 install anthropic pypdf openpyxl

   Windows

   Copypip install anthropic pypdf openpyxl

3. Download the script

   Use the download button in the sidebar. Save duplicate_finder.py anywhere convenient.

Running the tool

1. Start the script

   In Terminal or Command Prompt, navigate to the folder holding the script (type cd, a space, then drag the folder into the window and press Enter). Then run:

   Mac

   Copypython3 duplicate_finder.py

   Windows

   Copypython duplicate_finder.py

2. Choose your mode

   Mode 1  ← exact duplicates only — free, no API key
   Mode 2  ← exact + AI near-duplicates — needs an API key

   If you pick Mode 2, the script asks for your Anthropic API key (from console.anthropic.com). It's used for this run only and never stored.

3. Point it at the folder

   Drag the folder you want scanned into the window to paste its path, and press Enter. The script hashes every file, and in Mode 2 also compares document contents for near-matches.

4. Review the Excel report

   Duplicate groups are highlighted — red rows are exact copies, yellow rows are near-duplicates worth a side-by-side look. Decide what stays in the set.

Common errors