Your filesystem has doppelgängers. Let’s hunt.

---

doppel is a blazing-fast, concurrent CLI tool written in Go for scanning directories and finding duplicate files, aka doppelgängers! 🕵️‍♂️🗂️

Save disk space and keep your filesystem clean by quickly identifying and managing duplicate files. Doppel is designed for speed, flexibility, and reliability.

---

📋 Table of Contents

• 📋 Table of Contents
• ⚡️ Quick Start
• 🔮 Terminal Preview
• ✨ Features
• 📦 Installation
• 🚀 Usage
  • 🛠️ Command-Line Interface
  • ⚙️ Configuration Files
  • Environment Variables
    • Automatic Completion
  • 🔎 Find Command
    • ⚙️ Find Command Options
  • 🧹 Clean Command
• 🧬 How It Works
• 🏗️ Development
• 📜 License
• 🤝 Contributing

⚡️ Quick Start

Install (requires Go 1.27+):

go install github.com/dr8co/doppel@latest

Scan your home directory for duplicates:

doppel find ~

🔮 Terminal Preview

✨ Features

• ⚡️ Fast scanning with parallel hashing (Blake3, configurable workers)
• 🔍 Flexible filtering by file size and glob patterns
• 🔇 Noise reduction with path and file exclusions
• 📊 Detailed statistics and verbose output
• 🛠️ Dry-run mode to preview filters
• 📄 Structured output for easy integration with other tools. Supported formats:
  • JSON
  • JSONL (one duplicate group per line)
  • YAML
  • Text (default)
• 🧪 Tested with unit tests and integration tests
• 💻 Cross-platform: Works on Linux, macOS, and Windows
• 🛠️ Automatic completion for bash, zsh, fish, and PowerShell
• 📜 Structured logging for better automation, debugging, and monitoring. Formats:
  • JSON
  • Text
  • Pretty (default)

📦 Installation

With Go:

go install github.com/dr8co/doppel@latest

From source:

git clone https://github.com/dr8co/doppel.git
cd doppel
go build -o doppel main.go

Pre-built binaries:

See the 🚀 releases page.

🚀 Usage

🛠️ Command-Line Interface

Doppel provides a simple CLI interface. The main command is doppel, with subcommands for different operations.

doppel [global options] [command [command options]]

Run doppel --help to see global options and available commands.

Note

Running doppel with no command defaults to find.

⚙️ Configuration Files

Doppel supports configuration through TOML (recommended), YAML, or JSON files. Configuration files are automatically loaded from:

• $CONFIG_DIR/doppel/config.toml
• $CONFIG_DIR/doppel/config.yaml
• $CONFIG_DIR/doppel/config.json
• $CONFIG_DIR/doppel/config (Assume TOML if no extension)

where $CONFIG_DIR is your system's user configuration directory:

• Linux: ~/.config
• macOS: ~/Library/Application Support
• Windows: %AppData%
• Plan 9: ~/lib
• Other Unix: ~/.config

The configuration files can be used to set default values for any command-line options.

Use the global --config option to load a specific configuration file:

doppel --config /path/to/config.toml find ~/Documents

Pass --config=none or --config=ignore to bypass both configuration files and environment variables and use built-in defaults. Configuration is loaded when a search action runs, so invalid configuration does not prevent help, version, or completion output.

The key names in the configuration file match the long option names for each command, with dashes replaced with underscores. For example, to set the default minimum file size for the find command to 1.5MB, you would add the following to your TOML configuration file:

[find]
min_size = "1.5MB"
ignore_hardlinks = true

For more details on the TOML format, see the TOML spec.

Note

Command-line arguments take precedence over configuration file values.

Environment Variables

Doppel also supports configuration through environment variables. Environment variable names are derived from the command and option names, with the following rules:

• The prefix DOPPEL_ is added to all environment variable names.
• The command name is added after the prefix (if applicable).
• The option name is added after the command name.
• All names are converted to uppercase.
• Dashes (-) in option names are replaced with underscores (_).

For example, to set the default minimum file size for the find command to 1.5MB, you would set the following environment variable:

DOPPEL_FIND_MIN_SIZE=1.5MB

Note

Environment variables take precedence over configuration file values, but are overridden by command-line arguments.

Automatic Completion

Doppel supports automatic completion for various shells. To generate completion scripts, run:

doppel completion <shell>

Where <shell> is one of: bash, zsh, fish, or pwsh.

This will print the completion script to stdout. You can redirect it to a file or source it directly in your shell.

🔎 Find Command

Scan for duplicate files in the current directory:

doppel find
# or simply
doppel

Scan specific directories:

doppel find /path/to/dir1 /path/to/dir2

Scan an explicit list of regular files. In this mode, all directory, filename, glob, and size filters are ignored:

doppel find --files /path/to/file1 /path/to/file2
find . -type f -name '*.go' -print0 | xargs -0 doppel find --files

Every explicit path must exist and resolve to a regular file. Invalid paths cause the command to fail instead of being skipped.

Read an explicit file list from a file or stdin. Paths are newline-delimited by default; use --null for NUL-delimited input such as find -print0 output:

doppel find --files-from paths.txt
printf '%s\n' /path/to/file1 /path/to/file2 | doppel find --files-from=-
find . -type f -print0 | doppel find --files-from=- --null

--files-from implies explicit-file mode, so directory, filename, glob, and size filters are ignored. Every listed path must resolve to a regular file, and invalid paths cause the command to fail. It cannot be combined with --files or positional paths. Empty records are rejected; pass --ignore-empty-paths to skip empty or whitespace-only records. Symlinks are never followed: discovered symlinks are ignored, while symlink paths supplied explicitly cause the command to fail. Hard-linked paths are treated as separate path entries by default. Use --ignore-hardlinks to treat paths sharing the same underlying file as one candidate and avoid reporting them as reclaimable duplicate space.

⚙️ Find Command Options

See the complete find command reference for all options, input-mode constraints, filtering behavior, and output examples.

• -w, --workers <n>: Number of parallel hashing workers (default: number of CPUs)
• -v, --verbose: Enable verbose output
• --files: Treat positional arguments as regular files and ignore all filters
• --files-from <file>: Read explicit file paths from a file, or - for stdin
• --null: Read NUL-delimited paths from --files-from
• --ignore-empty-paths: Ignore empty or whitespace-only paths from --files-from
• --ignore-hardlinks: Treat paths to the same underlying file as one candidate
• --max-depth <n>: Limit directory scanning by containing-directory depth; 0 scans files directly in the root
• --paths-only: Emit every path in duplicate groups instead of a report
• --print0: Terminate --paths-only paths with NUL characters
• --quiet: Suppress progress and informational output
• --fail-on-duplicates: Return a nonzero status when duplicates are found
• --min-size <size>: Minimum file size to consider (default: 0 = no limit)
• --max-size <size>: Maximum file size to consider (default: 0 = no limit)
• --exclude <patterns>: Comma-separated glob patterns for both files and directories
• --exclude-dirs <patterns>: Comma-separated glob patterns for directories to exclude
• --exclude-files <patterns>: Comma-separated glob patterns for files to exclude
• --show-filters: Show active filters and exit
• --output-format <format>: Output format for duplicate groups (default: pretty, options: pretty, json, jsonl, yaml)
• --output-file <file>: Write output to a file instead of stdout

--exclude is mutually exclusive with --exclude-dirs and --exclude-files. Unified patterns match either an entry's basename or its full walked path. The specialized options remain available when --exclude is not used.

The same scan controls are available on clean. An omitted --max-depth is unlimited. Explicit --files and --files-from inputs ignore all scan filters, including exclusions and depth.

The hard-link policy can also be configured with ignore_hardlinks = true under [find] or with DOPPEL_FIND_IGNORE_HARDLINKS=true.

Automation output examples:

# One duplicate-group object per line
doppel find --quiet --output-format=jsonl ~/Downloads

# Every path in duplicate groups, NUL-delimited for another command
doppel find --quiet --paths-only --print0 ~/Downloads | xargs -0 -n1 printf '%s\n'

# Use the exit status to fail a job when duplicates exist
doppel find --quiet --fail-on-duplicates ~/Downloads

--paths-only emits every member of each duplicate group. It does not choose a file to keep or delete, so destructive actions require an explicit retention policy.

For more details, run:

doppel find --help
# or
doppel find help

Examples:

Find duplicates in ~/Downloads and ~/Documents, excluding .git directories and files smaller than 1MB:

doppel find ~/Downloads ~/Documents --exclude-dirs=.git --min-size=1000000 --verbose
# or
doppel find ~/Downloads ~/Documents --exclude-dirs=.git --min-size=1MB --verbose

--min-size and --max-size support the following formats:

• Bytes: 100, 100B, 100b are all equivalent
• Kilobytes: 10KB, 10kB, 10Kb, 10kb, 10000 are all equivalent
• Kibibytes: 10KiB, 10kiB, 10KIB, 10240 are all equivalent
• Megabytes: 1MB, 1mB, 1Mb, 1mb, 1000000 are all equivalent
• Mebibytes: 1MiB, 1miB, 1MIB. (same as 1048576)
• Gigabytes: 1GB, 1gB, 1Gb, 1gb. (1000000000)
• Gibibytes: 1GiB, 1giB, 1gIB. (1073741824)
• Terabytes: 1TB, 1tB, 1Tb, 1tb. (1000000000000)
• Tebibytes: 1TiB, 1tiB, 1TIb. (1099511627776)
• Petabytes: 1PB, 1pB, 1Pb, 1pb. (1000000000000000)
• Pebibytes: 1PiB, 1piB, 1PIB. (1125899906842624)
• Exabytes: 1EB, 1eB, 1Eb, 1eb. (1000000000000000000)
• Exbibytes: 1EiB, 1eiB, 1EIB. (1152921504606846976)

Find duplicates in /var/logs, excluding all .log files and directories starting with temp, and ignoring empty files:

doppel find /var/logs --min-size=1 --exclude-files="*.log" --exclude-dirs="temp*" # Be sure to quote patterns!

Note

When using glob patterns, be sure to quote them to prevent shell expansion.

🧹 Clean Command

See the complete clean command reference for retention policies, action modes, scan options, confirmation behavior, and incompatible options.

clean performs duplicate actions separately from find. It always requires an explicit retention policy and never assumes which file should be kept:

# Preview which files would be deleted
doppel clean ~/Downloads --keep shortest-path --dry-run

# Delete duplicates after explicitly bypassing confirmation
doppel clean ~/Downloads --keep newest --mode delete --yes

# Scan an explicit list of files
doppel clean --files --keep first --dry-run file-a file-b

The available policies are newest, oldest, shortest-path, and first. Ties are resolved by lexical path order. Available modes are delete, trash, and replace-with-hardlink. Without --dry-run or --yes, an interactive confirmation is required; non-interactive runs must pass --yes.

--files and --files-from use the same explicit-file behavior as find. Clean validates every action target with Lstat and rejects symlinks. Pass --ignore-hardlinks when scanning if paths sharing one underlying file should not be reported as duplicates. On Linux, trash uses the XDG user trash directory; unsupported platforms report an error rather than falling back to deletion.

For more details, run:

doppel clean --help

🧬 How It Works

1. File Discovery: Recursively scans specified directories (and their subdirectories), applying filters.
2. Grouping: Groups files by size to quickly eliminate non-duplicates.
3. Hashing: Computes Blake3 hashes for files with matching sizes.
4. Reporting: Displays groups of duplicate files and optional statistics.

🏗️ Development

• 📁 Code is organized in cmd/, internal/, and assets/ directories.

• 🧩 Uses urfave/cli/v3 for CLI parsing.

• 🔑 Uses blake3 for fast hashing.

• 🧪 Run tests with:

  go test -race -v ./...

📜 License

This project is licensed under the MIT License. See LICENSE for details.

🤝 Contributing

Contributions, issues, and feature requests are welcome! Please open an issue or pull request on GitHub.

See CONTRIBUTING.md for guidelines.

---

doppel — Find your duplicate files, fast and reliably. ✨