imagedatadupes - find duplicate images
imagedatadupes [option ...] file/directory ...
Options:
-I, --isolate
-d, --delete
-N, --no-prompt
-Q, --quick
-n, --number
-v, --verbose
-h, --help
--default-choice=CHOICE
--exif-ignore=TAG,...
--no-default-exif-ignore
--print-exif-ignore
--skip-mime-checkimagedatadupes compares image files to check for duplicates and optionally deletes them. Duplicates are found even if the files are not identical, as long as the image data is the same, and the metadata is similar enough based on certain criteria.
The images are first compared using signatures of the image data calculated with ImageMagick(1). Then, images with matching signatures are compared for exact equality of the image data. Lastly, the metadata returned by ExifTool(1) is compared (this includes various metadata tags, not just EXIF data). If the metadata in one file is a superset of the metadata in another file, the former is preferred over the latter when deciding which files to keep.
The specific use-case for this is files where the EXIF data was modified to add a comment (e.g. where a photo was taken). If an old copy of the same image still exists, it won't be found as a duplicate by programs such as fdupes(1) and jdupes(1) since the EXIF data is different, but the new version only contains more data than the old one, so there's no point in keeping the old copy. In order for this to work in practice, some metadata tags that ExifTool returns, such as "FileModifyDate", need to be ignored (see --exif-ignore, --no-default-exif-ignore, and --print-exif-ignore).
The output format consists of groups of files separated by empty lines, with each file being preceded by [+] or [-], depending on whether it was automatically selected for keeping or deletion, respectively. If --delete is not used, duplicates are only printed and no actual deletion is performed.
If multiple files are identical (both image data and metadata), files that were included in earlier command-line parameters are preferred for keeping. If multiple files included in the same command-line parameter are identical and --isolate is not used, the file whose absolute path comes first lexicographically is preferred for keeping.
Before being output, the groups are sorted first based on the number of files in each group, then based on the lexicographical order of the absolute path of the first file in each group. This is mainly so that it is easier to spot outliers, such as a group where the file chosen for keeping (i.e. the first file in the group) is not in the expected directory.
See CAVEATS for some issues that can arise.
WARNING: Always make backups of your files before using this program to delete duplicates. There is a nonzero chance that bugs still exist which could cause the wrong files to be deleted. If you have directories containing files that should not be modified in any way, and you only want to check if there are duplicates of those files in other directories, you may want to change the permissions of those files so they are read-only and a malfunction of this program cannot cause any issues. At least that's what a paranoid person such as the author of this program would do.
Isolate the command-line parameters from each other. In other words, for each group of duplicates, all files belonging to the same command-line parameter as the file selected for keeping are also kept.
Prompt user for deletion of duplicates. Note that the actual deletion is not performed until all duplicate groups have been processed and the user has been prompted once more to confirm that all the chosen files should really be deleted. The options are as follows:
Delete the files that were automatically selected for deletion in the current duplicate group.
Delete the files that were automatically selected for deletion in the current duplicate group and in all subsequent duplicate groups.
Do not delete the files that were automatically selected for deletion in the current duplicate group.
Do not delete the files that were automatically selected for deletion in the current duplicate group and in all subsequent duplicate groups.
Go back to the previous duplicate group. This is particularly useful together with --default-choice so that choices can be made very quickly, but mistakes can still be corrected by going back to previous duplicate groups. Previously chosen options are not saved, so when p is used to go back multiple groups, all choices have to be made again.
Note that Y and N also cause the final prompt asking if all chosen files should be deleted to be skipped.
When used in conjunction with --delete, automatically delete duplicates without prompting the user. This option should be used with care as it can lead to data loss. Make sure to always have backups of your data in case something goes wrong.
Only determine duplicates based on the data signature and metadata (skip the check for exact equality of the image data).
Print the number of the current group and the total number of groups for each duplicate group that is output. This is useful in interactive mode so there is an indication of how many groups are still left.
Print informational messages to stdout. This currently includes some file reading errors, as well as information on the metadata comparison (e.g. which tags were present in one file but not in the other one). Additionally, all filenames are printed once more right before they are actually deleted.
Print usage information, then exit.
Set a default choice to use in interactive mode. This must be one of the characters that can be entered at a prompt during interactive mode. When this is set, it is possible to simply press enter at each prompt in order to use the default choice. Note that this is only used during interactive mode, it does not have any effect when --no-prompt is in effect.
Add additional metadata tags to the list of tags that should be ignored when comparing two files. This option can be given multiple times. Alternatively, multiple tags can be given at once as a comma-separated list. The --verbose option might help to figure out why two images aren't being considered as duplicates. Additionally, ExifTool can be used to find the list of all tag names an image contains using the command "exiftool -s image.jpg". It is generally a good idea to start with the default ignore list and only add more tags later if needed. Adding too many tags to the ignore list can easily lead to data loss when files are considered to be duplicates even though the values for some ignored metadata tags were different.
Do not use the default list of ignored metadata tags.
WARNING: As this program has not reached version 1.0 yet, the default list of ignored metadata tags may still change without warning. Always use --exif-ignore together with --no-default-exif-ignore if you want to be certain that nothing unexpected happens.
Print all metadata tags that will be ignored, including the default ones if they haven't been turned off using --no-default-exif-ignore.
Do not check the MIME type of files. Instead, simply process anything that is supported by ImageMagick and ExifTool. By default, only MIME types starting with "image" are processed in order to avoid issues with ImageMagick trying to open large video files or similar.
The ordering of the files within a duplicate group in the output can be a bit weird. In particular, if a file is chosen for keeping because it contains more metadata, it will be displayed at the beginning of the file list, regardless of the regular sorting. This can seem strange when there are multiple duplicates each in multiple command-line parameters since it can happen that the duplicates from one command-line parameter are not written right after each another if one of them was moved to the beginning of the output due to its metadata.
Before printing the output, a final check is performed to make sure that none of the files in a duplicate group have the same path (this should only happen if the same directory/file was given multiple times). This can cause only one file to be displayed for a duplicate group. In a similar vein, when --isolate is used, it is possible for a duplicate group to be printed even though none of the files are considered for deletion.
If there are any errors while reading a file, the file is ignored during further processing. Errors that are only due to a filetype not being supported are normally not printed (except in rare circumstances) in order to avoid spamming the output with error messages for all non-image files. These are only printed when --verbose is used.
Originally, any warnings given by ExifTool also caused the file to be ignored. However, this caused issues because it meant that identical files which both included the same warning were not considered as duplicates. Thus, warnings are now treated almost the same as other metadata. The only difference is that warnings cannot be used to decide that one file contains a superset of another file's metadata. This is a safety feature as it is theoretically possible for two files to be identical, except that one is damaged in some way, leading to a warning. If warnings were treated exactly the same as other metadata, the file without the warning would be deleted as it contains less metadata, but this would probably not be the correct choice. In that case, a warning is printed and the files are not considered as duplicates so they can be examined manually later. If warnings should be disregarded entirely when comparing files, the "Warning" tag can be ignored using --exif-ignore.
If there is an unknown error during the exact comparison of the image data, this might be due to memory restrictions in the ImageMagick configuration. Try opening the ImageMagick configuration file "policy.xml" and changing the maximum memory. The configuration file is probably located at /etc/ImageMagick-6/policy.xml or /etc/ImageMagick-7/policy.xml, depending on the version of ImageMagick. There should be a line similar to '<policy domain="resource" name="memory" value="256MiB"/>' that can be changed to increase the maximum memory. There is also a similar line with "disk" instead of "memory" that can be used to control how much temporary disk space ImageMagick is allowed to use when there is not enough memory. The "map" and "area" options may also be of interest. See the ImageMagick documentation for a description of the other options in the policy.xml file. Note that, depending on how much memory is available, it might be a good idea to set the disk limit to 0 in order to fail more quickly and avoid a lot of disk writes when large files are opened.
ImageMagick seems to sometimes open files without any error even though they don't contain any image data. For instance, it managed to open some JPEG files that had been corrupted and only contained zeroes, calculating the same signature for all of them. This should not be an issue normally since a MIME type check is now performed before trying to read any files with ImageMagick. Another interesting case found in the wild dealt with .exi files, which appeared to contain just the EXIF data from JPEG files. These passed the MIME type check, and ImageMagick calculated the same signature for all of them, but returned an error later on during the exact image data comparison. Such cases should generally not be a problem since the image data or metadata comparison should still fail, causing the files to be treated as different, but it's probably still a good idea to keep these edge cases in mind.
The image data used for the comparisons is whatever ImageMagick decompresses the image files to, which might include various inaccuracies. However, that probably isn't an issue in any realistic scenario.
For simplicity, all comparisons are performed even when --isolate is used and it technically isn't necessary to compare files from the same command-line parameter.
This program completely ignores all symbolic links in order to avoid all the issues that can be caused by them.
Look for duplicates in the given directories, making sure that duplicate images belonging to the same command-line paramater are either all kept or all deleted. The user is prompted to confirm the deletion of each duplicate group, with the default choice being set to y so that it is easy to always press enter (if a mistake is made, p can be entered at the prompt to go back to the previous duplicate group). In order to give an indication of progress during the prompting for deletion, the output groups are numbered.
Same as above, but additionally ignore the "ThumbnailOffset" tag when comparing image metadata.
Always 0, unless the arguments given were invalid.
ExifTool(1), fdupes(1), findimagedupes(1p), ImageMagick(1), jdupes(1)
Copyright (c) 2026 lumidify <nobody@lumidify.org>
Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.