Fdupes - A Duplicate File Removal Tool
Fdupes is another CLI tool that I use frequently to automatically find and remove duplicate files. It does this by generating and comparing the md5sum of the files within the specified folder. This is the fastest duplicate file checker that I have ever used.
If you are using Ubuntu, you can install fdupes by executing:
sudo apt-get install fdupes -y
fdupes -rd /path/to/directory
-rswitch stands for "recursive" which means that it will loop through all subdirectories. Remove this if you do not want it to do that
-dswitch stands for "delete", which means that for every duplicate it finds, you will get a menu option to choose which of the duplicates you wish to keep, and it will delete the others. If you can't make your mind up, you can type "all" to keep them all.
First published: 6th January 2019