Find and remove duplicate files in Ubuntu / Mint Linux

If you have accumulated thousands of files , some times you will have more than one versions of same file. If you use Ubuntu of Mint linux you can find and remove these duplicates very easily.

There are two popular tools for this fdupes and fslint

Using fdubes

ftubes is the most popular, simple and powerful tool to find duplicates and remove them. It is a command line tool so if you don't like command line move to next heading. It compares file size  and MD5 signatures(Do not worry if you don't know what it is). So it will find duplicates even if they have different names.

Installation

From terminal execute following command 

sudo apt-get install fdupes

ftubes Syntax

fdupes [options] directory

where  options available are

-r --recurse

include files residing in subdirectories

-s --symlinks

follow symlinked directories

-H --hardlinks

normally, when two or more files point to the same disk area they are treated as non-duplicates; this option will change this behavior

-n --noempty

exclude zero-length files from consideration

-f --omitfirst

omit the first file in each set of matches

-1 --sameline

list each set of matches on a single line

-S --size

show size of duplicate files

-q --quiet

hide progress indicator

-d --delete

prompt user for files to preserve, deleting all others 

-v --version

display fdupes version

-h --help

displays help

directory is the path where you want to find duplicates

Examples

Let suppose that we have a music directory in current directory

See a list of duplicates 

fdupes music/

To find duplicates in subdirectories recursively

fdupes -r music/

To see file size in file list

fdupes -r -S music/

To delete files one by one on each duplicate. It will prompt for each set of duplicates

fdupes -r -S -d music/

Thats it. Comment if you need any other help.

fslint

The program that you will have to install is called FsLint (Filesystem Lint). It has both a GUI as well as a command line interface. Aside from finding duplicate files, FsLint is also capable of searching for problematic filenames, temporary files, bad symlinks, empty directories and non-stripped binaries.

Installation

From your terminal execute this command

sudo apt-get install fslint

Done!

Use

Start fslint from menu command line 

It will start UI interface like given below.  You can change the search directory and set required options and you are ready to go. Press Find button.

It will take some time if directory is veyr large. When done it will provide list of duplicates and options to delete, merge or save those duplicate files.

That's it. Help other clean duplicates by sharing this and feel free to comment if you have a specific problem.