Directories with a large number of files

Once there was an interesting situation, in the same directory there were millions of files.
And some of them are necessary.

When you try to view the list of files, you will naturally get a stupor for a long time.
Alternatively, they can be viewed via FTP, which has 10,000 for the frequent standard limit on the number of displayed files, for example, the FileZilla FTP client conveniently moves files in directories, but this option is long, because time is spent on FTP requests, the load on the drive is low.

If the files are not needed, you can delete them with the command (with the confirmation request to delete)

rm -r /dir/

Or delete everything without a request along with the directory:

rm -rf /dir/

In my case, small files were unnecessary, so going to the right directory, deleted the command below with anything that is smaller than the specified size:

cd /dir/
find -size -2 -type f -print -delete

Before deleting, you can see the number of such files and the total number, but this is also a lengthy process:

find -maxdepth 1 -size -2 -type f -print | wc -l
find -maxdepth 1 -type f -print | wc -l

If, instead of -2, you specify 0, then files with zero size will be deleted, that is, empty.

If you need to sort the files by directories, go to the directory with files, create the necessary directories, for example, by dates and move the files by template (all whose names begin on 2017, -maxdepth 1 indicates that you do not need to search for files in subdirectories):

cd /dir/
mkdir 2017
find -maxdepth 1 -type f -name '2017*' -exec mv -vn -t /dir/2017 {} \+

The result of the execution can be written to the file by adding to the command “> file”, for example:

find -maxdepth 1 -type f -name '2017*' -exec mv -vn -t /dir/2017 {} \+ > /dir/dir/file.log

Leave a comment

Leave a Reply