How can I clean up my logfile directory using Perl or shell scripting?

Question

I've got an Ubuntu Linux server that's got logrotate running on it nightly. That's great, but my /var/log directory is looking a bit messy...I've got the log files, and then logfile1.1.gz, logfile1.2.gz, etc, for all my different files. I want to make archive directories that will hold the .gz files (i.e., all mail archived logfiles (mail.1.gz, mail.2.gz, etc) go in in the mail directory; same for all different log files that rotate)

What I want to do is get a perl script in place (a shell script would work too, but I like perl better) that will do the following:

1.) ls -l /var/log
2.) check for lines in that output that have .gz all the way at the end
3.) of the lines that have .gz at the end, strip everything but the beginning of the file name, up to the first "."
4.) Take that word, and check to see if a directory by that name already exists.
4a.) if it does, then take all the log files in /var/log that begin with that name, and end in .gz, and move them into that dir.
4b.) if the dir doesn't exist, then make the dir, and follow 4a.

Answer 1

This should do everything you have requested:



#!/usr/bin/perl



# 1. Get the base file name of all files in the log directory that end in .gz

# 2. Create a sub-directory according to the base name if it doesn't already exist

# 3. Move all files with that base name to that sub-directory.

# -> Assumption: That you don't want to overwrite any existing files in the sub-directory that

# have the same name as the file you are moving. If you do, then set the $no_overwrite variable to 0

# and it will overwrite any files in the sub-directory with the same file name.



use strict;

use warnings;

use File::Basename; # Standard Perl Module



my $debug = 1; # Set to 0 to suppress console messages. Keep as 1 until you are happy the script works.

my $no_overwrite = 1; # Set to 0 if you want to overwrite existing .gz files in the sub-directory if they already exist

my $log_path = '/var/log';



my @log_files = glob "$log_path/*";



# Cycle through each file in the log directory and process them accordingly

foreach my $log_file (@log_files) {

next unless ($log_file =~ /\.gz$/i); # Process only files ending in .gz (regardless of case)

my ($file_name,$directory,$suffix) = fileparse($log_file, qr/\.gz/i);

my $base_file = $file_name;

$base_file =~ s/^(.*?)\..*$/$1/; # Extract the base filename up to the first dot if it has a dot

my $sub_dir = "$directory$base_file";

unless (-e $sub_dir) { # Create the sub-directory if it doesn't already exist

mkdir ($sub_dir) or die "Unable to create sub-directory: $sub_dir ($!)\n";

print "Created sub-directory: $sub_dir\n" if ($debug);

}

my $new_file = "$sub_dir/$file_name$suffix";

# Now move the file to the sub-directory. If you don't want to overwrite existing files in the

# subdirectory (that is, $no_overwrite is a true value) then the file name will need to be

# modified so that it becomes unique.

if ((-e "$new_file") and ($no_overwrite)) {

print "File $new_file already exists in sub-directory, modifying the file name before moving it as we don't want to overwrite the existing file.\n" if ($debug);

my $count = 2;

my $modified_file_name = "$file_name-$count";

my $modified_new_file = "$sub_dir/$modified_file_name$suffix";

until (! -e "$sub_dir/$modified_file_name$suffix") {

$count++; # Keep incrementing until we arrive at a unique file name

$modified_file_name = "$file_name-$count";

$modified_new_file = "$sub_dir/$modified_file_name$suffix";

}

print "Moving $log_file to $modified_new_file\n" if ($debug);

rename ($log_file, $modified_new_file) or die "Unable to move the file $modified_new_file ($!)\n";

}

# Move the file if it file doesn't already exist or if we don't mind overwriting existing files

else {

print "Moving $log_file to $new_file\n" if ($debug);

rename ($log_file, $new_file) or die "Unable to move the file $new_file ($!)\n";

}

}

Answer 2

they are the two good ... I wrote a small gadget quite a few years back at artwork. i began out writing all the scripts in bash initially. If i found that the job became too confusing for bash, i could change to perl. via the tip, maximum of my scripts have been written in bash, with something being written in Perl. So ... that's speedier to place in writing scripts in bash if the complexity of the job is low to medium. as quickly as the complexity of the job exceeds mid-point, then you certainly're going to be greater advantageous off using Perl.

Answer 3

use strict;

use warnings;

use File::Copy;

use File::Path;

opendir LOG, '/var/log' or die;

while (my $sItem = readdir LOG)

{

if ($sItem =~ m/\A(.+?)\..*\.gz\z/)

{

my $sNewDir = $1;

mkpath "/var/log/$sNewDir";

move "/var/log/$sItem", "/var/log/$sNewDir";

} # if

} # while