This code is tested and works and will do what you have asked. It works assuming your logfile is like the sample you have shown. You may need to tweak the code a little if the logfile is a little bit different. When you ask for help parsing a logfile it helps a great deal if you are able to give an exact sample.
Also, it helps a great deal if you give an example of the report you want displayed.
I scrapped this code together in a few minutes so it isn't the most elegant, but it should point you in the right direction. It will show you how to read a file line by line and how to use a hash in order to count elements.
Unfortunately Yahoo Answers does not show tabs/spaces so the following code will look a mess, take the time to indent/format the following if it is of use to you.
(The following is overly - and badly - commented, if it helps just remove the comments to see the code more clearly.)
#!/usr/bin/perl
# This is tested and does what you have requested given the sample
# of the file you have given in your question. If your sample file
# is different you may need to tweak the following code.
# It would be helpful if you provided an example of exactly how you
# wanted the results displayed eg. totals inbetween timestamps or as
# totals for the entire logfile. The following code does both.
# If this code doesn't do exactly what you wanted it should at least
# point you in the right direction.
use strict;
use warnings;
# Change the following to 0 if you only want the results displayed for the
# entire logfile. Otherwise the default action is to display the totals
# as they happen inbetween timestamps.
# If you don't know what I mean simply run the script
# with this variable as 1 and then run the script with the variable as 0
# and note the differences
my $display_results_inbetween_timestamps = 1; # Set to 0 to display logfile totals only.
my $log_file = 'logfile.log'; # Change this to whatever your logfile is called and include the path if the logfile is in a different directory.
# This is how you open a file for read-only access
open (my $fh, '<', $log_file) or die "Unable to open $log_file ($!)\n";
# We'll use two counter hashes. One to count the totals for the entire
# logfile and another to count totals inbetween timestamps.
my %counter_between_timestamps = ();
my %counter_logfile_total = ();
my $previous_timestamp;
print "Results for Items Counted Inbetween Logfile Timestamps:\n" if ($display_results_inbetween_timestamps);
# Cycle through the file line by line and add up each occurrence of
# what you want to have counted.
# This is how you go through an opened file line by line.
while (<$fh>) {
chomp (my $line = $_);
if ($line =~ m/TIMESTAMP\s(.*?)$/) {
my $timestamp = $1;
# We've encountered a new time stamp, so we want to display a report
# showing how often the things we're watching out for has already occurred.
# If this is the very start of the logfile, then it won't display anything
# as each of the items we're counting will be zero.
# The results are sorted alphabetically, if you don't want to sort the results
# then simply remove 'sort {$a cmp $b}' from the next line of code.
if ($display_results_inbetween_timestamps) {
foreach my $item_to_count (sort {$a cmp $b} keys %counter_between_timestamps) {
# NOTE: The next line gets chopped off by yahoo, see the note below:
my $item_freq = $counter_between_timestamps{$item_to_count};
# END_NOTE
print "From $previous_timestamp up til $timestamp: '$item_to_count' occured $item_freq time(s)\n" if ($item_freq);
}
print "-" x 40 . "\n";
$previous_timestamp = $timestamp;
}
# After we have displayed the report we reset the counter hash for inbetween timestamps
# simply destroying its hash.
%counter_between_timestamps = ();
}
else {
# We will count up whatever appears in the "" inverted commas
# that appears directly after OUT:
if ($line =~ m/^.*?OUT:\s"(.*?)"\s/) {
my $item_to_count = $1;
# NOTE: Yahoo has chopped off the following two lines, see the 'source' info below for what they should be
$counter_between_timestamps{$item_to_count}++;
$counter_logfile_total{$item_to_count}++;
# END_NOTE
}
}
}
# Display the total log file counter results...
print "Total Results for Entire Logfile:\n\n";
foreach my $item_to_count (sort {$a cmp $b} keys %counter_logfile_total) {
my $item_freq = $counter_logfile_total{$item_to_count};
print "Item '$item_to_count' occured $item_freq time(s)\n" if ($item_freq);
}
##############
If you have any further problems, let me know. If the code above is too messy without tabs/indentation I'll post it somewhere else for you.