Searching files by matching filename pattern and concatenating contents of the files

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • joeferns79
    New Member
    • Sep 2008
    • 37

    Searching files by matching filename pattern and concatenating contents of the files

    Hi,
    I wanted to write a Perl script that searches a given folder for all files that have filenames based on the previous day's date.

    eg. if the filenames of the files in the said folder are ....
    server1_Statist ics_0.20090618T 124050
    server1_Statist ics_0.20090619T 080652
    server1_Statist ics_0.20090220T 124502
    server1_Statist ics_0.20090621T 105927
    server1_Statist ics_0.20090621T 105638
    server1_Statist ics_0.20090621T 105611
    server1_Statist ics_0.20090621T 105518
    server1_Statist ics_0.20090621T 105108

    find all files with yesterday's date(20090621) and concatenate their contents into a single file with the filename "server1_Statis tics_20090621"
  • RonB
    Recognized Expert Contributor
    • Jun 2009
    • 589

    #2
    What have you tried?

    What errors and/or warnings are you receiving?

    What portion of the task do you not know how to accomplish?

    Comment

    • joeferns79
      New Member
      • Sep 2008
      • 37

      #3
      I am relatively new to Perl and we have it installed on AIX. I have worked with the same on Linux and when I tried getting the previous day's date itself, I am having trouble...

      Code:
      #!/usr/bin/perl
      
      $today=`date --date='1 day ago' +%Y%m%d`;
      I tried using the following command, but I was having trouble parsing the columns, to get only the date ...

      Code:
      perl -e 'print localtime(time() - 86400) . "\n"'
      Even after the date has been captured, the month is shown in words, whereas I want it to match the filenames which are numerals.
      Last edited by eWish; Jun 23 '09, 04:09 AM. Reason: Fixed code tags

      Comment

      • RonB
        Recognized Expert Contributor
        • Jun 2009
        • 589

        #4
        Use the strftime function from the POSIX module.

        Code:
        use POSIX 'strftime';
        
        my $date = strftime("%Y%m%d", localtime(time - 86400));
        print $date;

        Comment

        • joeferns79
          New Member
          • Sep 2008
          • 37

          #5
          That worked fine, thanks.

          I am now trying to include the output from that into a search for filenames containing yesterday's date.

          Code:
          #!/usr/bin/perl
          $hostname=`hostname`;
          chomp($hostname);
          #$today=`perl -e 'print localtime(time() - 86400) . "\n"'`;
          #chomp($today);
          
          #!/usr/bin/perl
          
          use POSIX 'strftime';
          
          my $date = strftime("%Y%m%d", localtime(time - 86400)). "\n";
          #print $date;
          
          $wldir="/usr/IBM/WebSphere/AppServer/logs/test";
          #$wlarc=$wldir/$date;
          
          #print "Archiving $wldir directory\n";
          do_dir($wldir);
          
          
          
          sub do_dir {
              my $dir = shift;
              opendir(D, $dir);
              my @f = readdir(D);
              closedir(D);
              foreach my $file (@f)
                 {
                  my $filename = $dir . '/' . $file;
                  if ($file eq '.' || $file eq '..')
                    {
                    }
                  elsif (-d $filename)
                    {
                      # depending on your needs you can do subdirs
                      do_dir($filename);
                    }
                  else {
                      # do something with $filename, like ...
                      if ($filename =~ m/$date/)
                        {
                        print "Concatenating $filename\n";
                        `cat $filename >> $date`;
                        }
                       }
                }
            }
          
          exit(0);
          Even though it doesn't throw an error, it doesn't concatenate either.
          Last edited by eWish; Jun 23 '09, 04:10 AM. Reason: Fixed code tags [code][/code]

          Comment

          • joeferns79
            New Member
            • Sep 2008
            • 37

            #6
            I tried the following script as well, to no avail...


            Code:
            #!/usr/bin/perl
            
            use POSIX 'strftime';
            
            my $date = strftime("%Y%m%d", localtime(time - 86400)). "\n";
            #print $date;
            
            $wldir="/usr/IBM/WebSphere/AppServer/logs/test";
            #$wlarc=$wldir/$date;
            
            do_dir($wldir);
            
            
            
            sub do_dir {
                my $dir = shift;
                opendir(D, $dir);
                my @f = readdir(D);
                closedir(D);
                foreach my $file (@f)
                   {
                    my $filename = $dir . '/' . $file;
                    if ($file eq '.' || $file eq '..')
                      {
                      }
                    elsif (-d $filename)
                      {
                        # depending on your needs you can do subdirs
                        do_dir($filename);
                      }
                    else {
                        # do something with $filename, like ...
                        if ($filename =~ m/server1(.*)\.$date(.*)$/)
                          {
                          print "Concatenating $filename\n";
                          `cat $filename >> $date`;
                          }
                         }
                  }
              }
            If I use ...

            Code:
            if ($filename =~ m/server1(.*)$/)
            it works but that takes the contents of all the files in the DIR.
            Last edited by eWish; Jun 23 '09, 04:11 AM. Reason: Fixed code tags [code][/code]

            Comment

            • RonB
              Recognized Expert Contributor
              • Jun 2009
              • 589

              #7
              Why are you appending "\n" to the date? That is the main problem, but there are others, such as "useless use of cat".

              perldoc -f open
              perldoc -f glob

              Comment

              • joeferns79
                New Member
                • Sep 2008
                • 37

                #8
                Thanks, I got it working once I removed the "\n". I was actually testing it out and forgot to remove it earlier. Thanks, again.

                Comment

                • RonB
                  Recognized Expert Contributor
                  • Jun 2009
                  • 589

                  #9
                  You should look at using:

                  File::Find - Traverse a directory tree.


                  or

                  File::Find::Rul e - Alternative interface to File::Find

                  Comment

                  Working...