Need time-out procedure for HTML perl procedure

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • lester kahn
    New Member
    • Aug 2010
    • 1

    Need time-out procedure for HTML perl procedure

    I have the following perl script which works nicely. It takes a list of URLs from a file, looks up each one, and returns the meta-keywords for that domain. Problem is, sometimes it hangs because it goes to a site which does not respond. I'm wondering if there is a method by which if it times out, or gets an error response, or in the alternative, a success response, it will know to continue on to the next line in the file. Here is the current script which works, but, as I said, hangs upon grabbing meta-keywords from a bad site:

    Code:
    #!/usr/bin/perl 
    #print "Content-type: text/html\n\n"; 
    use LWP::Simple; 
    use HTML::HeadParser; 
    open (OUTFILE, '>outfile.txt'); 
    open (MYFILE, 'url3.txt'); 
    foreach $line (<MYFILE>) { 
    chomp($line); 
    $URL = get($line); 
    $Head = HTML::HeadParser->new; 
    $Head->parse("$URL");  
    print OUTFILE $Head->header('X-Meta-Description') . "."; 
    } 
    close(MYFILE); 
    close(OUTFILE); 
    exit;
  • toolic
    Recognized Expert New Member
    • Sep 2009
    • 70

    #2
    I can't see a way to configure the timeout for LWP::Simple, but its POD directs you to LWP::UserAgent, which does seem to allow you to control the timeout.

    Comment

    • RonB
      Recognized Expert Contributor
      • Jun 2009
      • 589

      #3
      See: 'perldoc -f alarm

      Comment

      Working...