Retrieving many RSS feeds fast?

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Duke of Hazard

    Retrieving many RSS feeds fast?

    I have a simple scripts using magpie to retrieve upto 200 RSS feeds.
    Unfortunately it takes forever as the script has to sequentially get
    each feeds, wait for the response, then move on. Is it possible to
    retrieve them all at once or do them in parrallel?
  • George Maicovschi

    #2
    Re: Retrieving many RSS feeds fast?

    OK, me again, sorry for not being more specific earlier.

    You should create a script (lets say parse_rss_ext.p hp) that parses
    the RSS and outputs the result.
    Then create another script (lets say parse_rss_core) that makes calls
    to the parse_rss_ext.p hp script providing the RSS to be parsed and
    receiving the response, then doing whatever needs to be done to the
    response.

    The code that emulates multi-threading in PHP is the following:

    $rss_list is an array containing all the RSS feeds;

    [code start]
    $threads=array( );
    foreach ($rss_list as $rss)
    $threads[$rss]=fopen('http://localhost/path/to/file/
    parse_rss_ext.p hp?rss={$rss}', 'r');

    foreach ($threads as $thread_rss=>th read)
    if ($thread)
    {
    while (!feof($thread) )
    $responses[$thread_rss].=fgets($thread );
    fclose($thread) ;
    }
    [code end]

    Hope this helps you. Cheers!

    Comment

    Working...