array results > fputs > responses captured and stored

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • matt@londonstudent.co.uk

    array results > fputs > responses captured and stored

    This is a strange one but I've been stuck on it for days. Any help
    appreciated.


    THE PLAN:
    I've a database that I use a script to grab all the entries for a
    particular field.
    I then want to send each of these entries individually to another
    server using fputs.
    The server will send 1 response (a lin of text) back for each item
    sent to it.
    I'd like to capture these responses and store them in a table.

    WHAT I HAVE:
    I can send 1 item manually at a time and get a response back from the
    remote server. The connections and sockets are all working. I can
    print the list of entries I'm grabbing, ready to send to the server. I
    can print the server response to the screen for a single manually sent
    item.
    So my problem is that I don't know how to send or loop the field
    entries that I grab from my database 1 by 1 automatically and quickly,
    and then capture the remote server responses - inserting them to a
    table.

    MY CODE SO FAR (notated where I've fudged things a little):
    If anyone can help I would really appreciate it. Please reply if you
    have the skills to solve this - it's been wasting my days :(


    <?php

    $fp = fsockopen ('remote-server-address.com', port, $errno, $errstr,
    30);

    if (!$fp)
    {
    echo "$errstr ($errno)<br />\n";
    }
    else
    {

    // get list of items from local database

    $host="localhos t"; // Host name
    $username="loca lusername";
    $password="loca lpassword";
    $db_name="local databasename";
    $db_con = mysql_connect(" $host", "$username" , "$password" )or die
    ("cannot connect");
    mysql_select_db ("$db_name") or die("cannot select DB");


    $query = "SELECT fieldname FROM tablename";
    $result = mysql_query($qu ery);

    while($row = mysql_fetch_arr ay($result, MYSQL_ASSOC))
    {
    $items = "{$row['domain']} <br>";

    // prints a list of the items
    // i don't need the script to do this, but just checking the contents
    of the variable

    echo $items;
    }


    // send and receive to and from remote server
    // I really want to get rid of the first line below and just send
    $item results directly
    // into the $itementervaria ble
    // and have the fputs run quickly sending each of the items to the
    remote server and
    // capture the server responses
    // then eventually store each of the server responses into another
    table.


    $itementer = "testmessag e";

    fputs($fp, "$itementer\r\n ");

    $remote-server-response = fgets($fp, 128);


    // prints the server response
    // i don't need the script to do this, but just checking the contents
    of the variable
    // at the moment I'm specifying $itementer = testmessage so only a
    single
    // response is printed from the line below
    // eventually need the list of many responses to be entered into a
    table on the db

    echo $remote-server-response;

    fclose($fp);
    }
    mysql_close($db _con);

    ?>



  • Jerry Stuckle

    #2
    Re: array results &gt; fputs &gt; responses captured and stored

    matt@londonstud ent.co.uk wrote:
    This is a strange one but I've been stuck on it for days. Any help
    appreciated.
    >
    >
    THE PLAN:
    I've a database that I use a script to grab all the entries for a
    particular field.
    I then want to send each of these entries individually to another
    server using fputs.
    The server will send 1 response (a lin of text) back for each item
    sent to it.
    I'd like to capture these responses and store them in a table.
    >
    WHAT I HAVE:
    I can send 1 item manually at a time and get a response back from the
    remote server. The connections and sockets are all working. I can
    print the list of entries I'm grabbing, ready to send to the server. I
    can print the server response to the screen for a single manually sent
    item.
    So my problem is that I don't know how to send or loop the field
    entries that I grab from my database 1 by 1 automatically and quickly,
    and then capture the remote server responses - inserting them to a
    table.
    >
    MY CODE SO FAR (notated where I've fudged things a little):
    If anyone can help I would really appreciate it. Please reply if you
    have the skills to solve this - it's been wasting my days :(
    >
    >
    <?php
    >
    $fp = fsockopen ('remote-server-address.com', port, $errno, $errstr,
    30);
    >
    if (!$fp)
    {
    echo "$errstr ($errno)<br />\n";
    }
    else
    {
    >
    // get list of items from local database
    >
    $host="localhos t"; // Host name
    $username="loca lusername";
    $password="loca lpassword";
    $db_name="local databasename";
    $db_con = mysql_connect(" $host", "$username" , "$password" )or die
    ("cannot connect");
    mysql_select_db ("$db_name") or die("cannot select DB");
    >
    >
    $query = "SELECT fieldname FROM tablename";
    $result = mysql_query($qu ery);
    >
    while($row = mysql_fetch_arr ay($result, MYSQL_ASSOC))
    {
    $items = "{$row['domain']} <br>";
    >
    // prints a list of the items
    // i don't need the script to do this, but just checking the contents
    of the variable
    >
    echo $items;
    }
    >
    >
    // send and receive to and from remote server
    // I really want to get rid of the first line below and just send
    $item results directly
    // into the $itementervaria ble
    // and have the fputs run quickly sending each of the items to the
    remote server and
    // capture the server responses
    // then eventually store each of the server responses into another
    table.
    >
    >
    $itementer = "testmessag e";
    >
    fputs($fp, "$itementer\r\n ");
    >
    $remote-server-response = fgets($fp, 128);
    >
    >
    // prints the server response
    // i don't need the script to do this, but just checking the contents
    of the variable
    // at the moment I'm specifying $itementer = testmessage so only a
    single
    // response is printed from the line below
    // eventually need the list of many responses to be entered into a
    table on the db
    >
    echo $remote-server-response;
    >
    fclose($fp);
    }
    mysql_close($db _con);
    >
    ?>
    >
    >
    >
    You need to have your fputs/fgets within the while loop so you have
    access to each row.

    Just a word of caution - if you have a lot of data, your script may
    timeout before you get it all processed.

    --
    =============== ===
    Remove the "x" from my email address
    Jerry Stuckle
    JDS Computer Training Corp.
    jstucklex@attgl obal.net
    =============== ===

    Comment

    • matpac4@gmail.com

      #3
      Re: array results &gt; fputs &gt; responses captured and stored

      Thanks Jerry - that is what I thought but I've been getting so many
      errors when I try and put it into the loop - I just can't place it
      correctly.

      Could you show me? Would help a lot

      RE: timeout issues - do you know how I can stop that from happening -
      at some days there may be 500,000 items being sent with responses
      needed back and to be stored

      Thx

      Comment

      • matpac4@gmail.com

        #4
        Re: array results &gt; fputs &gt; responses captured and stored

        Thanks Jerry - You helped me crack half of the problem - I figured out
        the loop error I was making.

        Comment

        • Jerry Stuckle

          #5
          Re: array results &gt; fputs &gt; responses captured and stored

          matpac4@gmail.c om wrote:
          Thanks Jerry - that is what I thought but I've been getting so many
          errors when I try and put it into the loop - I just can't place it
          correctly.
          >
          Could you show me? Would help a lot
          >
          RE: timeout issues - do you know how I can stop that from happening -
          at some days there may be 500,000 items being sent with responses
          needed back and to be stored
          >
          Thx
          Nothing special about it - just something like:

          while($row = mysql_fetch_arr ay($result, MYSQL_ASSOC))
          {
          $items = "{$row['domain']} <br>";
          // Format the data in $row as necessary here
          // and put it in $itementer

          fputs($fp, "$itementer\r\n ");

          $remote_server_ response = fgets($fp, 128);

          // parse the response here and place it in the database
          }

          If you have the proper authority, you can use ini_set to set
          max_execution_t ime to 0 (disabled).

          However, with 500K items, you're going to take a long time. You can run
          into things like database or socket timeouts or all kinds of other
          possibilities. And what do you do if you process 495,000 rows and get a
          failure?

          I think your error detection/recovery code is going to be even bigger
          than your "work" code.

          I know you didn't ask, but personally, I probably wouldn't do this in
          PHP. I'd look at something like C or C++. Faster and lighter load on
          the system.

          --
          =============== ===
          Remove the "x" from my email address
          Jerry Stuckle
          JDS Computer Training Corp.
          jstucklex@attgl obal.net
          =============== ===

          Comment

          • matpac4@gmail.com

            #6
            Re: array results &gt; fputs &gt; responses captured and stored

            Thanks for your help with this Jerry - I've made a lot of progress
            with my project since a few hours ago - feels great when I'm getting
            stuff done compared to fixing some code that should be easy!

            Comment

            Working...