curl extension downloads 1 byte only from zip files

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • benji

    curl extension downloads 1 byte only from zip files

    I have set up a system to download datafeeds in pain text or zipped.
    The download part of this system uses the curl extension to download
    the files.

    All was well when I tested it with various datafeeds in zipped and
    unzipped formats until I came to download zipped files from the
    affiliate program we are starting with.

    I have verified that the datafeeds download properly in Internet
    Explorer but interestingly they also fail in Firefox.

    Plain text files will download correctly from the same server with the
    curl script.

    Using curl to download a file results in a 1 byte file. Downloading
    with Firefox produces a 4kb corrupted zip file. I have tried multiple
    files all with the same result.

    A php script appears to be being used to create the zips on the fly?
    could this be the cause of the problem?

    Any thoughts on this one are much appricated

    -Ben

  • Andy Hassall

    #2
    Re: curl extension downloads 1 byte only from zip files

    On 12 Jul 2005 12:57:35 -0700, "benji" <benpjohnson@gm ail.com> wrote:
    [color=blue]
    >I have set up a system to download datafeeds in pain text or zipped.
    >The download part of this system uses the curl extension to download
    >the files.
    >
    >All was well when I tested it with various datafeeds in zipped and
    >unzipped formats until I came to download zipped files from the
    >affiliate program we are starting with.
    >
    >I have verified that the datafeeds download properly in Internet
    >Explorer but interestingly they also fail in Firefox.
    >
    >Plain text files will download correctly from the same server with the
    >curl script.
    >
    >Using curl to download a file results in a 1 byte file. Downloading
    >with Firefox produces a 4kb corrupted zip file. I have tried multiple
    >files all with the same result.
    >
    >A php script appears to be being used to create the zips on the fly?
    >could this be the cause of the problem?[/color]

    Minimal example code reproducing the problem?

    Example URL?

    First place to look would be the Content-type header.

    --
    Andy Hassall / <andy@andyh.co. uk> / <http://www.andyh.co.uk >
    <http://www.andyhsoftwa re.co.uk/space> Space: disk usage analysis tool

    Comment

    • benji

      #3
      Re: curl extension downloads 1 byte only from zip files

      Thanks for the feedback it has helped my understanding of the problem.

      Here is a cut down version of the code I am using. Sorry about omitting
      the url I don't think the company would be impressed if I posted it in
      a public forum.

      I have also posted the responce headers below. I think that the
      Content-Disposition: attachment; part is the key. I am guessing that
      curl is downloading the result of the url (nothing) rather than the
      file the headers are pointing to unfortunality I still cant seem to
      find any relevent information that explains how curl handles this
      situation.

      -Ben

      $ch = curl_init('url' );

      $fp = fopen("download ed.zip", "w");

      curl_setopt($ch , CURLOPT_BINARYT RANSFER, 1);
      curl_setopt($ch , CURLOPT_FILE, $fp);

      $downloaded = curl_exec($ch);

      curl_close($ch) ;

      fclose($fp);

      HTTP/1.1 200 OK
      Date: Tue, 12 Jul 2005 23:42:34 GMT
      Server: Apache/2.0.46 (Red Hat)
      Accept-Ranges: bytes
      X-Powered-By: PHP/4.3.2
      Set-Cookie: PHPSESSID=ddb9c 654715d488d3f60 36127b1d624d; path=/
      Cache-Control: must-revalidate
      Content-Length:
      Content-Disposition: attachment; filename=awcsv. zip
      Connection: close
      Content-Type: application/zip

      Comment

      • Hilarion

        #4
        Re: curl extension downloads 1 byte only from zip files

        > Content-Length:


        The problem might be in empty Content-Length header field.
        You may signal this error to the company providing the data
        this way.


        Hilarion

        Comment

        • benji

          #5
          Re: curl extension downloads 1 byte only from zip files

          Thank you for your help with this problem I will add it to my support
          ticket and see if they do anything about it.

          Interestingly I have knocked up a solution using fsockopen which works
          with the data, although I would still prefer to use curl as I am using
          it for other parts of my application.

          Best Regards

          Ben

          Comment

          • Hilarion

            #6
            Re: curl extension downloads 1 byte only from zip files

            > Interestingly I have knocked up a solution using fsockopen which works[color=blue]
            > with the data, although I would still prefer to use curl as I am using
            > it for other parts of my application.[/color]


            You may try using curl_setopt with CURLOPT_WRITEFU NCTION and writing
            your own function handling data retrieval.


            Hilarion

            Comment

            • benji

              #7
              Re: curl extension downloads 1 byte only from zip files

              Thanks for all your help Hilarion,

              I will take a look at CURLOPT_WRITEFU NCTION

              Comment

              Working...