reading a file from a url to array

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • erbrose
    New Member
    • Oct 2006
    • 58

    reading a file from a url to array

    Hey All,
    I have been trying to research how to get a file (basically a csv file) from a url to an array and my search has come up with some examples, but not quite what I am looking for. Would appreciated any direction and or other examples of folks that have done this before.
    Basically we have a cloud computer set up (Hadoop) and the output of some of our processes is what I am trying to get to..
    the file is something like
    http://ourserver...com:port#/filetos...es/actual_file

    so if you put that in a browser, a 'save file dialog comes up' so i know the link if valid.
    In .net was looking into using SYSTEM.NET and a simple downloadfile sub
    Code:
    Public Sub DownLoadFile(ByVal address As String, _
                                ByVal fileName As String)
    
    
    End Sub
    but that just downloads it to a local file... which is what I want to do, but first have to to load each row into an array and manipulate the data just a bit....
    also read this bit of code somewhere but it has to do with images and again it only downloads doesn't really interact with the stream

    Code:
            Dim wr As HttpWebRequest = CType(WebRequest.Create(sfile), HttpWebRequest)
            Dim ws As HttpWebResponse = CType(wr.GetResponse(), HttpWebResponse)
            Dim str As Stream = ws.GetResponseStream()
            Dim inbuf(1000000) As Byte
            Dim bytesToRead As Integer = CInt(inbuf.Length)
            Dim bytesRead As Integer = 0
            While bytesToRead > 0
                Dim n As Integer = str.Read(inbuf, bytesRead, bytesToRead)
                If n = 0 Then
                    Exit While
                End If
                bytesRead += n
                bytesToRead -= n
            End While
    would definitely love some help!!!
    Thanks ahead of time,
    Eric
  • erbrose
    New Member
    • Oct 2006
    • 58

    #2
    alright after a bit more web searches.. found it, so thought I would share if other folks ever need to do this.
    Code:
    Imports System
    Imports System.IO
    Imports System.Net
    
    Dim uriWebSite As New Uri("http://ourserver...com:port#/filetos...es/actual_file")
            Dim WReq As WebRequest = System.Net.WebRequest.Create(uriWebSite)
            Dim wResp As WebResponse = WReq.GetResponse()
            Dim sr = New StreamReader(wResp.GetResponseStream())
            Dim sw = New StreamWriter("d:/temp/test.csv")
    
            Dim line As String = ""
    
            While Not sr.EndOfStream()
    
                line = sr.ReadLine()
                line = do some process
                sw.Write(line)
                sw.WriteLine()'not sure why but tried adding just a new line character to the the line and it didn't work, so just added writeline and it works fine
    
            End While
    
            sr.Close()
            sw.Close()

    Comment

    Working...