Size of Byte Array for big files?

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Anil Gupte

    #1

    Size of Byte Array for big files?

    I am using the following:
    Dim fsReadStream As New FileStream(Text BoxVideoFileNam e.Text, FileMode.Open,
    FileAccess.Read )
    Dim brReader As New BinaryReader(fs ReadStream)
    Dim ByteArray() As Byte

    ByteArray = brReader.ReadBy tes(fsReadStrea m.Length)

    This works fine for smaller files, but crashes with an overflow exception
    with a large file (2.5GB). Is the size of the array a problem? How can I
    change it? Any other solution?

    Thanx,
    --
    Anil Gupte
    Innovate with UsLeading the Way in Advanced Technology Solutions Discover state-of-the-art services in logistics, databases, and AI designed for evolving

    www.icinema.com


  • Patrice

    #2
    Re: Size of Byte Array for big files?

    AFAIK arrays are indexed using an integer whose max value allows for 2 Go.

    Even for smaller files it's likely better to avoid loading a whole file in
    memory unless this is strictly needed and especially if those files are not
    limited in size. A common practice is to read a file in "chunks" so that you
    consume only the size of a buffer that holds the current "access window" on
    file data...

    So it would depend what you are doing once the whole file is loaded in
    memory....

    ---
    Patrice

    "Anil Gupte" <anil-list@icinema.co ma écrit dans le message de news:
    urVPH984HHA.391 6@TK2MSFTNGP02. phx.gbl...
    >I am using the following:
    Dim fsReadStream As New FileStream(Text BoxVideoFileNam e.Text,
    FileMode.Open, FileAccess.Read )
    Dim brReader As New BinaryReader(fs ReadStream)
    Dim ByteArray() As Byte
    >
    ByteArray = brReader.ReadBy tes(fsReadStrea m.Length)
    >
    This works fine for smaller files, but crashes with an overflow exception
    with a large file (2.5GB). Is the size of the array a problem? How can I
    change it? Any other solution?
    >
    Thanx,
    --
    Anil Gupte
    Innovate with UsLeading the Way in Advanced Technology Solutions Discover state-of-the-art services in logistics, databases, and AI designed for evolving

    www.icinema.com
    >

    Comment

    • Spam Catcher

      #3
      Re: Size of Byte Array for big files?

      "Anil Gupte" <anil-list@icinema.co mwrote in
      news:urVPH984HH A.3916@TK2MSFTN GP02.phx.gbl:
      This works fine for smaller files, but crashes with an overflow
      exception with a large file (2.5GB). Is the size of the array a
      problem? How can I change it? Any other solution?
      Yes, you're creating a 2.5GB array in memory. I believe there is a 2GB
      limit?

      You should stream the file using a smaller array as data is needed.

      Comment

      • Anil Gupte

        #4
        Re: Size of Byte Array for big files?

        Thanx for your response. How do I "stream the file"?

        Best,
        --
        Anil Gupte
        Innovate with UsLeading the Way in Advanced Technology Solutions Discover state-of-the-art services in logistics, databases, and AI designed for evolving

        www.icinema.com

        "Spam Catcher" <spamhoneypot@r ogers.comwrote in message
        news:Xns99938A6 26E97Fusenethon eypotrogers@127 .0.0.1...
        "Anil Gupte" <anil-list@icinema.co mwrote in
        news:urVPH984HH A.3916@TK2MSFTN GP02.phx.gbl:
        >
        >This works fine for smaller files, but crashes with an overflow
        >exception with a large file (2.5GB). Is the size of the array a
        >problem? How can I change it? Any other solution?
        >
        Yes, you're creating a 2.5GB array in memory. I believe there is a 2GB
        limit?
        >
        You should stream the file using a smaller array as data is needed.

        Comment

        Working...