StringBuilder OutOfMemory

[Arnd Iffland]

Hi,

i want to use a StringBuilder to create a large string.

After instatiation, the MaxCapacity is 2.147.483.647, what is large
enough for my purpose.

During my procedure i get a SystemOutOfMEmo ryException at
System.String.G etStringForBuil der

The length of the StringBuilder is at this moment 17.825.779 and the
capacity 17.825.792.

Why do i get this exception? What can i do to get the MAxCapacity?

There is no other StringBuilder in the procedure.
The procedure is a WebService build with WebDeveloper Express and .Net
2.0.50727.42

Best regards,

Arnd

--

[Scully]

Re: StringBuilder OutOfMemory

Hi Arnd,

can you show me some code where the exception occurs? Maybe, I can be
of help.

Cheers.

[color=blue]
>
> --[/color]

[Hans Kesting]

Re: StringBuilder OutOfMemory

> Hi,[color=blue]
>
> i want to use a StringBuilder to create a large string.
>
> After instatiation, the MaxCapacity is 2.147.483.647, what is large
> enough for my purpose.
>
> During my procedure i get a SystemOutOfMEmo ryException at
> System.String.G etStringForBuil der
>
> The length of the StringBuilder is at this moment 17.825.779 and the
> capacity 17.825.792.
>
> Why do i get this exception? What can i do to get the MAxCapacity?
>
> There is no other StringBuilder in the procedure.
> The procedure is a WebService build with WebDeveloper Express and .Net
> 2.0.50727.42
>
> Best regards,
>
> Arnd[/color]

StringBuilder doesn't reserve MaxCapacity in memory, but a lower value
that you can specify when you instantiate it (or accept the default
value of 16 chars).
When you want to add text, so that the current capacity is not enough
anymore, then StringBuilder will try to *double* it's capacity.

So in your case it will want to go from 17MB to 34MB. I don't know if
it needs a *continuous* block of memory.

try it with a
StringBuilder sb = new StringBuilder(< your target capacity>);


Hans Kesting

[Willy Denoyette [MVP]]

Re: StringBuilder OutOfMemory


"Arnd Iffland" <iffland@rdv.de > wrote in message
news:%23bOKeU4j GHA.3496@TK2MSF TNGP02.phx.gbl. ..
| Hi,
|
| i want to use a StringBuilder to create a large string.
|
| After instatiation, the MaxCapacity is 2.147.483.647, what is large
| enough for my purpose.
|
| During my procedure i get a SystemOutOfMEmo ryException at
| System.String.G etStringForBuil der
|
| The length of the StringBuilder is at this moment 17.825.779 and the
| capacity 17.825.792.
|
| Why do i get this exception? What can i do to get the MAxCapacity?
|
| There is no other StringBuilder in the procedure.
| The procedure is a WebService build with WebDeveloper Express and .Net
| 2.0.50727.42
|
| Best regards,
|
| Arnd
|
| --
|

You can't create a StringBuilder with "MaxCapacit y is 2.147.483.647". The
largest StringBuilder on the current CLR versions available is ~1G
characters, but even such a SB can't be created on 32 bit Windows, you need
to run 64 bit windows for this. On 32 bit Windows the largest SB you are
able to create is subject of the amount of free "contiguous " virtual address
space, the size of free space highly depends on the amount of fragmentation
of both the 'process heap' and the 'large object heap', but it general it's
much smaller than ~1.2 GB - 1.6Gb (or ~600K - 800K characters). If your heap
is highly fragmented (which I guess it's the case here), it's even possible
that you can't allocate a 17.825.779 characters SB.

Willy.

[Kevin Spencer]

Re: StringBuilder OutOfMemory

Hi Arnd,

It's hard to say for sure what caused your OOM exception. There are many
possible factors involved. In fact, there are no methods of StringBuilder
that are documented to throw an OOM exception. So, it is entirely possible
that the StringBuilder did not throw the exception, but that the exception
was thrown as a result of a StringBuilder operation that resulted in a
memory allocation exceeding the amount of available memory on the system.

What I mean is, it looks to me like the problem is not related to the
MaxCapacity or Capacity properties of the StringBuilder, asevidenced by your
statistics. However, using a StringBuilder can cause memory allocation to
occur at different points, and this memory allocation may have caused the
exception.

For example, the default Capacity of a StringBuilder is 16 characters. When
you append more than 16 characters, the Capacity is doubled, and is doubled
again each time the length of the string exceeds the Capacity. Each time
this occurs, more memory must be allocated. Since it is allocated piecemeal,
you may be experiencing the result of memory fragmentation. This can be
avoided by setting the Capacity to a high number before beginning the
building of the String.

Another possibility lies in the usage of the ToString method of the
StringBuilder. The first time this is called, a reference to the actual
string in the buffer is returned. If you append to the String afterwards,
the existing string is copied to the buffer, so that the string reference
returned by the previous ToString call is not altered by your future
StringBuilder modification code. This means that additional memory is
allocated to hold the existing string buffer, and the new buffer.

You can read more about the technical details here:

http://www.codeproject.com/dotnet/strings.asp

--
HTH,

Kevin Spencer
Microsoft MVP
Professional Chicken Salad Alchemist

A lifetime is made up of
Lots of short moments.

"Arnd Iffland" <iffland@rdv.de > wrote in message
news:%23bOKeU4j GHA.3496@TK2MSF TNGP02.phx.gbl. ..[color=blue]
> Hi,
>
> i want to use a StringBuilder to create a large string.
>
> After instatiation, the MaxCapacity is 2.147.483.647, what is large
> enough for my purpose.
>
> During my procedure i get a SystemOutOfMEmo ryException at
> System.String.G etStringForBuil der
>
> The length of the StringBuilder is at this moment 17.825.779 and the
> capacity 17.825.792.
>
> Why do i get this exception? What can i do to get the MAxCapacity?
>
> There is no other StringBuilder in the procedure.
> The procedure is a WebService build with WebDeveloper Express and .Net
> 2.0.50727.42
>
> Best regards,
>
> Arnd
>
> --
>[/color]

[Ignacio Machin \( .NET/ C# MVP \)]

Re: StringBuilder OutOfMemory

hi


post the code of the method


--
--
Ignacio Machin,
ignacio.machin AT dot.state.fl.us
Florida Department Of Transportation

"Arnd Iffland" <iffland@rdv.de > wrote in message
news:%23bOKeU4j GHA.3496@TK2MSF TNGP02.phx.gbl. ..[color=blue]
> Hi,
>
> i want to use a StringBuilder to create a large string.
>
> After instatiation, the MaxCapacity is 2.147.483.647, what is large
> enough for my purpose.
>
> During my procedure i get a SystemOutOfMEmo ryException at
> System.String.G etStringForBuil der
>
> The length of the StringBuilder is at this moment 17.825.779 and the
> capacity 17.825.792.
>
> Why do i get this exception? What can i do to get the MAxCapacity?
>
> There is no other StringBuilder in the procedure.
> The procedure is a WebService build with WebDeveloper Express and .Net
> 2.0.50727.42
>
> Best regards,
>
> Arnd
>
> --
>[/color]

[Arnd Iffland]

Re: StringBuilder OutOfMemory

Willy Denoyette [MVP] wrote:
[color=blue]
>
> "Arnd Iffland" <iffland@rdv.de > wrote in message
> news:%23bOKeU4j GHA.3496@TK2MSF TNGP02.phx.gbl. ..[color=green]
> > Hi,
> >
> > i want to use a StringBuilder to create a large string.
> >
> > After instatiation, the MaxCapacity is 2.147.483.647, what is large
> > enough for my purpose.
> >
> > During my procedure i get a SystemOutOfMEmo ryException at
> > System.String.G etStringForBuil der
> >
> > The length of the StringBuilder is at this moment 17.825.779 and
> > the capacity 17.825.792.
> >
> > Why do i get this exception? What can i do to get the MAxCapacity?
> >
> > There is no other StringBuilder in the procedure.
> > The procedure is a WebService build with WebDeveloper Express and
> > .Net 2.0.50727.42
> >
> > Best regards,
> >
> > Arnd
> >
> > --
> >[/color]
>
> You can't create a StringBuilder with "MaxCapacit y is 2.147.483.647".
> The largest StringBuilder on the current CLR versions available is
> ~1G characters, but even such a SB can't be created on 32 bit
> Windows, you need to run 64 bit windows for this. On 32 bit Windows
> the largest SB you are able to create is subject of the amount of
> free "contiguous " virtual address space, the size of free space
> highly depends on the amount of fragmentation of both the 'process
> heap' and the 'large object heap', but it general it's much smaller
> than ~1.2 GB - 1.6Gb (or ~600K - 800K characters). If your heap is
> highly fragmented (which I guess it's the case here), it's even
> possible that you can't allocate a 17.825.779 characters SB.
>
> Willy.[/color]

Thanks for the explanation!

--

[Willy Denoyette [MVP]]

Re: StringBuilder OutOfMemory

Try this:

string s = new string('a', 1024*1024*32);
StringBuilder sb = new StringBuilder(1 024*512); // start with a SB of
512K chars.
for (int i = 0; i < 100; i++)
{
Console.WriteLi ne(sb.Capacity) ;
sb.Append(s);
}


The output should look like:
524288
33554432
67108864
134217728
134217728
268435456
268435456
268435456
268435456

notice the SB expanding, but as the SB expands, the preceeding buffer cannot
be re-used as it's contents must be preserved in order to copy the contents
to the new buffer. When the SB has expanded a second (and third and
fourth...) time, the preceeding "free buffers" are simply too small to hold
the new required SB buffer (they can be used to hold other objects however).
This results in a max. SB of 268435456 chars. in this sample.

That is why you should pre-allocate your SB when you have to deal with very
large strings. To prove this, just change the initial size into 620Mb
(1024*1024*620) and all should be fine for this simple console application,
but no-one will guarantee that it will do for a more complex program, nor
will it work for a Windows Forms application.


Willy.

"Ignacio Machin ( .NET/ C# MVP )" <ignacio.mach in AT dot.state.fl.us > wrote
in message news:uw5QJl7jGH A.1276@TK2MSFTN GP03.phx.gbl...
| hi
|
|
| post the code of the method
|
|
| --
| --
| Ignacio Machin,
| ignacio.machin AT dot.state.fl.us
| Florida Department Of Transportation
|
| "Arnd Iffland" <iffland@rdv.de > wrote in message
| news:%23bOKeU4j GHA.3496@TK2MSF TNGP02.phx.gbl. ..
| > Hi,
| >
| > i want to use a StringBuilder to create a large string.
| >
| > After instatiation, the MaxCapacity is 2.147.483.647, what is large
| > enough for my purpose.
| >
| > During my procedure i get a SystemOutOfMEmo ryException at
| > System.String.G etStringForBuil der
| >
| > The length of the StringBuilder is at this moment 17.825.779 and the
| > capacity 17.825.792.
| >
| > Why do i get this exception? What can i do to get the MAxCapacity?
| >
| > There is no other StringBuilder in the procedure.
| > The procedure is a WebService build with WebDeveloper Express and .Net
| > 2.0.50727.42
| >
| > Best regards,
| >
| > Arnd
| >
| > --
| >
|
|