Memory Management

[elziko]

I have an .NET application that calls a DLL compiled with a fortran
compiler. Users are having a problem that the fortran DLL complains that is
unable to allocate memory when the arrays it is using are too big.

After reading around I decided that increasing the MaxWorkingSet for my
process may help.

However, the Fortran DLL fails (when it tries to allocate memory for an
array containing abotu 900Mb of data) before it gets to this MaxWorkingSet
value (about 1400Mb).

So, to try and get an understanding of this I tried to set the MaxWorkignSet
to about 500Mb. However, the fortran DLL is quite happily able to allocate
memory for an array that contains several 100Mb over this value. During
this, looking in the task manager I aslo see the peak memory usage has also
gone well over the MaxWorkingSet value.

Can someone explain to me what MaxWorkignSet really means? And is there
anyway I can get windows to let the fortran DLL allocate more memory?

Since someone is sure to ask, the fortran DLL contains a subroutine that
implements a mathematical model and I have now control over its desgn or how
much memory it tries to allocate.

TIA

[Jamin Guy]

Re: Memory Management

It sounds like you might have a memory leak in your application. If
you are in a looping routine that allocates memory, which a DLL call
could very well do, then make sure that you are deallocating the memory
and cleaning up appropriately because .Net can not automatically clean
up unmanaged resources.

[elziko]

Re: Memory Management

> It sounds like you might have a memory leak in your application.

Thanks but this is not a memory leak. The DLL **needs** to use this amount
of memory and also quite happily cleans up after itself. What I need is an
understanding of what the MaxWorkingSet is/does and what changing its value
should do. The MSDN documentation is very thin on this.

Thanks

[Sean Hederman]

Re: Memory Management

"elziko" <elziko@yahoo.c o.uk> wrote in message
news:uPsbqFgVFH A.3636@TK2MSFTN GP14.phx.gbl...[color=blue][color=green]
>> It sounds like you might have a memory leak in your application.[/color]
>
> Thanks but this is not a memory leak. The DLL **needs** to use this amount
> of memory and also quite happily cleans up after itself. What I need is an
> understanding of what the MaxWorkingSet is/does and what changing its
> value should do. The MSDN documentation is very thin on this.[/color]

The working set consists of the pages of RAM that are resident (i.e. not on
disk) for process. Setting the MaxWorkingSet constrains the process into
using a smaller amount of physical memory, which in your case will result in
more memory being paged to disk. However, it is best to think of this value
as a suggestion to the operating system rather than a constraint. When you
set the Max Working Set to a small value, the OS will indeed truncate the
working set and page the processes memory away to disk, but given no other
memory constraints, it will reallocate physical memory as the application
requires it.

As for memory limits:
- A Windows Application is constrained to 2GB (out of a 4GB space, 2GB is
for the system).
- .NET chews up a fair bit of this, and will constrain you to about
800MB-1.2GB total IIRC.

If you boot your OS in /3GB mode , and your program is linked with
LARGEADDRESSAWA RE, Windows will only allocate 1GB to system, leaving 3GB for
your app, but .NET will still take a large chunk out of this. However you
must be sure that all your DLL's will be capable of handling large
addressing. If they're .NET they should be fine, but if that Fortran DLL is
not capable, you have a problem.

http://www.runuo.com/forum/showthread.php?t=38360

http://www.dotnet247.com/247reference/msgs/55/276803.aspx

[color=blue]
> Thanks[/color]

[elziko]

Re: Memory Management

> The working set consists of the pages of RAM that are resident (i.e. not[color=blue]
> on disk) for process. Setting the MaxWorkingSet constrains the process
> into using a smaller amount of physical memory, which in your case will
> result in more memory being paged to disk. However, it is best to think of
> this value as a suggestion to the operating system rather than a
> constraint. When you set the Max Working Set to a small value, the OS will
> indeed truncate the working set and page the processes memory away to
> disk, but given no other memory constraints, it will reallocate physical
> memory as the application requires it.[/color]

Thanks very much for that... and the links... our client is upgrading to 4Gb
so I'll be looking into this!

[Willy Denoyette [MVP]]

Re: Memory Management


"elziko" <elziko@yahoo.c o.uk> wrote in message
news:u$1iEwsVFH A.1148@tk2msftn gp13.phx.gbl...[color=blue][color=green]
>> The working set consists of the pages of RAM that are resident (i.e. not
>> on disk) for process. Setting the MaxWorkingSet constrains the process
>> into using a smaller amount of physical memory, which in your case will
>> result in more memory being paged to disk. However, it is best to think
>> of this value as a suggestion to the operating system rather than a
>> constraint. When you set the Max Working Set to a small value, the OS
>> will indeed truncate the working set and page the processes memory away
>> to disk, but given no other memory constraints, it will reallocate
>> physical memory as the application requires it.[/color]
>
> Thanks very much for that... and the links... our client is upgrading to
> 4Gb so I'll be looking into this!
>[/color]

Be careful, there is no guarantee this will help you out.
The problem is that your array is that big you probably won't find a single
FREE contiguous array of that size in your process space due to
fragmentation.
Your best bet is to move to a 64bit OS like XP 64 or W2K3 64, running on
these gives you 4GB user space in 32bit mode.

Willy.

[elziko]

Re: Memory Management

> Your best bet is to move to a 64bit OS like XP 64 or W2K3 64, running on[color=blue]
> these gives you 4GB user space in 32bit mode.[/color]


Thanks for that. And out of interest, how much user space do you get in
WinXp64 in 64bit mode?

[Willy Denoyette [MVP]]

Re: Memory Management


"elziko" <elziko@yahoo.c o.uk> wrote in message
news:eRePwRwVFH A.1404@TK2MSFTN GP09.phx.gbl...[color=blue][color=green]
>> Your best bet is to move to a 64bit OS like XP 64 or W2K3 64, running on
>> these gives you 4GB user space in 32bit mode.[/color]
>
>
> Thanks for that. And out of interest, how much user space do you get in
> WinXp64 in 64bit mode?
>[/color]

X64 user mode address space = 8TB.

Willy.

[Sean Hederman]

Re: Memory Management

"Willy Denoyette [MVP]" <willy.denoyett e@telenet.be> wrote in message
news:e7i2k$wVFH A.3620@TK2MSFTN GP09.phx.gbl...[color=blue]
> X64 user mode address space = 8TB.[/color]

Yeah, but .NET will need a fair whack of that for it's bookkeeping, so given
the lesson for 32bit, I'd assume that the amount available to your app will
be around 4TB.

Anyone know if LARGEADDRESSAWA RE and a /7TB switch will be around for apps
that want to create a 6TB array? ;-)

[elziko]

Re: Memory Management

> Anyone know if LARGEADDRESSAWA RE and a /7TB switch will be around for apps[color=blue]
> that want to create a 6TB array? ;-)[/color]

I'd imagine we'd be able to show-horn the model into 4TB. LOL I'm sure in
ten years time some body will come across this post in Google Groups and
think its not such a silly consideration after all!

[Willy Denoyette [MVP]]

Re: Memory Management


"Sean Hederman" <email.jpg@codi ngsanity.blogsp ot.com> wrote in message
news:d61lr2$o2c $1@ctb-nnrp2.saix.net. ..[color=blue]
> "Willy Denoyette [MVP]" <willy.denoyett e@telenet.be> wrote in message
> news:e7i2k$wVFH A.3620@TK2MSFTN GP09.phx.gbl...[color=green]
>> X64 user mode address space = 8TB.[/color]
>
> Yeah, but .NET will need a fair whack of that for it's bookkeeping, so
> given the lesson for 32bit, I'd assume that the amount available to your
> app will be around 4TB.
>
> Anyone know if LARGEADDRESSAWA RE and a /7TB switch will be around for apps
> that want to create a 6TB array? ;-)
>[/color]

Don't forget the OS will be written in managed code by that time which
brings us back at square one waiting for the first stable Beta of WinX128
;-)

Willy.