Abysmal performance of Scripting.Dictionary object on 2003 server

[Rodd Snook]

I have an application which makes extensive use of the
Scripting.Dicti onary object. I'm not doing anything silly like putting
them outside the page scope -- just creating quite a few of them and
stuffing quite a lot of data (from and MS SQL database) into them.

On Windows 2000 server, everything is fine. If the data structures get
really big it slows down, but for normal operation it's no problem.

Recently our hosting provider moved to Windows Server 2003, and everything
went to the dogs. Pages that used to load in 2-3 seconds can now take in
excess of 30 seconds to load. I've done extensive profiling, and it seems
that the dictionary is to blame. Specifically, putting data into them.
Creating them is fine, getting results from the database is fine, adding
the key/value pairs is a nightmare.

The extreme difference in performance makes me think that the server is
out of memory and thrashing, but the provider insists that there is
plenty of free memory, and that my application is the only one with a
problem. Maybe I'm wrong, and it's a data marshalling issue, not a memory
allocation issue.

The paranoid side of me says that ASP is intentionally "unoptimise d" in
2003 to "encourage" people to move to .NET.

Has anyone else seen this, or have any ideas what causes it?

Rodd

[Ken Schaefer]

Re: Abysmal performance of Scripting.Dicti onary object on 2003 server

Are you adding a lot of key/value pairs? Or adding a lot of data for a few
pairs?

I also use Scripting.Dicti onary in several Classic ASP applications (mostly
for holding a list of potential user errors, and then accessing them when
producing "error summary" for the user), both on Windows 2000 and Windows
2003, and have noticed no perceptable difference. However, I'm not adding
much data (just a short error message), though potentially might add 25-30
key/value pairs.

Cheers
Ken

"Rodd Snook" <usenet@nospam. snookums.port5. com> wrote in message
news:pan.2004.0 5.25.08.59.34.3 75000@nospam.sn ookums.port5.co m...
: I have an application which makes extensive use of the
: Scripting.Dicti onary object. I'm not doing anything silly like putting
: them outside the page scope -- just creating quite a few of them and
: stuffing quite a lot of data (from and MS SQL database) into them.
:
: On Windows 2000 server, everything is fine. If the data structures get
: really big it slows down, but for normal operation it's no problem.
:
: Recently our hosting provider moved to Windows Server 2003, and everything
: went to the dogs. Pages that used to load in 2-3 seconds can now take in
: excess of 30 seconds to load. I've done extensive profiling, and it seems
: that the dictionary is to blame. Specifically, putting data into them.
: Creating them is fine, getting results from the database is fine, adding
: the key/value pairs is a nightmare.
:
: The extreme difference in performance makes me think that the server is
: out of memory and thrashing, but the provider insists that there is
: plenty of free memory, and that my application is the only one with a
: problem. Maybe I'm wrong, and it's a data marshalling issue, not a memory
: allocation issue.
:
: The paranoid side of me says that ASP is intentionally "unoptimise d" in
: 2003 to "encourage" people to move to .NET.
:
: Has anyone else seen this, or have any ideas what causes it?
:
: Rodd
:

[Michael D. Kersey]

Re: Abysmal performance of Scripting.Dicti onary object on 2003 server

Ken Schaefer wrote:[color=blue]
> Are you adding a lot of key/value pairs? Or adding a lot of data for a few
> pairs?
>
> I also use Scripting.Dicti onary in several Classic ASP applications (mostly
> for holding a list of potential user errors, and then accessing them when
> producing "error summary" for the user), both on Windows 2000 and Windows
> 2003, and have noticed no perceptable difference. However, I'm not adding
> much data (just a short error message), though potentially might add 25-30
> key/value pairs.
>
> Cheers
> Ken
>
> "Rodd Snook" <usenet@nospam. snookums.port5. com> wrote in message
> news:pan.2004.0 5.25.08.59.34.3 75000@nospam.sn ookums.port5.co m...
> : I have an application which makes extensive use of the
> : Scripting.Dicti onary object. I'm not doing anything silly like putting
> : them outside the page scope -- just creating quite a few of them and
> : stuffing quite a lot of data (from and MS SQL database) into them.
> :
> : On Windows 2000 server, everything is fine. If the data structures get
> : really big it slows down, but for normal operation it's no problem.
> :
> : Recently our hosting provider moved to Windows Server 2003, and everything
> : went to the dogs. Pages that used to load in 2-3 seconds can now take in
> : excess of 30 seconds to load. I've done extensive profiling, and it seems
> : that the dictionary is to blame. Specifically, putting data into them.
> : Creating them is fine, getting results from the database is fine, adding
> : the key/value pairs is a nightmare.
> :
> : The extreme difference in performance makes me think that the server is
> : out of memory and thrashing, but the provider insists that there is
> : plenty of free memory, and that my application is the only one with a
> : problem. Maybe I'm wrong, and it's a data marshalling issue, not a memory
> : allocation issue.
> :
> : The paranoid side of me says that ASP is intentionally "unoptimise d" in
> : 2003 to "encourage" people to move to .NET.
> :
> : Has anyone else seen this, or have any ideas what causes it?
> :
> : Rodd[/color]
Scripting.Dicti onary object had (has?) some threading issues; Eric
Lippert of Microsoft wrote it while learning about threading:

http://blogs.msdn.com/ericlippert/archive/2003/09/19/53054.aspx

There he talks about problems that can occur when using that object.

Possible cures:
There is a component, called "Lookup Table object", written by Microsoft
programmers who wanted better performance than the Dictionary object
provided. Perhaps you could use it.

BTW why use the Scripting.Dicti onary object when you have Session and
Application variables available?

Good Luck,
Michael D. Kersey

[Ken Schaefer]

Re: Abysmal performance of Scripting.Dicti onary object on 2003 server

"Michael D. Kersey" <mdkersey@hal-pc.org> wrote in message
news:ugd3vFmQEH A.3732@TK2MSFTN GP10.phx.gbl...
: BTW why use the Scripting.Dicti onary object when you have Session and
: Application variables available?

Because you might run into scoping issues. Session/Application objects are
global, and there is no way to make them "local" (or reduce their scope in
any way).

If I use a Script.Dictiona ry component, then I can limit the scope of that
component (eg to just a routine, or a page). I do not need to worry about
accidently over-writing some value on another page, or in another routine.

Also, I'm wary of adding lots of unnecessary stuff to Session or Application
objects. That makes your app unportable. I have plenty of applications that
I've having issues porting to .NET (or adding new features using .NET)
because of excessive overuse of Session variables, which makes it difficult
to run part of the app using ASP, and part using ASP.NET

Cheers
Ken


"Michael D. Kersey" <mdkersey@hal-pc.org> wrote in message
news:ugd3vFmQEH A.3732@TK2MSFTN GP10.phx.gbl...
: Ken Schaefer wrote:
: > Are you adding a lot of key/value pairs? Or adding a lot of data for a
few
: > pairs?
: >
: > I also use Scripting.Dicti onary in several Classic ASP applications
(mostly
: > for holding a list of potential user errors, and then accessing them
when
: > producing "error summary" for the user), both on Windows 2000 and
Windows
: > 2003, and have noticed no perceptable difference. However, I'm not
adding
: > much data (just a short error message), though potentially might add
25-30
: > key/value pairs.
: >
: > Cheers
: > Ken
: >
: > "Rodd Snook" <usenet@nospam. snookums.port5. com> wrote in message
: > news:pan.2004.0 5.25.08.59.34.3 75000@nospam.sn ookums.port5.co m...
: > : I have an application which makes extensive use of the
: > : Scripting.Dicti onary object. I'm not doing anything silly like putting
: > : them outside the page scope -- just creating quite a few of them and
: > : stuffing quite a lot of data (from and MS SQL database) into them.
: > :
: > : On Windows 2000 server, everything is fine. If the data structures get
: > : really big it slows down, but for normal operation it's no problem.
: > :
: > : Recently our hosting provider moved to Windows Server 2003, and
everything
: > : went to the dogs. Pages that used to load in 2-3 seconds can now take
in
: > : excess of 30 seconds to load. I've done extensive profiling, and it
seems
: > : that the dictionary is to blame. Specifically, putting data into them.
: > : Creating them is fine, getting results from the database is fine,
adding
: > : the key/value pairs is a nightmare.
: > :
: > : The extreme difference in performance makes me think that the server
is
: > : out of memory and thrashing, but the provider insists that there is
: > : plenty of free memory, and that my application is the only one with a
: > : problem. Maybe I'm wrong, and it's a data marshalling issue, not a
memory
: > : allocation issue.
: > :
: > : The paranoid side of me says that ASP is intentionally "unoptimise d"
in
: > : 2003 to "encourage" people to move to .NET.
: > :
: > : Has anyone else seen this, or have any ideas what causes it?
: > :
: > : Rodd
: Scripting.Dicti onary object had (has?) some threading issues; Eric
: Lippert of Microsoft wrote it while learning about threading:
: http://blogs.msdn.com/ericlippert/ar.../19/53054.aspx
:
: There he talks about problems that can occur when using that object.
:
: Possible cures:
: There is a component, called "Lookup Table object", written by Microsoft
: programmers who wanted better performance than the Dictionary object
: provided. Perhaps you could use it.
:
: BTW why use the Scripting.Dicti onary object when you have Session and
: Application variables available?
:
: Good Luck,
: Michael D. Kersey

[Nikolay]

Re: Abysmal performance of Scripting.Dicti onary object on 2003 server

Probably, solution described in

http://support.microsoft.com/default.aspx?scid=kb;en-us;840875

Nick

"Rodd Snook" <usenet@nospam. snookums.port5. com> wrote in message
news:pan.2004.0 5.25.08.59.34.3 75000@nospam.sn ookums.port5.co m...[color=blue]
> I have an application which makes extensive use of the
> Scripting.Dicti onary object. I'm not doing anything silly like putting
> them outside the page scope -- just creating quite a few of them and
> stuffing quite a lot of data (from and MS SQL database) into them.
>
> On Windows 2000 server, everything is fine. If the data structures get
> really big it slows down, but for normal operation it's no problem.
>
> Recently our hosting provider moved to Windows Server 2003, and everything
> went to the dogs. Pages that used to load in 2-3 seconds can now take in
> excess of 30 seconds to load. I've done extensive profiling, and it seems
> that the dictionary is to blame. Specifically, putting data into them.
> Creating them is fine, getting results from the database is fine, adding
> the key/value pairs is a nightmare.
>
> The extreme difference in performance makes me think that the server is
> out of memory and thrashing, but the provider insists that there is
> plenty of free memory, and that my application is the only one with a
> problem. Maybe I'm wrong, and it's a data marshalling issue, not a memory
> allocation issue.
>
> The paranoid side of me says that ASP is intentionally "unoptimise d" in
> 2003 to "encourage" people to move to .NET.
>
> Has anyone else seen this, or have any ideas what causes it?
>
> Rodd
>[/color]

[Rodd Snook]

Re: Abysmal performance of Scripting.Dicti onary object on 2003 server

On Wed, 26 May 2004 11:54:36 +0400, Nikolay wrote:
[color=blue]
> Probably, solution described in
>
> http://support.microsoft.com/default...b;en-us;840875
>
> Nick[/color]

Well, the performance hit isn't writing to the network because all my
responses are buffered, and I've done profiling which points the finger
squarely at the dictionary object. This article about RDS sounds closer

http://support.microsoft.com/default.aspx?kbid=555093

but I'm not using RDS (which goes over HTTP, hence the problem)

However, it sounds like 2003 server has some serious issues. I'm going to
move the application to another hosting provider who can supply win2k
boxes.

Rodd

[Rodd Snook]

Re: Abysmal performance of Scripting.Dicti onary object on 2003 server

On Tue, 25 May 2004 09:13:33 -0500, Michael D. Kersey wrote:
[color=blue]
> Ken Schaefer wrote:[color=green]
>> Are you adding a lot of key/value pairs? Or adding a lot of data for a few
>> pairs?
>>
>> I also use Scripting.Dicti onary in several Classic ASP applications (mostly
>> for holding a list of potential user errors, and then accessing them when
>> producing "error summary" for the user), both on Windows 2000 and Windows
>> 2003, and have noticed no perceptable difference. However, I'm not adding
>> much data (just a short error message), though potentially might add 25-30
>> key/value pairs.
>>
>> Cheers
>> Ken
>>
>> "Rodd Snook" <usenet@nospam. snookums.port5. com> wrote in message
>> news:pan.2004.0 5.25.08.59.34.3 75000@nospam.sn ookums.port5.co m...
>> : I have an application which makes extensive use of the
>> : Scripting.Dicti onary object. I'm not doing anything silly like putting
>> : them outside the page scope -- just creating quite a few of them and
>> : stuffing quite a lot of data (from and MS SQL database) into them.
>> :
>> : On Windows 2000 server, everything is fine. If the data structures get
>> : really big it slows down, but for normal operation it's no problem.
>> :
>> : Recently our hosting provider moved to Windows Server 2003, and everything
>> : went to the dogs. Pages that used to load in 2-3 seconds can now take in
>> : excess of 30 seconds to load. I've done extensive profiling, and it seems
>> : that the dictionary is to blame. Specifically, putting data into them.
>> : Creating them is fine, getting results from the database is fine, adding
>> : the key/value pairs is a nightmare.
>> :
>> : The extreme difference in performance makes me think that the server is
>> : out of memory and thrashing, but the provider insists that there is
>> : plenty of free memory, and that my application is the only one with a
>> : problem. Maybe I'm wrong, and it's a data marshalling issue, not a memory
>> : allocation issue.
>> :
>> : The paranoid side of me says that ASP is intentionally "unoptimise d" in
>> : 2003 to "encourage" people to move to .NET.
>> :
>> : Has anyone else seen this, or have any ideas what causes it?
>> :
>> : Rodd[/color]
> Scripting.Dicti onary object had (has?) some threading issues; Eric
> Lippert of Microsoft wrote it while learning about threading:
> http://blogs.msdn.com/ericlippert/ar.../19/53054.aspx
>
> There he talks about problems that can occur when using that object.
>
> Possible cures:
> There is a component, called "Lookup Table object", written by Microsoft
> programmers who wanted better performance than the Dictionary object
> provided. Perhaps you could use it.[/color]

I was aware of the threading issues, and would like to move away from
dictionary objects so that I can more easily cache my data in session and
application scope. However, I have a lot of code that would need to be
rewritten, and no time to do it. Also, the current hosting environment
doesn't allow for the installation on 3rd-party components in most
situations. The code is also part of a product which is used outside of my
controlled hosting environment, and hence needs to be as portable as
possible.
[color=blue]
>
> BTW why use the Scripting.Dicti onary object when you have Session and
> Application variables available?[/color]

What Ken said. Plus, I have a good many of them organised in hierarchies
-- i.e. the values in some of the dictionary objects are other dictionary
objects. To answer his original question, I probably have two objects with
about 50 key/value pairs, but the values of each of these are themselves
dictionary objects with about 5-10 key/value pairs. Before you say "Lord!
That's an obscene amount.", I know it, and a lot of it could be cached at
the application level (in some other kind of data structure, or maybe even
as HTML), but the point is that the code performed acceptably on Win2k.

At the moment my preferred solution (apart from moving to another hosting
provider who has Win2k) is to rewrite the whole thing in .NET, or maybe
ASP/JScript or PHP (there, I said it!). Something with native support for
sparse, associative arrays and regular expressions.

Rodd

[Michael D. Kersey]

Re: Abysmal performance of Scripting.Dicti onary object on 2003 server

Rodd Snook wrote:[color=blue]
> On Tue, 25 May 2004 09:13:33 -0500, Michael D. Kersey wrote:[/color]
<snipped>[color=blue][color=green]
>>BTW why use the Scripting.Dicti onary object when you have Session and
>>Application variables available?[/color]
>
> What Ken said. Plus, I have a good many of them organised in hierarchies
> -- i.e. the values in some of the dictionary objects are other dictionary
> objects. To answer his original question, I probably have two objects with
> about 50 key/value pairs, but the values of each of these are themselves
> dictionary objects with about 5-10 key/value pairs. Before you say "Lord!
> That's an obscene amount.", I know it, and a lot of it could be cached at
> the application level (in some other kind of data structure, or maybe even
> as HTML), but the point is that the code performed acceptably on Win2k.[/color]
So a total of ~500-1000 items: not many, although we know nothing of
their content and, if they're built up and then discarded with each
page, that's a lot of overhead.

Are these dictionary objects running at low, medium(pooled) or high
(Isolated) protection level or running remotely?

Are they saved in the Session or Application objects or to files or a
database between page executions, or are they created and rebuilt with
each page execution?[color=blue]
>
> At the moment my preferred solution (apart from moving to another hosting
> provider who has Win2k) is to rewrite the whole thing in .NET, or maybe
> ASP/JScript or PHP (there, I said it!). Something with native support for
> sparse, associative arrays and regular expressions.[/color]
That would be Perl ( http://www.ActiveState.com/ ).

[Rodd Snook]

Re: Abysmal performance of Scripting.Dicti onary object on 2003 server

On Fri, 28 May 2004 00:20:47 -0500, Michael D. Kersey wrote:
[color=blue]
> Rodd Snook wrote:[color=green]
>> On Tue, 25 May 2004 09:13:33 -0500, Michael D. Kersey wrote:[/color]
> <snipped>[color=green][color=darkred]
>>>BTW why use the Scripting.Dicti onary object when you have Session and
>>>Applicatio n variables available?[/color]
>>
>> What Ken said. Plus, I have a good many of them organised in hierarchies
>> -- i.e. the values in some of the dictionary objects are other dictionary
>> objects. To answer his original question, I probably have two objects with
>> about 50 key/value pairs, but the values of each of these are themselves
>> dictionary objects with about 5-10 key/value pairs. Before you say "Lord!
>> That's an obscene amount.", I know it, and a lot of it could be cached at
>> the application level (in some other kind of data structure, or maybe even
>> as HTML), but the point is that the code performed acceptably on Win2k.[/color]
> So a total of ~500-1000 items: not many, although we know nothing of
> their content and, if they're built up and then discarded with each
> page, that's a lot of overhead.
>
> Are these dictionary objects running at low, medium(pooled) or high
> (Isolated) protection level or running remotely?[/color]

That's a good question, but I'm not sure of the answer. The environment
that's giving problems is a shared server at a hosting provider, so I
can't check on the configuration. I'm also not familiar with the
configuration of Win2k3 and IIS6. Apparently each customer "slot" has it's
own application pool, so I'm guessing that the ASP application is
isolated. The Dictionary object is in-process by default in NT4 and Win2k,
and I would imagine that is also the case on Win2k3, but I may be wrong.
[color=blue]
>
> Are they saved in the Session or Application objects or to files or a
> database between page executions, or are they created and rebuilt with
> each page execution?[/color]

They are rebuilt from the database with each page execution. I agree that
this is a lot of overhead, but the application performs acceptably on
Win2k. I would cache them at the application level if it weren't for the
threading limitation of the object.
[color=blue][color=green]
>>
>> At the moment my preferred solution (apart from moving to another hosting
>> provider who has Win2k) is to rewrite the whole thing in .NET, or maybe
>> ASP/JScript or PHP (there, I said it!). Something with native support for
>> sparse, associative arrays and regular expressions.[/color]
> That would be Perl ( http://www.ActiveState.com/ ).[/color]

PHP and JScript have these features too, but are semantically closer to
VBScript and thus the porting would be easier.