ISA Server DestinationSet Performance

**Willy Denoyette [MVP]** · Nov 16 '05, 03:48 PM

Re: ISA Server DestinationSet Performance

Are you sure the list is completely filled (try scrolling to the bottom) in
2 seconds when running from the management console, could it be possible
that the list is filled with data taken asynchronously?
Could you try the same using a simple script (vbscript or jscript) and
check the time to load the whole 50K elements and check memory consumption?
As for the time spent to get and remove the elements from the hashtable, it
would help if you could post some code.

Willy.

"Joris Dobbelsteen" <joris.dobbelst een@mail.com> wrote in message
news:41a4f51e$0 $76510$b83b6cc0 @news.wanadoo.n l...[color=blue]
> Dear,
>
> I've made a little tool that synchronizes my ISA Server block list with a
> database. However the performance of this is very low.
>
> I'm retreiving 41140 rows from the database, storing them in a hash table
> and clearing the input tables within 1 or 2 seconds.
> Getting 41140 rows from the ISA server destination set takes me an awsome
> 173 seconds!
>
> Doing some processing (lookup and deletes on the hashtable) makes this
> retreiving good for 379 seconds.
> 173 seconds for retreiving the entries from the ISA server.
> 206 seconds for getting the entry elements and removing the entries from
> the
> hash table.
>
> The objects that I'm working with and probably gives the problem is
> FPCDestinationS et (and FPCDestination) . These are COM objects.
> I use a enumerator to work through the FPCDestinationS et (there is no
> other
> way to do so).
>
> My initial conclusion (though premature) would be that the COM interface
> is
> very slow. Loading the same list in the ISA Managment console takes below
> 2
> seconds!
>
> What can I do to improve the performance of this application?
> Should I use unsafe code for the COM parts?
> A secondary problem is using over 100 MB of RAM for the application (I got
> it down already from 160 MB). 50k* 200 Bytes = 10MB (50k rows with <80
> byte
> strings (take 120 byte for overhead). How do I get this down to a
> respectible level?
>
> In .NET the FPCDestinationS et has the function SerializeToBuff er returning
> a
> signed byte array, but this function is not documented anywhere.
>
> - Joris
>
>[/color]

**Joris Dobbelsteen** · Nov 16 '05, 03:51 PM

Re: ISA Server DestinationSet Performance

"Willy Denoyette [MVP]" <willy.denoyett e@pandora.be> wrote in message
news:uN7X5an0EH A.2568@TK2MSFTN GP10.phx.gbl...[color=blue]
> Are you sure the list is completely filled (try scrolling to the bottom)[/color]
in[color=blue]
> 2 seconds when running from the management console, could it be possible
> that the list is filled with data taken asynchronously?[/color]

I wasn't intending to count 40000 rows :(
But a light look made it look like its was OK. The start was good and so was
the end. Also the list didn't scroll too fast, making it all appear it had
40.000 rows listed.
(Immediately after it started).

The FPCDestinationS et has a function called:
SerializeToBuff er (which results in a array of signed bytes).
This function works is very fast, but its completely undocumented.
[color=blue]
> Could you try the same using a simple script (vbscript or jscript) and
> check the time to load the whole 50K elements and check memory[/color]
consumption?

I did it in VB6 (code at the end of the post)
Startup 19 MB.
Ending 50 MB
Delta is 30 MB for the destination array (I staticly allocated 99999
elements, 41140 are filled).

This only takes 2 seconds (oposite to 2-3 minutes!).
I would conclude that .NET-COM interop is extremely slow.

Are the any recommendations on how to increase its performance?
[color=blue]
> As for the time spent to get and remove the elements from the hashtable,[/color]
it[color=blue]
> would help if you could post some code.[/color]

The hash table isn't the problem.
The problem is getting the data from the enumerated objects (they are COM
too). And the above conclusion was that .NET - COM support is very crappy.

The hashtable works at nearly instant speeds for even 40000 elements.
The source is a DataTable that is put into it, removed and put into it
again.
Code below (all is done within a second of CPU time):
Sorry for the crappy layout (I need to drop Outlook Express some day).

*** START ***
#region "Table To Hashtable"
protected static Hashtable TableToStrings( DataTable dt)
{
Hashtable ht = new Hashtable(dt.Ro ws.Count, (float)0.1);

System.Diagnost ics.Debug.Write Line("* " + DateTime.Now.To String("hh:mm:s s")
+ " filling");
foreach (DataRow dr in dt.Rows)
{
string key = ((string)dr[0]).Replace('%', '*');
if (dr[1] != null && !(dr[1] is DBNull))
key += "$" + ((string)dr[1]).Replace('%', '*');
ht.Add(key, null);
}

System.Diagnost ics.Debug.Write Line("* " + DateTime.Now.To String("hh:mm:s s")
+ " clearing");
foreach (DataRow dr in dt.Rows)
{
string key = ((string)dr[0]).Replace('%', '*');
if (dr[1] != null && !(dr[1] is DBNull))
key += "$" + ((string)dr[1]).Replace('%', '*');
if (ht.Contains(ke y))
ht.Remove(key);
}

System.Diagnost ics.Debug.Write Line("* " + DateTime.Now.To String("hh:mm:s s")
+ " " + ht.Count + " remainders, clearing rest");
ht.Clear();

System.Diagnost ics.Debug.Write Line("* " + DateTime.Now.To String("hh:mm:s s")
+ " filling again");
foreach (DataRow dr in dt.Rows)
{
string key = ((string)dr[0]).Replace('%', '*');
if (dr[1] != null && !(dr[1] is DBNull))
key += "$" + ((string)dr[1]).Replace('%', '*');
ht.Add(key, null);
}
return ht;
}
#endregion
*** END ***
[color=blue]
>
> Willy.
>
>
>[/color]
[snip]

*** NOTES ***
Add "Microsoft Internet Securty and Acceleration Server Adminstrative
Objects" as a reference
*** START VB6 CODE FOR READING ELEMENTS ***
Private Sub Form_Load()
Dim thefpc As FPCLib.fpc
Set thefpc = CreateObject("F PC.Root")

Dim fpce As FPCLib.FPCEnter prise
Set fpce = thefpc.Enterpri se

Dim polelem As FPCLib.FPCPolic yElements
Set polelem = fpce.PolicyElem ents

Dim destSet As FPCLib.FPCDesti nationSet
Set destSet = polelem.Destina tionSets.Item(" Explicit Content")

Debug.Print Format(Now, "hh:mm:ss") & " navigation complete..."

Dim da() As String
ReDim da(0 To 99999) As String

Debug.Print Format(Now, "hh:mm:ss") & " allocation complete..."

Dim dest As FPCLib.FPCDesti nation
Dim x As Long
x = 0
For Each dest In destSet
Select Case dest.Type
Case FPCLib.FpcDesti nationAddressTy pe.fpcDestinati onTypeDomain:
da(x) = dest.DomainName
Case
FPCLib.FpcDesti nationAddressTy pe.fpcDestinati onTypeSingleIP:
da(x) = dest.IP_From
Case FPCLib.FpcDesti nationAddressTy pe.fpcDestinati onTypeIPRange:
da(x) = dest.IP_From & "^" & dest.IP_To
End Select

If Not IsNull(dest.Pat h) Then
da(x) = da(x) & "$" & dest.Path
End If

x = x + 1
Next

Debug.Print Format(Now, "hh:mm:ss") & " finished with " & x & "
elements..."
MsgBox "Done..."
End Sub
*** END ***

ISA Server DestinationSet Performance

ISA Server DestinationSet Performance

Comment

Comment