Which Collection class to use

[Scott Meddows]

I'm writing an NT service to provide security information enterprise wide. I would like to have all of this information cached
memory if a previous request loaded it into memory. Then using a common interface I would be able to invalidate and unload old
information as it is updated (Removing the item from the collection?)

My big question is what collection class will give me the greatest flexibility and greatest performance?

I hope this is clear... I really don't know how to describe what I'm trying to say.

[Imran Koradia]

Re: Which Collection class to use

IMHO, a hashtable would be a good choice. I'm not sure what kind of
information you are planning to store but I assume it would be some object
most likely with some unique identifier. You can add the information to the
hashtable as:

dim oHash as New Hashtable
o.Add(ID, mySecurityObjec t)

Then, when you need to access it again (either because you want to update
the information or remove the stale information) you can do something like:

if o.Contains(ID) then
'either this...
o.Remove(ID)
'or this..
o.Item(ID) = myUpdatedSecuri tyObject
end if

The Contains method lets you search for an existing object in the hashtable
structure extremely fast. The only requirement is that all the keys (IDs in
this case) must be unique.

I hope that gives you a good enough idea to proceed..
Imran.


"Scott Meddows" <scott_meddows_ no_spm@tsged-removeme.com> wrote in message
news:eNX7E4pmEH A.512@TK2MSFTNG P10.phx.gbl...[color=blue]
> I'm writing an NT service to provide security information enterprise wide.[/color]
I would like to have all of this information cached[color=blue]
> memory if a previous request loaded it into memory. Then using a common[/color]
interface I would be able to invalidate and unload old[color=blue]
> information as it is updated (Removing the item from the collection?)
>
> My big question is what collection class will give me the greatest[/color]
flexibility and greatest performance?[color=blue]
>
> I hope this is clear... I really don't know how to describe what I'm[/color]
trying to say.[color=blue]
>
>[/color]

[Scott Meddows]

Re: Which Collection class to use

Yeah, that helps. What kind of memory requirements are we talking about with this, though?

I will basically be keeping an object hierarchy in memory

"Imran Koradia" <nospam@microso ft.com> wrote in message news:%23nYKSCqm EHA.3452@TK2MSF TNGP15.phx.gbl. ..[color=blue]
> IMHO, a hashtable would be a good choice. I'm not sure what kind of
> information you are planning to store but I assume it would be some object
> most likely with some unique identifier. You can add the information to the
> hashtable as:
>
> dim oHash as New Hashtable
> o.Add(ID, mySecurityObjec t)
>
> Then, when you need to access it again (either because you want to update
> the information or remove the stale information) you can do something like:
>
> if o.Contains(ID) then
> 'either this...
> o.Remove(ID)
> 'or this..
> o.Item(ID) = myUpdatedSecuri tyObject
> end if
>
> The Contains method lets you search for an existing object in the hashtable
> structure extremely fast. The only requirement is that all the keys (IDs in
> this case) must be unique.
>
> I hope that gives you a good enough idea to proceed..
> Imran.
>
>
> "Scott Meddows" <scott_meddows_ no_spm@tsged-removeme.com> wrote in message
> news:eNX7E4pmEH A.512@TK2MSFTNG P10.phx.gbl...[color=green]
>> I'm writing an NT service to provide security information enterprise wide.[/color]
> I would like to have all of this information cached[color=green]
>> memory if a previous request loaded it into memory. Then using a common[/color]
> interface I would be able to invalidate and unload old[color=green]
>> information as it is updated (Removing the item from the collection?)
>>
>> My big question is what collection class will give me the greatest[/color]
> flexibility and greatest performance?[color=green]
>>
>> I hope this is clear... I really don't know how to describe what I'm[/color]
> trying to say.[color=green]
>>
>>[/color]
>
>[/color]

[Scott Meddows]

Re: Which Collection class to use

Also, would I just inherit from this class or create a new hashtable and just shove my objects in there?

"Imran Koradia" <nospam@microso ft.com> wrote in message news:%23nYKSCqm EHA.3452@TK2MSF TNGP15.phx.gbl. ..[color=blue]
> IMHO, a hashtable would be a good choice. I'm not sure what kind of
> information you are planning to store but I assume it would be some object
> most likely with some unique identifier. You can add the information to the
> hashtable as:
>
> dim oHash as New Hashtable
> o.Add(ID, mySecurityObjec t)
>
> Then, when you need to access it again (either because you want to update
> the information or remove the stale information) you can do something like:
>
> if o.Contains(ID) then
> 'either this...
> o.Remove(ID)
> 'or this..
> o.Item(ID) = myUpdatedSecuri tyObject
> end if
>
> The Contains method lets you search for an existing object in the hashtable
> structure extremely fast. The only requirement is that all the keys (IDs in
> this case) must be unique.
>
> I hope that gives you a good enough idea to proceed..
> Imran.
>
>
> "Scott Meddows" <scott_meddows_ no_spm@tsged-removeme.com> wrote in message
> news:eNX7E4pmEH A.512@TK2MSFTNG P10.phx.gbl...[color=green]
>> I'm writing an NT service to provide security information enterprise wide.[/color]
> I would like to have all of this information cached[color=green]
>> memory if a previous request loaded it into memory. Then using a common[/color]
> interface I would be able to invalidate and unload old[color=green]
>> information as it is updated (Removing the item from the collection?)
>>
>> My big question is what collection class will give me the greatest[/color]
> flexibility and greatest performance?[color=green]
>>
>> I hope this is clear... I really don't know how to describe what I'm[/color]
> trying to say.[color=green]
>>
>>[/color]
>
>[/color]

[Imran Koradia]

Re: Which Collection class to use

I believe memory shouldn't be an issue; When you add an object to the
hashtable, what you are really adding is a reference to the object which is
a 4-byte integer. Whatever collection you use to store your objects, thats
how items are going to be added to the collection - reference to your
objects (atleast for reference types; for value types I believe boxing is
involved since hashtable, arraylist, etc take in an Object type for the add
methods - although I'm not absolutely sure about that). The only extra
memory you need for hashtables is the memory required for storing the keys.
Thus, I think you should still be fine in terms of memory consumption. The
advantage that a hashtable data structure provides is the quick retrieval of
the items stored. The downside is that a hashtable does not store the items
in the sequence you added them which means you can't be sure of the order of
the items. Also, it does not have any sorting mechanism.

hope that helps..
Imran.

"Scott Meddows" <scott_meddows_ no_spm@tsged-removeme.com> wrote in message
news:OE1MQEqmEH A.3428@TK2MSFTN GP09.phx.gbl...[color=blue]
> Yeah, that helps. What kind of memory requirements are we talking about[/color]
with this, though?[color=blue]
>
> I will basically be keeping an object hierarchy in memory
>
> "Imran Koradia" <nospam@microso ft.com> wrote in message[/color]
news:%23nYKSCqm EHA.3452@TK2MSF TNGP15.phx.gbl. ..[color=blue][color=green]
> > IMHO, a hashtable would be a good choice. I'm not sure what kind of
> > information you are planning to store but I assume it would be some[/color][/color]
object[color=blue][color=green]
> > most likely with some unique identifier. You can add the information to[/color][/color]
the[color=blue][color=green]
> > hashtable as:
> >
> > dim oHash as New Hashtable
> > o.Add(ID, mySecurityObjec t)
> >
> > Then, when you need to access it again (either because you want to[/color][/color]
update[color=blue][color=green]
> > the information or remove the stale information) you can do something[/color][/color]
like:[color=blue][color=green]
> >
> > if o.Contains(ID) then
> > 'either this...
> > o.Remove(ID)
> > 'or this..
> > o.Item(ID) = myUpdatedSecuri tyObject
> > end if
> >
> > The Contains method lets you search for an existing object in the[/color][/color]
hashtable[color=blue][color=green]
> > structure extremely fast. The only requirement is that all the keys (IDs[/color][/color]
in[color=blue][color=green]
> > this case) must be unique.
> >
> > I hope that gives you a good enough idea to proceed..
> > Imran.
> >
> >
> > "Scott Meddows" <scott_meddows_ no_spm@tsged-removeme.com> wrote in[/color][/color]
message[color=blue][color=green]
> > news:eNX7E4pmEH A.512@TK2MSFTNG P10.phx.gbl...[color=darkred]
> >> I'm writing an NT service to provide security information enterprise[/color][/color][/color]
wide.[color=blue][color=green]
> > I would like to have all of this information cached[color=darkred]
> >> memory if a previous request loaded it into memory. Then using a[/color][/color][/color]
common[color=blue][color=green]
> > interface I would be able to invalidate and unload old[color=darkred]
> >> information as it is updated (Removing the item from the collection?)
> >>
> >> My big question is what collection class will give me the greatest[/color]
> > flexibility and greatest performance?[color=darkred]
> >>
> >> I hope this is clear... I really don't know how to describe what I'm[/color]
> > trying to say.[color=darkred]
> >>
> >>[/color]
> >
> >[/color]
>
>[/color]

[Imran Koradia]

Re: Which Collection class to use

"Scott Meddows" <scott_meddows_ no_spm@tsged-removeme.com> wrote in message
news:eHr4QNqmEH A.1980@TK2MSFTN GP12.phx.gbl...[color=blue]
> Also, would I just inherit from this class or create a new hashtable and
> just shove my objects in there?[/color]

You don't need to inherit from this class. As I mentioned in my first post,
just create an instance of the hashtable and start adding your objects to
the collection.
This MSDN article has sample code:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfSystemCollectionsHashtableClassTopic.asp

Imran.

[rawCoder]

Re: Which Collection class to use

You might wanna implement DictionaryBase if you want typed structure instead
of Object Based.

HTH
rawCoder

"Imran Koradia" <nojunk@microso ft.com> wrote in message
news:%23F325%23 qmEHA.556@tk2ms ftngp13.phx.gbl ...[color=blue]
> "Scott Meddows" <scott_meddows_ no_spm@tsged-removeme.com> wrote in message
> news:eHr4QNqmEH A.1980@TK2MSFTN GP12.phx.gbl...[color=green]
> > Also, would I just inherit from this class or create a new hashtable and
> > just shove my objects in there?[/color]
>
> You don't need to inherit from this class. As I mentioned in my first[/color]
post,[color=blue]
> just create an instance of the hashtable and start adding your objects to
> the collection.
> This MSDN article has sample code:
>[/color]
http://msdn.microsoft.com/library/de...ClassTopic.asp[color=blue]
>
>
> Imran.
>
>[/color]

[Cor Ligthert]

Re: Which Collection class to use

Scott,

The class with the most posibilities is in my opinion the datatabel, it is
not necessary to use that with a database you know. Talking about
performance with this is in my opinion talking about parts of nanoseconds.

Just my thought on the general part of your message. (You even can set that
in a dataset and than save it for tempory or saving use on disk)

Cor

"Scott Meddows" <scott_meddows_ no_spm@tsged-removeme.com>
[color=blue]
> I'm writing an NT service to provide security information enterprise wide.[/color]
I would like to have all of this information cached[color=blue]
> memory if a previous request loaded it into memory. Then using a common[/color]
interface I would be able to invalidate and unload old[color=blue]
> information as it is updated (Removing the item from the collection?)
>
> My big question is what collection class will give me the greatest[/color]
flexibility and greatest performance?[color=blue]
>
> I hope this is clear... I really don't know how to describe what I'm[/color]
trying to say.[color=blue]
>
>[/color]

[Scott Meddows]

Re: Which Collection class to use

Will it be as fast as the hashtable?

"rawCoder" <rawCoder@hotma il.com> wrote in message news:%23om52ivm EHA.3172@TK2MSF TNGP10.phx.gbl. ..[color=blue]
> You might wanna implement DictionaryBase if you want typed structure instead
> of Object Based.
>
> HTH
> rawCoder
>
> "Imran Koradia" <nojunk@microso ft.com> wrote in message
> news:%23F325%23 qmEHA.556@tk2ms ftngp13.phx.gbl ...[color=green]
>> "Scott Meddows" <scott_meddows_ no_spm@tsged-removeme.com> wrote in message
>> news:eHr4QNqmEH A.1980@TK2MSFTN GP12.phx.gbl...[color=darkred]
>> > Also, would I just inherit from this class or create a new hashtable and
>> > just shove my objects in there?[/color]
>>
>> You don't need to inherit from this class. As I mentioned in my first[/color]
> post,[color=green]
>> just create an instance of the hashtable and start adding your objects to
>> the collection.
>> This MSDN article has sample code:
>>[/color]
> http://msdn.microsoft.com/library/de...ClassTopic.asp[color=green]
>>
>>
>> Imran.
>>
>>[/color]
>
>[/color]

[Cor Ligthert]

Re: Which Collection class to use

Scott,

Almost every collection derives from IList or Icontainer so there is not so
much diverence, even when you make your own collection and you do it right,
you implement those.

What performance are you looking for? Adding to a simple arraylist
10.000.000 objects takes one second on an avarage modern computer?

Cor

[Imran Koradia]

Re: Which Collection class to use

I agree with Cor - indeed, performance in terms of what? Adding to the
collection, looping the collection, retrieving items from the collection,
etc etc. You need to define your metrics for performance. If you are talking
about adding your objects to the collections, I believe most of the
collections perform more or less the same. For retrieving items from a
collection, as a I mentioned in my earlier post Hashtable (or any other
class inherited from the DictionaryBase) perform the best simply because of
the underlying data structure used to store objects by this collection. As
rawCoder mentioned, if you want a strongly typed collection, you can inherit
from the DictionaryBase class and it should perform equally well.

hope that helps...
Imran.

"Scott Meddows" <scott_meddows_ no_spm@tsged-removeme.com> wrote in message
news:uhXBniymEH A.748@TK2MSFTNG P15.phx.gbl...[color=blue]
> Will it be as fast as the hashtable?
>
> "rawCoder" <rawCoder@hotma il.com> wrote in message[/color]
news:%23om52ivm EHA.3172@TK2MSF TNGP10.phx.gbl. ..[color=blue][color=green]
> > You might wanna implement DictionaryBase if you want typed structure[/color][/color]
instead[color=blue][color=green]
> > of Object Based.
> >
> > HTH
> > rawCoder
> >
> > "Imran Koradia" <nojunk@microso ft.com> wrote in message
> > news:%23F325%23 qmEHA.556@tk2ms ftngp13.phx.gbl ...[color=darkred]
> >> "Scott Meddows" <scott_meddows_ no_spm@tsged-removeme.com> wrote in[/color][/color][/color]
message[color=blue][color=green][color=darkred]
> >> news:eHr4QNqmEH A.1980@TK2MSFTN GP12.phx.gbl...
> >> > Also, would I just inherit from this class or create a new hashtable[/color][/color][/color]
and[color=blue][color=green][color=darkred]
> >> > just shove my objects in there?
> >>
> >> You don't need to inherit from this class. As I mentioned in my first[/color]
> > post,[color=darkred]
> >> just create an instance of the hashtable and start adding your objects[/color][/color][/color]
to[color=blue][color=green][color=darkred]
> >> the collection.
> >> This MSDN article has sample code:
> >>[/color]
> >[/color][/color]
http://msdn.microsoft.com/library/de...ClassTopic.asp[color=blue][color=green][color=darkred]
> >>
> >>
> >> Imran.
> >>
> >>[/color]
> >
> >[/color]
>
>[/color]

[Scott Meddows]

Re: Which Collection class to use

Thanks a lot for all your help, guys.

I think I'm going to try the hash table and the dictionary base and do some performance testing on each of those metrics.

Thanks

"Imran Koradia" <nospam@microso ft.com> wrote in message news:%23XK684ym EHA.2612@TK2MSF TNGP15.phx.gbl. ..[color=blue]
>I agree with Cor - indeed, performance in terms of what? Adding to the
> collection, looping the collection, retrieving items from the collection,
> etc etc. You need to define your metrics for performance. If you are talking
> about adding your objects to the collections, I believe most of the
> collections perform more or less the same. For retrieving items from a
> collection, as a I mentioned in my earlier post Hashtable (or any other
> class inherited from the DictionaryBase) perform the best simply because of
> the underlying data structure used to store objects by this collection. As
> rawCoder mentioned, if you want a strongly typed collection, you can inherit
> from the DictionaryBase class and it should perform equally well.
>
> hope that helps...
> Imran.
>
> "Scott Meddows" <scott_meddows_ no_spm@tsged-removeme.com> wrote in message
> news:uhXBniymEH A.748@TK2MSFTNG P15.phx.gbl...[color=green]
>> Will it be as fast as the hashtable?
>>
>> "rawCoder" <rawCoder@hotma il.com> wrote in message[/color]
> news:%23om52ivm EHA.3172@TK2MSF TNGP10.phx.gbl. ..[color=green][color=darkred]
>> > You might wanna implement DictionaryBase if you want typed structure[/color][/color]
> instead[color=green][color=darkred]
>> > of Object Based.
>> >
>> > HTH
>> > rawCoder
>> >
>> > "Imran Koradia" <nojunk@microso ft.com> wrote in message
>> > news:%23F325%23 qmEHA.556@tk2ms ftngp13.phx.gbl ...
>> >> "Scott Meddows" <scott_meddows_ no_spm@tsged-removeme.com> wrote in[/color][/color]
> message[color=green][color=darkred]
>> >> news:eHr4QNqmEH A.1980@TK2MSFTN GP12.phx.gbl...
>> >> > Also, would I just inherit from this class or create a new hashtable[/color][/color]
> and[color=green][color=darkred]
>> >> > just shove my objects in there?
>> >>
>> >> You don't need to inherit from this class. As I mentioned in my first
>> > post,
>> >> just create an instance of the hashtable and start adding your objects[/color][/color]
> to[color=green][color=darkred]
>> >> the collection.
>> >> This MSDN article has sample code:
>> >>
>> >[/color][/color]
> http://msdn.microsoft.com/library/de...ClassTopic.asp[color=green][color=darkred]
>> >>
>> >>
>> >> Imran.
>> >>
>> >>
>> >
>> >[/color]
>>
>>[/color]
>
>[/color]

[rawCoder]

Re: Which Collection class to use

Hi Cor,

Sure DataTable is by far the most usable data container facilitating easiest
data retrieval.

But compared to just a little more than an arrayList ( Dictionary Classes ),
can there be any performance loss visible to naked eye of the user ( and I
am not talking about the nanoseconds here )

There should be some place where all these benchmarks are placed for
different data structures not jsut theoritical assumptions of slow insertion
fast retrieval.

Talking about DataStructures, some of them which are not available in .NET -
are they available via third party lib or something ( Tree being missed the
most ). I see some five to six parts article on MSDN about DataStructures
using the prime and the elite language of .NET .. Ms. C# ... Why the code is
not available in VB.NET ???

Thank You
rawCoder


"Cor Ligthert" <notfirstname@p lanet.nl> wrote in message
news:%23B2V%23r vmEHA.4068@tk2m sftngp13.phx.gb l...[color=blue]
> Scott,
>
> The class with the most posibilities is in my opinion the datatabel, it is
> not necessary to use that with a database you know. Talking about
> performance with this is in my opinion talking about parts of nanoseconds.
>
> Just my thought on the general part of your message. (You even can set[/color]
that[color=blue]
> in a dataset and than save it for tempory or saving use on disk)
>
> Cor[/color]

[Cor Ligthert]

Re: Which Collection class to use

rawCoder,

Because of your question, I did a little test, creating a datatable with
columns item, name, address, city was about 3 times slower than creating the
same thing with objects in an hashtable. (I created 1000 times a table from
1000 rows in both, at me the hastable was about 3 seconds while the
datatabel about 9). When I did a test with 10 times a table from 100000
records both times doubles.

The question was however greathest flexibility and performance.

Suppose I want to get a persons with a specific name and city. With the
datatable I have a lot of very fast inbuild methods as select,
datarrowcollect ion.find or dataview.

The fastest method with a hashtable is probably loop through it and you
should create that loop and when you than want a collection of doubles you
even has to do more.

When you want to make another object with methods in that, feel free,
however in my opinion you would always lost in flexibility against the
datatable, and I will come everytime with a problem that is to do with the
datatable just by adding a method or using an extra ADONET class.

Cor