Performance issue. Will DictionaryBase help me?

[Gustaf]

I got an ArrayList with a large amount of objects in (probably no more
than 20000). Every time I add a new object to the array, I need to check
that it hasn't been added already, by comparing two properties from the
new object with every object in the array. It's a slow but necessary
process, because the input files are often overlapping. I'm looking for
ways to make it faster. Someone recommended DictionaryBase. Does anyone
here have positive experiences with it in cases like mine?

Gustaf

[SP]

Re: Performance issue. Will DictionaryBase help me?


"Gustaf" <gustafl@algone t.se> wrote in message
news:%23830$INm GHA.2056@TK2MSF TNGP03.phx.gbl. ..[color=blue]
>I got an ArrayList with a large amount of objects in (probably no more than
>20000). Every time I add a new object to the array, I need to check that it
>hasn't been added already, by comparing two properties from the new object
>with every object in the array. It's a slow but necessary process, because
>the input files are often overlapping. I'm looking for ways to make it
>faster. Someone recommended DictionaryBase. Does anyone here have positive
>experiences with it in cases like mine?[/color]

You need to use a hashtable and create a key from the 2 properties that
uniquely identifiesthe object. Check if the hashtable contains the key and
add the object if it doesn't. Dictionary implements a hashtable internally
and you get type safety as it uses generics. What were you currently doing?

SP

[Gustaf]

Re: Performance issue. Will DictionaryBase help me?

SP wrote:
[color=blue]
> You need to use a hashtable and create a key from the 2 properties that
> uniquely identifiesthe object. Check if the hashtable contains the key and
> add the object if it doesn't. Dictionary implements a hashtable internally
> and you get type safety as it uses generics. What were you currently doing?[/color]

It's a small app to consolidate MSN chat logs. The objects I need to
keep in memory are Message objects, and there's a MessageCollecti on
class already, but it doesn't implement any interface for collections.
Message objects are added to MessageCollecti on one file at a time with
AddChatFile(). That's where I run into this problem of checking which
messages are already added.

public void AddChatFile(str ing file)
{
ChatFileReader chatFileReader = new ChatFileReader( );
chatFileReader. Load(file);
Message[] messagesInFile = chatFileReader. Messages;
foreach (Message message in messagesInFile)
{
if (!IsAlreadyAdde d(message))
this.messages.A dd(message);
}
}

private bool IsAlreadyAdded( Message message)
{
if (this.messages == null) return false;
foreach (Message m in this.messages)
{
if (m.DateTime == message.DateTim e &&
m.From == message.From)
{
return true;
}
}
return false;
}

Gustaf

[SP]

Re: Performance issue. Will DictionaryBase help me?


"Gustaf" <gustafl@algone t.se> wrote in message
news:%23VnIq2Nm GHA.3980@TK2MSF TNGP02.phx.gbl. ..[color=blue]
> SP wrote:
>[color=green]
>> You need to use a hashtable and create a key from the 2 properties that
>> uniquely identifiesthe object. Check if the hashtable contains the key
>> and add the object if it doesn't. Dictionary implements a hashtable
>> internally and you get type safety as it uses generics. What were you
>> currently doing?[/color]
>
> It's a small app to consolidate MSN chat logs. The objects I need to keep
> in memory are Message objects, and there's a MessageCollecti on class
> already, but it doesn't implement any interface for collections. Message
> objects are added to MessageCollecti on one file at a time with
> AddChatFile(). That's where I run into this problem of checking which
> messages are already added.
>
> public void AddChatFile(str ing file)
> {
> ChatFileReader chatFileReader = new ChatFileReader( );
> chatFileReader. Load(file);
> Message[] messagesInFile = chatFileReader. Messages;
> foreach (Message message in messagesInFile)
> {
> if (!IsAlreadyAdde d(message))
> this.messages.A dd(message);
> }
> }
>
> private bool IsAlreadyAdded( Message message)
> {[/color]

Here is where you will check a hashtable / Dictionary for a key based on the
DateTime and From properties, e.g. m.DateTime.ToSt ring("yyyyMMddh hmmssffff")
+ m.From should give you a unique key.

SP

[color=blue]
> if (this.messages == null) return false;
> foreach (Message m in this.messages)
> {
> if (m.DateTime == message.DateTim e &&
> m.From == message.From)
> {
> return true;
> }
> }
> return false;
> }
>
> Gustaf[/color]

[Gustaf]

Re: Performance issue. Will DictionaryBase help me?

SP wrote:
[color=blue]
> Here is where you will check a hashtable / Dictionary for a key based on the
> DateTime and From properties, e.g. m.DateTime.ToSt ring("yyyyMMddh hmmssffff")
> + m.From should give you a unique key.[/color]

Thanks SP. That's a great improvment in performance, and smaller code
aswell. IsAlreadyAdded( ) could be replaced with a simple if clause.

public void AddChatFile(str ing file)
{
ChatFileReader chatFileReader = new ChatFileReader( );
chatFileReader. Load(file);
Message[] messagesInFile = chatFileReader. Messages;
foreach (Message message in messagesInFile)
{
string key = message.DateTim e + "_" + message.Alias;
if (!this.Dictiona ry.Contains(key ))
{
this.Dictionary .Add(key, message);
}
}
}

Gustaf