Hash Code?

[Joseph Lee]

Dear All,

Is there a function in C# that will hash code a large byte array into an int
value?

I am searching for a hash function that will convert any size data(as for me
the input is a byte array of size 20) to a value range of my choosing.

If there is no such thing, would need some idea how to program this in C#.

In Java a byte array can be dump into a BigInteger object and then reduced
to a smaller value by dividing it. but this is a bad way and it is not like
real hashing...

Thanks

Joey

[Justin Rogers]

Re: Hash Code?

System.Security .Cryptography then look at HashAlgorithm and it's derived
implementations .
What you'll note is that each hashing algorithm has output of different lengths,
so you'll have
to convert the smaller byte array hash value that you get into some other form
for storage.
I often turn hash values into hex strings for storage in the database.


--
Justin Rogers
DigiTec Web Consultants, LLC.
Blog: http://weblogs.asp.net/justin_rogers

"Joseph Lee" <joeylta1979@ho tmail.com> wrote in message
news:ekt4WwwNEH A.624@TK2MSFTNG P11.phx.gbl...[color=blue]
> Dear All,
>
> Is there a function in C# that will hash code a large byte array into an int
> value?
>
> I am searching for a hash function that will convert any size data(as for me
> the input is a byte array of size 20) to a value range of my choosing.
>
> If there is no such thing, would need some idea how to program this in C#.
>
> In Java a byte array can be dump into a BigInteger object and then reduced
> to a smaller value by dividing it. but this is a bad way and it is not like
> real hashing...
>
> Thanks
>
> Joey
>
>[/color]

[Joseph Lee]

Re: Hash Code?

Thanks..

By the way, is it possible to set(overwrite?) the output size of the hash
algorithms?
example SHA-1 will give 20 bytes and cannot be changed and i might need only
12 bytes? so izzit possible

Thanks again

Joey

"Justin Rogers" <Justin@games4d otnet.com> wrote in message
news:OLBPL1wNEH A.3988@TK2MSFTN GP09.phx.gbl...[color=blue]
> System.Security .Cryptography then look at HashAlgorithm and it's derived
> implementations .
> What you'll note is that each hashing algorithm has output of different[/color]
lengths,[color=blue]
> so you'll have
> to convert the smaller byte array hash value that you get into some other[/color]
form[color=blue]
> for storage.
> I often turn hash values into hex strings for storage in the database.
>
>
> --
> Justin Rogers
> DigiTec Web Consultants, LLC.
> Blog: http://weblogs.asp.net/justin_rogers
>
> "Joseph Lee" <joeylta1979@ho tmail.com> wrote in message
> news:ekt4WwwNEH A.624@TK2MSFTNG P11.phx.gbl...[color=green]
> > Dear All,
> >
> > Is there a function in C# that will hash code a large byte array into an[/color][/color]
int[color=blue][color=green]
> > value?
> >
> > I am searching for a hash function that will convert any size data(as[/color][/color]
for me[color=blue][color=green]
> > the input is a byte array of size 20) to a value range of my choosing.
> >
> > If there is no such thing, would need some idea how to program this in[/color][/color]
C#.[color=blue][color=green]
> >
> > In Java a byte array can be dump into a BigInteger object and then[/color][/color]
reduced[color=blue][color=green]
> > to a smaller value by dividing it. but this is a bad way and it is not[/color][/color]
like[color=blue][color=green]
> > real hashing...
> >
> > Thanks
> >
> > Joey
> >
> >[/color]
>
>[/color]

[Daniel O'Connell [C# MVP]]

Re: Hash Code?


"Joseph Lee" <joeylta1979@ho tmail.com> wrote in message
news:umujzZyNEH A.1388@TK2MSFTN GP09.phx.gbl...[color=blue]
> Thanks..
>
> By the way, is it possible to set(overwrite?) the output size of the hash
> algorithms?
> example SHA-1 will give 20 bytes and cannot be changed and i might need
> only
> 12 bytes? so izzit possible[/color]

No, you should either use a hash provider that has the proper size(don't
think any are 12) or discard some of the bytes returned so that it gets down
to your size.

What exactly are you trying to achieve here, anyway? If its for duplicate
checking or for security\crypto graphy reasons then its probably best to keep
all the bytes, but if you are trying to convert it down simply for an
identifier you might be able to find a simple algorthim to do what you want
or safely use the TripleDes keyed hash which returns an 8 byte hash.[color=blue]
>
> Thanks again
>
> Joey
>
> "Justin Rogers" <Justin@games4d otnet.com> wrote in message
> news:OLBPL1wNEH A.3988@TK2MSFTN GP09.phx.gbl...[color=green]
>> System.Security .Cryptography then look at HashAlgorithm and it's derived
>> implementations .
>> What you'll note is that each hashing algorithm has output of different[/color]
> lengths,[color=green]
>> so you'll have
>> to convert the smaller byte array hash value that you get into some other[/color]
> form[color=green]
>> for storage.
>> I often turn hash values into hex strings for storage in the database.
>>
>>
>> --
>> Justin Rogers
>> DigiTec Web Consultants, LLC.
>> Blog: http://weblogs.asp.net/justin_rogers
>>
>> "Joseph Lee" <joeylta1979@ho tmail.com> wrote in message
>> news:ekt4WwwNEH A.624@TK2MSFTNG P11.phx.gbl...[color=darkred]
>> > Dear All,
>> >
>> > Is there a function in C# that will hash code a large byte array into
>> > an[/color][/color]
> int[color=green][color=darkred]
>> > value?
>> >
>> > I am searching for a hash function that will convert any size data(as[/color][/color]
> for me[color=green][color=darkred]
>> > the input is a byte array of size 20) to a value range of my choosing.
>> >
>> > If there is no such thing, would need some idea how to program this in[/color][/color]
> C#.[color=green][color=darkred]
>> >
>> > In Java a byte array can be dump into a BigInteger object and then[/color][/color]
> reduced[color=green][color=darkred]
>> > to a smaller value by dividing it. but this is a bad way and it is not[/color][/color]
> like[color=green][color=darkred]
>> > real hashing...
>> >
>> > Thanks
>> >
>> > Joey
>> >
>> >[/color]
>>
>>[/color]
>
>[/color]

[Horatiu Ripa]

Re: Hash Code?

It's trivial to make your own hash algoritm, why didn't you just do it from
scratch? Than you can have whatever output you want.

--
Horatiu Ripa

"Joseph Lee" <joeylta1979@ho tmail.com> wrote in message
news:umujzZyNEH A.1388@TK2MSFTN GP09.phx.gbl...[color=blue]
> Thanks..
>
> By the way, is it possible to set(overwrite?) the output size of the hash
> algorithms?
> example SHA-1 will give 20 bytes and cannot be changed and i might need[/color]
only[color=blue]
> 12 bytes? so izzit possible
>
> Thanks again
>
> Joey
>
> "Justin Rogers" <Justin@games4d otnet.com> wrote in message
> news:OLBPL1wNEH A.3988@TK2MSFTN GP09.phx.gbl...[color=green]
> > System.Security .Cryptography then look at HashAlgorithm and it's derived
> > implementations .
> > What you'll note is that each hashing algorithm has output of different[/color]
> lengths,[color=green]
> > so you'll have
> > to convert the smaller byte array hash value that you get into some[/color][/color]
other[color=blue]
> form[color=green]
> > for storage.
> > I often turn hash values into hex strings for storage in the database.
> >
> >
> > --
> > Justin Rogers
> > DigiTec Web Consultants, LLC.
> > Blog: http://weblogs.asp.net/justin_rogers
> >
> > "Joseph Lee" <joeylta1979@ho tmail.com> wrote in message
> > news:ekt4WwwNEH A.624@TK2MSFTNG P11.phx.gbl...[color=darkred]
> > > Dear All,
> > >
> > > Is there a function in C# that will hash code a large byte array into[/color][/color][/color]
an[color=blue]
> int[color=green][color=darkred]
> > > value?
> > >
> > > I am searching for a hash function that will convert any size data(as[/color][/color]
> for me[color=green][color=darkred]
> > > the input is a byte array of size 20) to a value range of my choosing.
> > >
> > > If there is no such thing, would need some idea how to program this in[/color][/color]
> C#.[color=green][color=darkred]
> > >
> > > In Java a byte array can be dump into a BigInteger object and then[/color][/color]
> reduced[color=green][color=darkred]
> > > to a smaller value by dividing it. but this is a bad way and it is not[/color][/color]
> like[color=green][color=darkred]
> > > real hashing...
> > >
> > > Thanks
> > >
> > > Joey
> > >
> > >[/color]
> >
> >[/color]
>
>[/color]

[Joseph Lee]

Re: Hash Code?

Thanks for helping me :)
[color=blue]
> What exactly are you trying to achieve here, anyway? If its for duplicate
> checking or for security\crypto graphy reasons then its probably best to[/color]
keep[color=blue]
> all the bytes, but if you are trying to convert it down simply for an
> identifier you might be able to find a simple algorthim to do what you[/color]
want[color=blue]
> or safely use the TripleDes keyed hash which returns an 8 byte hash.[color=green]
> >[/color][/color]

Before i contiunue, i would like to explain that i am trying to write a
research paper comparing different search method on encrypted data.
I have look at papers by Song, Wagner, Perring

http://citeseer.ist.psu.edu/song00practical.html

and others...

One of them i am looking at is by Eu Jin Goh
http://crypto.stanford.edu/~eujin/papers/secureindex/ -> page
http://crypto.stanford.edu/~eujin/pa...-encsearch.pdf ->
the slides

So I am trying to implement this method of searching on encrypted data

I will try to summarize and simplify about the method implementation
1>a keyword is hashed using a hash method, the author suggests HMAC-SHA1
2>the ouput value determines the location of an array where the value in
that location is changed to flag an entry.(the author does not define
anything about how the array is created in programming, just that it acts
like a database of single bit value 0 and 1 to mark as flagged or not)
3>this is done to every keywords there is filling up the array

So I am having a problem where the output value is a 20 bytes(HMAC-SHA1).
I don't think that an array that i define can be of that size?? am i wrong
here?

So i need a way to either
reduce the size of the output value 20 bytes to a smaller value that i can
define array[value] and flag the value in the array
or a way to represent a huge array for a value up to 20 bytes location
spaces 2^20*8

Any other way of implementation is greatly appreciated.

[Jon Skeet [C# MVP]]

Re: Hash Code?

Joseph Lee <joeylta1979@ho tmail.com> wrote:

<snip>
[color=blue]
> I will try to summarize and simplify about the method implementation
> 1>a keyword is hashed using a hash method, the author suggests HMAC-SHA1
> 2>the ouput value determines the location of an array where the value in
> that location is changed to flag an entry.(the author does not define
> anything about how the array is created in programming, just that it acts
> like a database of single bit value 0 and 1 to mark as flagged or not)
> 3>this is done to every keywords there is filling up the array
>
> So I am having a problem where the output value is a 20 bytes(HMAC-SHA1).
> I don't think that an array that i define can be of that size?? am i wrong
> here?
>
> So i need a way to either
> reduce the size of the output value 20 bytes to a smaller value that i can
> define array[value] and flag the value in the array
> or a way to represent a huge array for a value up to 20 bytes location
> spaces 2^20*8
>
> Any other way of implementation is greatly appreciated.[/color]

Unless you're willing to use really large amounts of memory for this
array, I would suggest a different technique - the same technique used
in normal hashtables.

Break your sparse array into "buckets". The more entries you have, the
more buckets you have. Each bucket represents part of the array - each
bucket is usually the same size, and they're set at regular intervals.
Within each bucket, you have a straight list of entries. Within a
hashtable each entry would be the hash code, the key and the value.

When you are asked to look up a key (or a hashcode) you find the right
bucket (which is very quick, as it's basically just a division
operation) and then you walk down the list of entries in that bucket.
The more buckets you have, the faster it is to find an entry but the
more memory is taken.

I would suggest cutting the 20 byte hash down to 8 bytes to start with,
as you can then use a long to represent it in a simple way. You can
always keep the "full" hash in the list entries, if you want.

--
Jon Skeet - <skeet@pobox.co m>

http://www.pobox.com/~skeet

If replying to the group, please do not mail me too