compressing data stored in a database

[Tarscher]

Hi all,

My application uses a database to store big blobs (0.5 - 1 Gb). I have
a List of a class Sample that I store in a blob. Since the blob can
become very big I want to use compression.

The class Sample contains a lot of properties. When I store the List
of samples only a few of the properties are used. Does it makes sense
to make a class sampleSimple that only contains the properties I use
for database storage and thus save storage space?

Is it possible to first compress the blob and then store it in the
database? Does .NET provides such functionality?

Many thanks in advance.

Stijn

[Peter Duniho]

Re: compressing data stored in a database

Tarscher wrote:

[...]
The class Sample contains a lot of properties. When I store the List
of samples only a few of the properties are used. Does it makes sense
to make a class sampleSimple that only contains the properties I use
for database storage and thus save storage space?

I would say so. Though, my first thought would be to try to understand
better why you are using a class that has many more properties than you
consider essential for saving.

Also, I admit I don't really get the "blob" stuff -- outside my database
use experience -- but if it's anything like the regular serialization
stuff, you should be able to suppress serialization of specific
properties without creating a whole new class (there's an attribute that
does this per-property).

Is it possible to first compress the blob and then store it in the
database? Does .NET provides such functionality?

Depends on how you are creating the blob. If the mechanism involves
getting a MemoryStream, byte array or similar binary representation of
the data, then you can probably use the GzipStream class to compress the
data before you store it in the database.

Pete

[Peter K]

Re: compressing data stored in a database

Peter Duniho <NpOeStPeAdM@Nn OwSlPiAnMk.comw rote in news:13cihm9ip5 mpje4
@corp.supernews .com:

<snip>

Depends on how you are creating the blob. If the mechanism involves
getting a MemoryStream, byte array or similar binary representation of
the data, then you can probably use the GzipStream class to compress the
data before you store it in the database.

Note also that some databases can be configured to automatically compress &
decompress data.

[Cor Ligthert [MVP]]

Re: compressing data stored in a database

Tarcher,

If your blob is used for images, than AFAIK you can spare the time. Most
image types are so optimalized that compressing can even result in some
extra bytes.

Cor

"Tarscher" <tarscher@gmail .comschreef in bericht
news:1187594537 .109055.120930@ 19g2000hsx.goog legroups.com...

Hi all,
>
My application uses a database to store big blobs (0.5 - 1 Gb). I have
a List of a class Sample that I store in a blob. Since the blob can
become very big I want to use compression.
>
The class Sample contains a lot of properties. When I store the List
of samples only a few of the properties are used. Does it makes sense
to make a class sampleSimple that only contains the properties I use
for database storage and thus save storage space?
>
Is it possible to first compress the blob and then store it in the
database? Does .NET provides such functionality?
>
Many thanks in advance.
>
Stijn
>

[Rad [Visual C# MVP]]

Re: compressing data stored in a database

On Mon, 20 Aug 2007 00:22:17 -0700, Tarscher <tarscher@gmail .com>
wrote:

>Hi all,
>
>My application uses a database to store big blobs (0.5 - 1 Gb). I have
>a List of a class Sample that I store in a blob. Since the blob can
>become very big I want to use compression.
>
>The class Sample contains a lot of properties. When I store the List
>of samples only a few of the properties are used. Does it makes sense
>to make a class sampleSimple that only contains the properties I use
>for database storage and thus save storage space?
>
>Is it possible to first compress the blob and then store it in the
>database? Does .NET provides such functionality?
>
>Many thanks in advance.
>
>Stijn

Well, first you might want to rethink your class design. If there are
so many properties that are not necessary for persistence, any
particular reason why they're there?

Also, like Cor has said, for some types of data like MP3, Image, Video
etc you are unlikely to get much savings from additional compression.
What sort of data are you storing in the blob?

--

http://bytes.thinkersroom.com

[Tarscher]

Re: compressing data stored in a database

On 20 aug, 14:30, "Rad [Visual C# MVP]" <r...@nospam.co mwrote:

On Mon, 20 Aug 2007 00:22:17 -0700,Tarscher<t arsc...@gmail.c om>
wrote:
>
>
>

Hi all,

>

My application uses a database to store big blobs (0.5 - 1 Gb). I have
a List of a class Sample that I store in a blob. Since the blob can
become very big I want to use compression.

>

The class Sample contains a lot of properties. When I store the List
of samples only a few of the properties are used. Does it makes sense
to make a class sampleSimple that only contains the properties I use
for database storage and thus save storage space?

>

Is it possible to first compress the blob and then store it in the
database? Does .NET provides such functionality?

>

Many thanks in advance.

>

Stijn

>
Well, first you might want to rethink your class design. If there are
so many properties that are not necessary for persistence, any
particular reason why they're there?
>
Also, like Cor has said, for some types of data like MP3, Image, Video
etc you are unlikely to get much savings from additional compression.
What sort of data are you storing in the blob?
>
--http://bytes.thinkersr oom.com

many thanks for all the replies.

There aren't actually that many properties but (4 in total) of which I
use 3 when stored in the database. Since I store a large number of
objects I think every byte I can save in serialization process will
eventuelly add up to the savings.

My object contains a string, an integer and a double. I create a list
of the object with about 1 million objects. This data I thus want to
compress