detecting corrupt/stuffed files (how to?)

[Jole]

Hi
I'm writing a program that needs to read from a file. In order for the
program to be robust, it should somehow check that the file isn't corrupt,
or stuffed in any way. For example, that file may have been created but a
crash occurred at that point in time (while it was being created), damaging
the file. Now, my program which needs to read from this file, should first
check that it's in good condition, and that it hasn't been stuffed up in
any way. What is the normal way of doing this?

thanks in advance
Jole

I'm using Java and am aware of some of those File.XX methods. perhaps the
File.isReadable () methods will fail if the files have been damaged or
corrupted? or another way ?

[John Harlow]

Re: detecting corrupt/stuffed files (how to?)

> I'm using Java and am aware of some of those File.XX methods.[color=blue]
> perhaps the File.isReadable () methods will fail if the files have
> been damaged or corrupted? or another way ?[/color]

You won't be able to tell if a file was partially written from a crash.
I'll just abruptly end or contain nothing.

If you control the file structure, you can do such things as put a known
structure at the head of the file which contain "magic words" which must
match, and include items in the header such as file length and checksum
which must match.

Whatever you do, just make sure your reader can handle anything you throw at
it, because users will find a way to do just that! ;)

[perry]

Re: detecting corrupt/stuffed files (how to?)

well, it's really not necessary to go thru the trouble of setting up a
isReadable() method... in fact, from what you have described, i'd be
inclined to advise against it, unless you want to maintain static /
post-mortem data...

for you see, the java i/o libraries have all the "isReadable "
functionality you could ever imagine. and should a file be altered or
damaged or never properly written in the first place, java is more than
capable of detecting something wrong depending on the level of
sensitivity you are interested in.

for simplicity in your design, you should store and retrieve data as
objects as a preference with any object oriented langauge. unless your
application is importing or exporting data this ought to be the default.
the reason being due to the excellent design put into saving objects as
well as detecting any changes to the object image on the restore.

now by objects, i am referring to the Object streaming classes and not
so much a vendor of object databases.

now for whatever reason you should be interested in investigating a
corrupted object stream, you have the flexibility of opening the same
file/stream with a lower level protocol, say a data stream. this will
allow you to examine the bytes without throwing higher level exceptions

finally, the exception mechanism became a main stream procedure in the
object design community for a reason.... namely simplicity in design
with respect to error management. the phylosophy of exceptions is that
you are separating error detection from error handling. this was a major
breakthrew in code design as it allowed developers of commercial
frameworks (or simply previous projects) to identify errors and allow
future programmers to be able to handle future error handling needs
without out having to "change" the origional code.... a lesson well
learned from the days of spagetti design...

i hope this helped

- perry


Jole wrote:[color=blue]
> Hi
> I'm writing a program that needs to read from a file. In order for the
> program to be robust, it should somehow check that the file isn't corrupt,
> or stuffed in any way. For example, that file may have been created but a
> crash occurred at that point in time (while it was being created), damaging
> the file. Now, my program which needs to read from this file, should first
> check that it's in good condition, and that it hasn't been stuffed up in
> any way. What is the normal way of doing this?
>
> thanks in advance
> Jole
>
> I'm using Java and am aware of some of those File.XX methods. perhaps the
> File.isReadable () methods will fail if the files have been damaged or
> corrupted? or another way ?
>
>[/color]

[perry]

Re: detecting corrupt/stuffed files (how to?)

well, it's really not necessary to go thru the trouble of setting up a
isReadable() method... in fact, from what you have described, i'd be
inclined to advise against it, unless you want to maintain static /
post-mortem data...

for you see, the java i/o libraries have all the "isReadable "
functionality you could ever imagine. and should a file be altered or
damaged or never properly written in the first place, java is more than
capable of detecting something wrong depending on the level of
sensitivity you are interested in.

for simplicity in your design, you should store and retrieve data as
objects as a preference with any object oriented langauge. unless your
application is importing or exporting data this ought to be the default.
the reason being due to the excellent design put into saving objects as
well as detecting any changes to the object image on the restore.

now by objects, i am referring to the Object streaming classes and not
so much a vendor of object databases.

now for whatever reason you should be interested in investigating a
corrupted object stream, you have the flexibility of opening the same
file/stream with a lower level protocol, say a data stream. this will
allow you to examine the bytes without throwing higher level exceptions

finally, the exception mechanism became a main stream procedure in the
object design community for a reason.... namely simplicity in design
with respect to error management. the phylosophy of exceptions is that
you are separating error detection from error handling. this was a major
breakthrew in code design as it allowed developers of commercial
frameworks (or simply previous projects) to identify errors and allow
future programmers to be able to handle future error handling needs
without out having to "change" the origional code.... a lesson well
learned from the days of spagetti design...

i hope this helped

- perry


Jole wrote:[color=blue]
> Hi
> I'm writing a program that needs to read from a file. In order for the
> program to be robust, it should somehow check that the file isn't corrupt,
> or stuffed in any way. For example, that file may have been created but a
> crash occurred at that point in time (while it was being created), damaging
> the file. Now, my program which needs to read from this file, should first
> check that it's in good condition, and that it hasn't been stuffed up in
> any way. What is the normal way of doing this?
>
> thanks in advance
> Jole
>
> I'm using Java and am aware of some of those File.XX methods. perhaps the
> File.isReadable () methods will fail if the files have been damaged or
> corrupted? or another way ?
>
>[/color]

[Silvio Bierman]

Re: detecting corrupt/stuffed files (how to?)


"perry" <yeehaw_99@yaho o.com> wrote in message
news:%9Loc.2790 5$dr1.767493@ne ws20.bellglobal .com...[color=blue]
> well, it's really not necessary to go thru the trouble of setting up a
> isReadable() method... in fact, from what you have described, i'd be
> inclined to advise against it, unless you want to maintain static /
> post-mortem data...
>
> for you see, the java i/o libraries have all the "isReadable "
> functionality you could ever imagine. and should a file be altered or
> damaged or never properly written in the first place, java is more than
> capable of detecting something wrong depending on the level of
> sensitivity you are interested in.
>
> for simplicity in your design, you should store and retrieve data as
> objects as a preference with any object oriented langauge. unless your
> application is importing or exporting data this ought to be the default.
> the reason being due to the excellent design put into saving objects as
> well as detecting any changes to the object image on the restore.
>
> now by objects, i am referring to the Object streaming classes and not
> so much a vendor of object databases.
>
> now for whatever reason you should be interested in investigating a
> corrupted object stream, you have the flexibility of opening the same
> file/stream with a lower level protocol, say a data stream. this will
> allow you to examine the bytes without throwing higher level exceptions
>
> finally, the exception mechanism became a main stream procedure in the
> object design community for a reason.... namely simplicity in design
> with respect to error management. the phylosophy of exceptions is that
> you are separating error detection from error handling. this was a major
> breakthrew in code design as it allowed developers of commercial
> frameworks (or simply previous projects) to identify errors and allow
> future programmers to be able to handle future error handling needs
> without out having to "change" the origional code.... a lesson well
> learned from the days of spagetti design...
>
> i hope this helped
>
> - perry
>
>
> Jole wrote:[color=green]
> > Hi
> > I'm writing a program that needs to read from a file. In order for the
> > program to be robust, it should somehow check that the file isn't[/color][/color]
corrupt,[color=blue][color=green]
> > or stuffed in any way. For example, that file may have been created but[/color][/color]
a[color=blue][color=green]
> > crash occurred at that point in time (while it was being created),[/color][/color]
damaging[color=blue][color=green]
> > the file. Now, my program which needs to read from this file, should[/color][/color]
first[color=blue][color=green]
> > check that it's in good condition, and that it hasn't been stuffed up in
> > any way. What is the normal way of doing this?
> >
> > thanks in advance
> > Jole
> >
> > I'm using Java and am aware of some of those File.XX methods. perhaps[/color][/color]
the[color=blue][color=green]
> > File.isReadable () methods will fail if the files have been damaged or
> > corrupted? or another way ?
> >
> >[/color]
>[/color]

By default you should NOT serialize objects to store information. Apart from
poor performance and terrible integration characteristics with other
programs (the main appeal of Unix was that is was very easy to combine all
programs because they could read each others output and generate output
themselves that was also readable) it creates a very tight coupling between
class versions and file formats.

Object serialization is usually propagated by "OO-purists" who dislike the
fact that data is not stored as an "object". Although it has its application
areas people should think twice before using it.

For simple data consider plain text,. for complexer data structures consider
XML and ALWAYS consider a relational database system. Most successfull
software systems of this world use a combination of these.

Silvio Bierman

[John Harlow]

Re: detecting corrupt/stuffed files (how to?)

> For simple data consider plain text,. for complexer data structures[color=blue]
> consider XML and ALWAYS consider a relational database system. Most
> successfull software systems of this world use a combination of these.[/color]

GREAT point. You will never "lock yourself out" of your own system if you
can get in it through notepad!