Exception Logging Strategy

[Cramer]

I plan to implement an exception logging feature in an ASP.NET Web
application that writes encountered exceptions to disk. The exception data
will be stored as XML.

I am planning on having each exception written to its own XML file. I plan
to NOT append exceptions to a single XML file because I don't want for the
exception logging component to have to deal with contention for access to
the file. This is possible when, for example, multiple sessions of the Web
application are encountering the exception at the same time... each would
want access to the file. So rather than having "the file" (one file), I'm
thinking that it might be good to have the exception logging component keep
track of which exceptions have already been logged (within a sliding window)
and log only the exceptions that have not yet been logged. This would
prevent excessive logging of the exact same exception.

Thoughts?

I can't be the first person to come up with a robust exception logging
capability for ASP.NET Web applications - in which thousands of sessions
could encounter the exact same exception simultaneously (or within seconds
of each other).

Any suggestions to help me not "re-invent the wheel" on this would be much
appreciated.

-Cramer

[OHM \( One Handed Man \)]

Re: Exception Logging Strategy

Well my first question would be why on earth store exceptions in an xml data
file when you can use SQL ( or other ) to store it in, that way you wont
have any contentions to deal with and you can use reporting services or
crystal reports to report errors etc.

You can categorize the errors under things like, Security, Data, Validation
etc and have severitiy codes, etc. If you use reflection you can also store
the class and member function which contains the line which generated the
exception.

I also dont understand why you would not want to log exceptions which had
already been logged, they would be seperate events and could have been
caused for different reasons. When you are trying to diagnose problems in
your applications, you need as much information as possible to help
establish a route to a rapid remedy.

Thats my two euros worth anyway.

HTH




"Cramer" <A@B.comwrote in message
news:Os%2302JKq IHA.1772@TK2MSF TNGP03.phx.gbl. ..

>I plan to implement an exception logging feature in an ASP.NET Web
>application that writes encountered exceptions to disk. The exception data
>will be stored as XML.
>
I am planning on having each exception written to its own XML file. I plan
to NOT append exceptions to a single XML file because I don't want for the
exception logging component to have to deal with contention for access to
the file. This is possible when, for example, multiple sessions of the Web
application are encountering the exception at the same time... each would
want access to the file. So rather than having "the file" (one file), I'm
thinking that it might be good to have the exception logging component
keep track of which exceptions have already been logged (within a sliding
window) and log only the exceptions that have not yet been logged. This
would prevent excessive logging of the exact same exception.
>
Thoughts?
>
I can't be the first person to come up with a robust exception logging
capability for ASP.NET Web applications - in which thousands of sessions
could encounter the exact same exception simultaneously (or within seconds
of each other).
>
Any suggestions to help me not "re-invent the wheel" on this would be much
appreciated.
>
-Cramer
>
>

[Cramer]

Re: Exception Logging Strategy


OHM ( One Handed Man )" <me@myplace.com wrote :

Well my first question would be why on earth store exceptions in an xml
data file when you can use SQL ( or other ) to store it in, that way you
wont have any contentions to deal with and you can use reporting services
or crystal reports to report errors etc.

You assume that SQL Server is available. What would you do if connectivity
to the SQL Server was broken? Oh, let me guess - you'd write the exception
data to the local disk. At this point you can store it in a plain text file,
system log, or XML file. I'm going with XML for reasons that should be
obvious compared to the alternatives.

It's not relevant to the OP, but my plan is to have a separate process
periodically transfer the exception XML data to the SQL Server. So I agree
with the benefits you point out, but I completely disagree that the
exceptions should be logged directly to the SQL Server.

You can categorize the errors under things like, Security, Data,
Validation etc and have severitiy codes, etc. If you use reflection you
can also store the class and member function which contains the line
which generated the exception.

What? You seem to think that if I were to store exception data in an XML
file that it couldn't include all the detail you talk about.

The storage medium has nothing to do with whether I can use Reflection, log
security, have severity codes, etc.

I also dont understand why you would not want to log exceptions which had
already been logged,

Consider that this is an ASP.NET Web application that could have thousands
of concurrent users. Suppose it connects to a SQL Server database to get
data to display on pages. The connectivity to the SQL Server gets broken. I
don't need or want thousands of duplicate errors getting logged - regardless
of the storage medium.

Note that in my IOP I referred to a sliding window. That could be minutes to
hours. Tell me about a broken SQL Server connection once. I don't need to
know about the many different features that consequently broke and reported
an exception. The sliding window lets me know about the first that happens
and not the subsequent thousands. BTW, this sliding window only impacts
what's written to the XML files. The component that manages the sliding
window could keep track of the total number of exceptions NOT logged, so I
would still know the overall number of times the exception was encountered.

>they would be seperate events and could have been caused for different
>reasons.

Then, by definition, and my exception handling model, those are different
exceptions - each of which gets logged (once per sliding window).

>When you are trying to diagnose problems in your applications, you need as
>much information as possible to help establish a route to a rapid remedy.

Yes - but you also want a good signal-to-noise ratio. It's simply not
helpful to have redundant information redundant information redundant
information redundant information redundant information redundant
information redundant information redundant information redundant
information redundant information redundant information redundant
information redundant information redundant information redundant
information redundant information redundant information redundant
information redundant information redundant information redundant
information redundant information.

right?

[=?ISO-8859-1?Q?Arne_Vajh=F8j?=]

Re: Exception Logging Strategy

Cramer wrote:

I plan to implement an exception logging feature in an ASP.NET Web
application that writes encountered exceptions to disk. The exception data
will be stored as XML.
>
I am planning on having each exception written to its own XML file. I plan
to NOT append exceptions to a single XML file because I don't want for the
exception logging component to have to deal with contention for access to
the file. This is possible when, for example, multiple sessions of the Web
application are encountering the exception at the same time... each would
want access to the file. So rather than having "the file" (one file), I'm
thinking that it might be good to have the exception logging component keep
track of which exceptions have already been logged (within a sliding window)
and log only the exceptions that have not yet been logged. This would
prevent excessive logging of the exact same exception.
>
Thoughts?
>
I can't be the first person to come up with a robust exception logging
capability for ASP.NET Web applications - in which thousands of sessions
could encounter the exact same exception simultaneously (or within seconds
of each other).
>
Any suggestions to help me not "re-invent the wheel" on this would be much
appreciated.

log4net has a robust logging capability.

I don't like the multiple file approach.

Single file, database, syslog whatever would be better.

The synchronization overhead should not be a problem. Actually I would
expect the creating many files to have more overhead than synchronizing
writing to a single file.

I would log all exceptions. In case you want to figure out what happen,
then it is nice to have everything and not try to avoid redundancy.

Arne

[=?ISO-8859-1?Q?Arne_Vajh=F8j?=]

Re: Exception Logging Strategy

Cramer wrote:

OHM ( One Handed Man )" <me@myplace.com wrote :

>Well my first question would be why on earth store exceptions in an xml
>data file when you can use SQL ( or other ) to store it in, that way you
>wont have any contentions to deal with and you can use reporting services
>or crystal reports to report errors etc.

>
You assume that SQL Server is available. What would you do if connectivity
to the SQL Server was broken? Oh, let me guess - you'd write the exception
data to the local disk. At this point you can store it in a plain text file,
system log, or XML file. I'm going with XML for reasons that should be
obvious compared to the alternatives.

For many applications if the database is down the entire app is
completely down until the database comes up again.

And usually it is not really that useful to have logged the
exceptions telling that the database was down.

It's not relevant to the OP, but my plan is to have a separate process
periodically transfer the exception XML data to the SQL Server. So I agree
with the benefits you point out, but I completely disagree that the
exceptions should be logged directly to the SQL Server.

It has been done many times before.

It can be done again.

>I also dont understand why you would not want to log exceptions which had
>already been logged,

>
Consider that this is an ASP.NET Web application that could have thousands
of concurrent users. Suppose it connects to a SQL Server database to get
data to display on pages. The connectivity to the SQL Server gets broken. I
don't need or want thousands of duplicate errors getting logged - regardless
of the storage medium.
>
Note that in my IOP I referred to a sliding window. That could be minutes to
hours. Tell me about a broken SQL Server connection once. I don't need to
know about the many different features that consequently broke and reported
an exception. The sliding window lets me know about the first that happens
and not the subsequent thousands. BTW, this sliding window only impacts
what's written to the XML files. The component that manages the sliding
window could keep track of the total number of exceptions NOT logged, so I
would still know the overall number of times the exception was encountered.
>

>they would be seperate events and could have been caused for different
>reasons.

>
Then, by definition, and my exception handling model, those are different
exceptions - each of which gets logged (once per sliding window).
>

>When you are trying to diagnose problems in your applications, you need as
>much information as possible to help establish a route to a rapid remedy.

>
Yes - but you also want a good signal-to-noise ratio. It's simply not
helpful to have redundant information redundant information redundant
information redundant information redundant information redundant

I am still a bit skeptical about whether you will be able to code
logic that correctly will be able to guess whether an exception is
sufficient closely related to another exception, that the troubleshooter
will not be interested.

And disk space is rather cheap.

I would rather log MB's of stuff that is never needed than one day
sit there and "I wish I had ...".

Arne

[Cramer]

Re: Exception Logging Strategy

>You assume that SQL Server is available. What would you do if

>connectivity to the SQL Server was broken? Oh, let me guess - you'd write
>the exception data to the local disk.

>
What is it with you and broken SQL Servers, are they allways broken in
your
part of the galaxy, and why cant you have an sql server instance on your
host machine ?

Just because we *can* install SQL Server on a Web server doesn't mean we
*should*. In my case I can, but I don't want to have a busy Web server doing
any important tasks that have absolutely nothing to do with serving Web
pages.

And, in my part of the galaxy I don't always have broken or unreliable
connectivity to SQL Server. But, like for many systems, when the
connectivity is gone, that's a major problem that I want to know about as
soon as it happens.

BTW: I'm not attacking you - but I will point out bad ideas when I see
them... ideas like installing SQL Server on a Web server just for the sake
of logging exceptions (or even worse, doing more than logging exceptions). I
understand that this is a forum for freely exchanging ideas. It's also a
forum for critiquing ideas and pointing out both the good and the bad. You
don't need to be offended if one of your ideas is found to be less than
useful (like your most recent implication to install SQL Server on a Web
server. Nobody who knows what they are doing would ever recommend doing
that).

[=?ISO-8859-1?Q?Arne_Vajh=F8j?=]

Re: Exception Logging Strategy

Cramer wrote:

BTW: I'm not attacking you - but I will point out bad ideas when I see
them... ideas like installing SQL Server on a Web server just for the sake
of logging exceptions (or even worse, doing more than logging exceptions).

That is fair.

Nobody who knows what they are doing would ever recommend doing
that).

But this is more than pointing out it is a bad idea. This *is* a
personal attack.

Arne

[Alvin Bruney [ASP.NET MVP]]

Re: Exception Logging Strategy

the functionality is already implemented in detail in the Enterprise
Library. All the scenarios described in this thread are covered by design
time policy. I note here that this has to be coded into the logic if you
aren't using EL. Why re-invent the wheel? For EL, code does not decide where
to log, policy determines that and policy is set by the maintenance folk not
the programmer. Have a look at EL, I see it as your best option with very
little coding.

"Cramer" <A@B.comwrote in message
news:ev8vI3LqIH A.2068@TK2MSFTN GP05.phx.gbl...

>>You assume that SQL Server is available. What would you do if
>>connectivit y to the SQL Server was broken? Oh, let me guess - you'd
>>write the exception data to the local disk.

>>
>What is it with you and broken SQL Servers, are they allways broken in
>your
>part of the galaxy, and why cant you have an sql server instance on your
>host machine ?

>
Just because we *can* install SQL Server on a Web server doesn't mean we
*should*. In my case I can, but I don't want to have a busy Web server
doing any important tasks that have absolutely nothing to do with serving
Web pages.
>
And, in my part of the galaxy I don't always have broken or unreliable
connectivity to SQL Server. But, like for many systems, when the
connectivity is gone, that's a major problem that I want to know about as
soon as it happens.
>
BTW: I'm not attacking you - but I will point out bad ideas when I see
them... ideas like installing SQL Server on a Web server just for the sake
of logging exceptions (or even worse, doing more than logging exceptions).
I understand that this is a forum for freely exchanging ideas. It's also a
forum for critiquing ideas and pointing out both the good and the bad. You
don't need to be offended if one of your ideas is found to be less than
useful (like your most recent implication to install SQL Server on a Web
server. Nobody who knows what they are doing would ever recommend doing
that).
>
>

[=?Utf-8?B?UGV0ZXIgQnJvbWJlcmcgW0MjIE1WUF0=?=]

RE: Exception Logging Strategy

I really have to agree with some of the other posters here. In almost all
cases, if you are unable to write an exception log record to your database,
the rest of your application is already hosed anyway. I think what you're
suggesting represents a bit of "overkill", and is unlikely to buy you much
more than some extra headaches due to unneccessary complexity in your app.
-- Peter
To be a success, arm yourself with the tools you need and learn how to use
them.

Site: http://www.eggheadcafe.com

http://petesbloggerama.blogspot.com

http://ittyurl.net

"Cramer" wrote:

I plan to implement an exception logging feature in an ASP.NET Web
application that writes encountered exceptions to disk. The exception data
will be stored as XML.
>
I am planning on having each exception written to its own XML file. I plan
to NOT append exceptions to a single XML file because I don't want for the
exception logging component to have to deal with contention for access to
the file. This is possible when, for example, multiple sessions of the Web
application are encountering the exception at the same time... each would
want access to the file. So rather than having "the file" (one file), I'm
thinking that it might be good to have the exception logging component keep
track of which exceptions have already been logged (within a sliding window)
and log only the exceptions that have not yet been logged. This would
prevent excessive logging of the exact same exception.
>
Thoughts?
>
I can't be the first person to come up with a robust exception logging
capability for ASP.NET Web applications - in which thousands of sessions
could encounter the exact same exception simultaneously (or within seconds
of each other).
>
Any suggestions to help me not "re-invent the wheel" on this would be much
appreciated.
>
-Cramer
>
>
>
>

[Cramer]

Re: Exception Logging Strategy

"Peter Bromberg [C# MVP]" wrote :

>I really have to agree with some of the other posters here. In almost all
cases, if you are unable to write an exception log record to your
database,
the rest of your application is already hosed anyway.

Yes - but the implication then becomes that we don't log the fact that the
exception occurred at all ("we can't connect to the SQL Server, so don't log
the exception").

So then all we have to go on is knowledge that the application is hosed,
with no error log to look at as to why it's hosed, or what went wrong just
prior to connectivity going awry.

In my case I have a Windows server that looks for the XML exceptions. When
connectivity to the database is lost (and my app is hosed as you accurately
summed), then the Windows service tells me right away (sends a text message
to my cell phone). This happened only once in the last 3 years - but I had
the Web server up and running in a matter of minutes - with zero calls to
tech support. I doubt any of my customers even knew their sites were
offline.

>I think what you're
suggesting represents a bit of "overkill", and is unlikely to buy you much
more than some extra headaches due to unneccessary complexity in your app.

Given the above production support scenario (which actually happened) I
don't consider it to be overkill. Maybe it would be if this were a hobby Web
site or I could afford to let angry customers tell me when their Web sites
are down. In my case I'm trying to be proactive.

I'm still wondering if anyone else who supports ASP.NET Web applications
does anything even remotely similar to what I'm doing for logging - or if
you all just write directly to SQL Server and therefore knowingly losing
knowledge of all exceptions that occur when the SQL Server is unavailable.

[Daniel Ruehle]

Re: Exception Logging Strategy

Simple answer, yes, there are others that do what your end goal seems to be.
Just maybe not the way you are proposing.

I typically don't use logging to determine if a system is down. Logs are
useful for figuring out what happened, not what is happening. I always use
monitoring to ensure that applications are running. Simply build test case
handlers that exercise key functions of your system. Connect to the
database, log a test user in, perform a query, run a report, etc. Assuming
all goes well, provide an appropriate response. There are many monitoring
tools out there that can do this easily. This method has the added benefit
that you may know about an outage before any clients receive an error where
logging only informs you when a problem occurs.

That said, log4net is an easy way to consolidate logs. You can configure it
live to write logs to files, database tables, web services or emails if you
like to get spammed. It takes care of all of the contention and can even be
configured to work correctly to write to files in a web garden. Of course
EL can as well, but it takes more setup and configuration. What I usually
do is make database connection exceptions cause Fatal level log entries
which I configure to send an email (to me or hopefully a support person) so
if the monitor doesn't catch it, someone will be notified immediately, and
no additional Windows service is required to monitor log files.

"Cramer" <A@B.comwrote in message
news:%236nKe8Mq IHA.524@TK2MSFT NGP05.phx.gbl.. .

"Peter Bromberg [C# MVP]" wrote :

>>I really have to agree with some of the other posters here. In almost all
>cases, if you are unable to write an exception log record to your
>database,
>the rest of your application is already hosed anyway.

>
Yes - but the implication then becomes that we don't log the fact that the
exception occurred at all ("we can't connect to the SQL Server, so don't
log the exception").
>
So then all we have to go on is knowledge that the application is hosed,
with no error log to look at as to why it's hosed, or what went wrong just
prior to connectivity going awry.
>
In my case I have a Windows server that looks for the XML exceptions. When
connectivity to the database is lost (and my app is hosed as you
accurately summed), then the Windows service tells me right away (sends a
text message to my cell phone). This happened only once in the last 3
years - but I had the Web server up and running in a matter of minutes -
with zero calls to tech support. I doubt any of my customers even knew
their sites were offline.
>

>>I think what you're
>suggesting represents a bit of "overkill", and is unlikely to buy you
>much
>more than some extra headaches due to unneccessary complexity in your
>app.

>
Given the above production support scenario (which actually happened) I
don't consider it to be overkill. Maybe it would be if this were a hobby
Web site or I could afford to let angry customers tell me when their Web
sites are down. In my case I'm trying to be proactive.
>
I'm still wondering if anyone else who supports ASP.NET Web applications
does anything even remotely similar to what I'm doing for logging - or if
you all just write directly to SQL Server and therefore knowingly losing
knowledge of all exceptions that occur when the SQL Server is unavailable.
>
>
>

[Alexey Smirnov]

Re: Exception Logging Strategy

On Apr 27, 10:08 pm, "Cramer" <A...@B.comwrot e:

Any suggestions to help me not "re-invent the wheel" on this would be much
appreciated.

Look at the ELMAH project, it does logging to the SQL Server database

http://www.google.com/search?hl=en&q=elmah

[Microsoft Newsserver]

Re: Exception Logging Strategy

Exactly.

But for the record, I disagree that this is necessarily a bad idea, it
really depends on the circumstances. In my opinion, there are seldom bad
ideas, just ones which form a better solution better than others depending
on ones criteria.

You quote it as a "Busy Server" but where is you impirical evidence to
support the fact that adding an SQL instance would have any noticable impact
on performance. 'error' logging by its nature tends not to be an intensive
operation, if it did, then you have major issues, and in such, the
performance of the application is likely to be secondary to the faults which
precipitated any onslaught of the same.

If I revisit the OP, then it is clear that most of the detail related to
this has come out as a result of discussion rather than having been
presented as a cogent argument for your present strategy in the first
instance; although no doubt you will counterpoint that assertion.

You (OP) need in my opinion to be more objective and less emotional about
solving the issue. That way, having removed the emotion and having had
reasoned discussions with members of this forum, some concensus may drop out
and lead you to the most favourable solution.





"Arne Vajhøj" <arne@vajhoej.d kwrote in message
news:481510f2$0 $90269$14726298 @news.sunsite.d k...

Cramer wrote:

>BTW: I'm not attacking you - but I will point out bad ideas when I see
>them... ideas like installing SQL Server on a Web server just for the
>sake of logging exceptions (or even worse, doing more than logging
>exceptions).

>
That is fair.
>

Nobody who knows what they are doing would ever recommend doing
>that).

>
But this is more than pointing out it is a bad idea. This *is* a
personal attack.
>
Arne

[Patrice]

Re: Exception Logging Strategy

If ASP.NET 2.0 you may want to check the health management section in the
doc that already provides logging capabilities (including buffering events
for example to send them in a single message after a dealy or a number of
exceptions whatever comes first).

--
Patrice

"Cramer" <A@B.coma écrit dans le message de groupe de discussion :
Os#02JKqIHA.177 2@TK2MSFTNGP03. phx.gbl...

I plan to implement an exception logging feature in an ASP.NET Web
application that writes encountered exceptions to disk. The exception data
will be stored as XML.
>
I am planning on having each exception written to its own XML file. I plan
to NOT append exceptions to a single XML file because I don't want for the
exception logging component to have to deal with contention for access to
the file. This is possible when, for example, multiple sessions of the Web
application are encountering the exception at the same time... each would
want access to the file. So rather than having "the file" (one file), I'm
thinking that it might be good to have the exception logging component
keep track of which exceptions have already been logged (within a sliding
window) and log only the exceptions that have not yet been logged. This
would prevent excessive logging of the exact same exception.
>
Thoughts?
>
I can't be the first person to come up with a robust exception logging
capability for ASP.NET Web applications - in which thousands of sessions
could encounter the exact same exception simultaneously (or within seconds
of each other).
>
Any suggestions to help me not "re-invent the wheel" on this would be much
appreciated.
>
-Cramer
>
>

[=?ISO-8859-1?Q?Arne_Vajh=F8j?=]

Re: Exception Logging Strategy

Cramer wrote:

"Peter Bromberg [C# MVP]" wrote :

>I really have to agree with some of the other posters here. In almost all
>cases, if you are unable to write an exception log record to your
>database,
>the rest of your application is already hosed anyway.

>
Yes - but the implication then becomes that we don't log the fact that the
exception occurred at all ("we can't connect to the SQL Server, so don't log
the exception").
>
So then all we have to go on is knowledge that the application is hosed,
with no error log to look at as to why it's hosed, or what went wrong just
prior to connectivity going awry.
>
In my case I have a Windows server that looks for the XML exceptions. When
connectivity to the database is lost (and my app is hosed as you accurately
summed), then the Windows service tells me right away (sends a text message
to my cell phone). This happened only once in the last 3 years - but I had
the Web server up and running in a matter of minutes - with zero calls to
tech support. I doubt any of my customers even knew their sites were
offline.
>

>I think what you're
>suggesting represents a bit of "overkill", and is unlikely to buy you much
>more than some extra headaches due to unneccessary complexity in your app.

>
Given the above production support scenario (which actually happened) I
don't consider it to be overkill. Maybe it would be if this were a hobby Web
site or I could afford to let angry customers tell me when their Web sites
are down. In my case I'm trying to be proactive.
>
I'm still wondering if anyone else who supports ASP.NET Web applications
does anything even remotely similar to what I'm doing for logging - or if
you all just write directly to SQL Server and therefore knowingly losing
knowledge of all exceptions that occur when the SQL Server is unavailable.

Application logging is for finding problems in the application.

To detect whether your SQLServer is running or not you should not use
your applications log file.

You should monitor the SQLServer directly for that.

Arne