Thinking about saving error logs in web application based on web-work(or struts2)

You may  meet java errors every day if you are a java developer. Especially if a lot of or critical error logs in web application get to meet, you will be panic. So, there is need to develop error log DB and error-collectiong log module.

I will introduce my idea about developing error log DB.

According to your company policies, methods are applied for error logging system.

Architecture of error logging system consists of

1) Error DB can be stored

2) Admin web application can show errors on error db

3) Java classes and log4j configuration to transfer logs to DB.

<Error DB>

In web application, there are many properties to save such as belows.

project-name, server-name, loginid(userid), access date-time, log level, url, form, cookie, referer, execpetionmessage 

You can specify those more concisely. For example, exceptionmessage is divided to head and body according to java.lang.Exception.

I made error log db, table. Specially I made indexing in mysql. And in mysql DB, I made a batch script to rotate log db.

#!/bin/sh

TODAY=`date -d "-1 day" +%Y%m%d`

TO_DELETE_DAY=`date -d "-30 day" +%Y%m%d`

mysql  -u userid -p password mysql db-schema-name << !


# rotate log table 
alter table lgt_logdata rename logdata_${TODAY}; 


drop table if  exists logdata_${TO_DELETE_DAY};

create table logdata (

  logid int(11) NOT NULL auto_increment,

  projectname varchar(30) ,

  servername varchar(20) ,
....
  KEY lgt_logdata_ix1 (projectname,loglevel)
)

<Error-collectiong>

Error-collecting log module is implemented by log4j in simple. You can change the properites to save several log-level like error, warning. So, you can make warning log DB as well.

Let us suppose to save the errors to mysql. 

<Log4j.xml>

<appender name="ERROR-OUT" class="org.apache.log4j.jdbcplus.JDBCAppender">

<param name="Threshold" value="ERROR"/>

<param name="dbclass" value="com.mysql.jdbc.Driver"/>

<param name="url" value="jdbc:mysql://in.log.google.com"/>

<param name="username" value="error"/> 

<param name="password" value="nobodynobodywantyou"/>

<param name="sqlhandler" value="com.google.kr.MySQLHandler"/>

<param name="Buffer" value="1"/>

</appender>

<logger name="com" additivity="false">

<level value="DEBUG"/>

            <appender-ref ref="ERROR-OUT"/>

</logger>

You have to save error-loging policy in your web application.

I think every logs have to store error log DB, so, I have policy to save all tomcat logs. Because every any kinds of errors can affect the performance of web application.

I have three policies to store the DB. I think those are very important to design error DB.

1. Using Interceptor in web work or struts2, web data such as url, form have to save MDC. (MDC is api of log4j framework)

Before web application meet java exceptions, the interceptor save the web information and after invoking action it checks the result of invoking action are failed(or throw exception). If the action throws the exception, those errors will sqlhandler (ERROR-OUT)

MDC.put("projectname", projectname);
MDC.put("cookie",cookie);

2. Use Servlet in web.xml. 

You may extend and implements extends javax.servlet.http.HttpServlet class. Through the implemented class to handle error, you can store errors.

<servlet>

<servlet-name>errorHandler</servlet-name>

<servlet-class>com.google.kr.ErrorHandlerServlet</servlet-class>

</servlet>

3. Adding some fuctions to sqlhandler.
Most of error cases occur in calling action from users, but in some case internal web application invoker can make errors. For example batch or cache or scheduling based on Thread could make fault, but belows method can not include saving error logs.
And then, when you implement sqlhandler, you can add those.
You've got to adds some exceptional case on the sqlhandler.

import org.apache.log4j.jdbcplus.JDBCSqlHandler;

public class ErrorHandler implements JDBCSqlHandler {
...
}

<web applicatiion for error log>

You can show the errors stored in DB in simple.