Jad 사용법

general java 2009. 4. 10. 00:23
1. jad 사용법
특정 디렉토리에 있는 모든 클래스 파일을 모두 java 소스로 변경한다.
D:\javasrc\ojdbc14>jad -r -d src -sjava oracle\**\*.class

2. 클래스 파일 자바로 보기
클래스 파일 더블클릭하면 자동으로 노트패드로 보여주고, 노트패드 종료하면 jad 파일 삭제한다.
1) jad를 다운받고, jad.exe가 실행이 되도록 그 디렉토리를 path에 추가한다.

2) 그 디렉토리에 jad.bat를 아래와 같이 생성한다.

@echo off
jad -o -d. %*
notepad %~n1.jad
del %~n1.jad

3) class 파일들의 속성에 위의 jad.bat 프로그램을 연동시킬 수 있도록 한다. class 파일에 대해서 더블클릭하면 자동으로 노트패드가 뜨는 것을 확인할 수 있다.
 


'general java' 카테고리의 다른 글

java sources for ojdbc  (0) 2009.04.17
중요한 DBCP 설정 공부  (0) 2009.04.10
Jad 사용법  (0) 2009.04.10
JMX 개발중 POJO를 이용하기  (0) 2009.03.31
jdk 5,6 클래스를 jdk 3,4에서 실행하기  (0) 2009.03.30
JMX에서의 Internalization  (0) 2009.03.25
Posted by 김용환 '김용환'

댓글을 달아 주세요


JMX상에서 POJO를 Standard MBean 또는 MXBean으로 만드는 법

 출처 : 
http://blogs.sun.com/jmxetc/entry/dynamicmbeans,_modelmbeans,_and_pojos...



The JMX forum on SDN is a lively place. Recently, Athahar was trying to expose a POJO resource through JMX - and quite naturally, decided to try out doing this with a ModelMBean. However, ModelMBeans are not the only way to this - and here are a few thoughts, which borrow heavily on some of Eamonn's most recent blog entries.

Turning your POJO into a Standard MBean or an MXBean

This is the traditional way of exposing resources using the JMX API: have the Java Object you want to expose implement an interface which groups all the attributes (getters and setters) and operations (methods) you want to expose for management.

If you are using JDK 6, then tag this interface as MXBean
If you are using earlier versions of the JDK, then name this interface by appending "MBean" to your object's java class name - or wrap your object in a StandardMBean
Then at runtime, register your object(s) in the (platformMBeanServer.

The platform MBeanServer is the MBeanServer that is used by the JVM default Management Agent, so if you register your MBeans in there, you will be able to see them directly when connecting JConsole to the default JVM agent.

This is the easiest way to proceed, and often the best way. Having an interface makes it possible to create MBean proxies, and to code remote clients against that interface. Using MXBean ensure that remote clients which do not have access to your interface/classes will still be able to interact with your MBeans. The MBean (or MXBean) interface makes it also possible to separate those method that are suited for remote invocation from those that should only be called locally, from your application.

However, there can be some cases where such a method is not possible. For instance, if the interface of your MBean is not known at compile time (rare but possible), or if the object you want to manage is a third party object, obtained e.g. from a legacy application whose source code cannot be modified. In those cases, you may want to turn to the other methods listed below.

Using a Model MBean

Model MBeans were designed to dynamically map some java object resource to a JMX Model specified at runtime. In theory, Model MBeans could be a perfect match for what we are attempting to do here. In practice, you might find that Model MBeans with their very powerful list of configurable features are a bit of an over kill. But should you wish to create and configure a Model MBean to expose your Java resource, here is an example of code that creates a trivial Model MBean to exposes all getter, setters, and operations of a simple POJO. The method makeModelMBean(Object resource) shown below introspect the given resource in order to configure aRequiredModelMBean that will expose that resource. This code example outline a few details which may not be intuitive:

  1. All getters and setters must also be declared as operations.
  2. When creating a ModelMBeanAttributeInfo object you need to explicitely specify, for that attribute, the names of the getter and setter methods in the Descriptor, even when you pass those methods to the attribute info constructor.

    


    public static ModelMBean makeModelMBean(Object resource)
    throws JMException, InvalidTargetObjectTypeException {
        
        final Method[] methods = resource.getClass().getMethods();
        
        final List<Method> operations = new ArrayList<Method>();
        final List<Method> getters = new ArrayList<Method>();
        final Map<String,Method> setters = new LinkedHashMap<String,Method>();
        
        for (Method method : methods) {
            // don't want to expose getClass(), hashCode(), equals(), etc...
            if (method.getDeclaringClass().equals(Object.class)) continue;
            
            if (method.getName().startsWith("get") &&
                    !method.getName().equals("get") &&
                    !method.getName().equals("getClass") &&
                    method.getParameterTypes().length == 0 &&
                    method.getReturnType() != void.class) {
                getters.add(method);
            } 
            if (method.getName().startsWith("set") &&
                    !method.getName().equals("set") &&
                    method.getParameterTypes().length == 1 &&
                    method.getReturnType().equals(void.class)) {
                setters.put(method.getName(),method);
            }

            operations.add(method);
        }
        
        
        final List<ModelMBeanAttributeInfo> attrinfo = 
                new ArrayList<ModelMBeanAttributeInfo>();

        for (Method getter:getters) {            
            
            final String attrName = getter.getName().substring(3);
            final String setterMethod = "set"+attrName ; // construct setter method
            
            // Check whether there's a setter and if so removes it from the
            // setter's map.
            //
            Method setter = setters.remove(setterMethod);
            
            if (setter != null) {
                // If there's a setter, it must have the same "type" than 
                // the getter
                if (!getter.getReturnType().equals(
                        setter.getParameterTypes()[0])) {
                    System.err.println("Warning: setter "+setter.getName()+
                            " doesn't have the expected type: setter ignored.");
                    setter = null;
                }
            }
            
            attrinfo.add( makeAttribute(getter,setter));
        }
             
        // check if there are setters for which there was no getter
        //
        for (Method setter:setters.values()) {
            // It would be unusual to have a setter with no getter! 
            System.err.println("Warning: setter "+setter.getName()+
                    " has no corresponding getter!");
            attrinfo.add( makeAttribute(null,setter));
        }
        
        final ModelMBeanAttributeInfo[] attrs =
            attrinfo.toArray(new ModelMBeanAttributeInfo[attrinfo.size()]);
        
        final int opcount = operations.size();
        final ModelMBeanOperationInfo[] ops = 
                new ModelMBeanOperationInfo[opcount];
        for (int i=0;i<opcount;i++){
            final Method m = operations.get(i);
            ops[i] = new ModelMBeanOperationInfo(m.getName(),m);
        }
        
        
        ModelMBeanInfo mmbi =
                new ModelMBeanInfoSupport(resource.getClass().getName(),
                resource.getClass().getName(),
                attrs,
                null,  // constructors
                ops,
                null); // notifications
        ModelMBean mmb = new RequiredModelMBean(mmbi);
        mmb.setManagedResource(resource, "ObjectReference");
        return mmb;
    }

    private static ModelMBeanAttributeInfo makeAttribute(Method getter,
                                                         Method setter)
                                 throws IntrospectionException {
        final String attrName;
        if (getter != null)
            attrName = getter.getName().substring(3);
        else 
            attrName = setter.getName().substring(3);

        final List<String> descriptors = new ArrayList<String>();
        descriptors.add("name=" + attrName);
        descriptors.add("descriptorType=attribute");
        if (getter!=null) {
           descriptors.add("getMethod=" + getter.getName());
        }
        if (setter!=null) {
           descriptors.add("setMethod=" + setter.getName());
        }

        final Descriptor attrD = new DescriptorSupport(
                descriptors.toArray(new String[descriptors.size()]));
       
        return new ModelMBeanAttributeInfo(attrName, attrName, getter, setter,
                attrD);
    }

To test my Model MBean, I have created a dummy resource POJO, wrapped it in a Model MBean, and registered it in the platform MBeanServer.

    
    public static class MyPojo {
        private String foo = "foo!";
        private long weird = -1;
        private Object bad = new Long(0);
        
        // RW attribute
        public String getFoo() {return foo;}
        public void setFoo(String foo) {this.foo=foo;}
        
        // RO attribute
        public int getBar() {return 1;}
        
        // WO attribute (weird)
        public void setWeird(long weird) {this.weird=weird;}
        
        // Bad attribute: doesn't respect JavaBean patterns...
        public Object getBad() {return bad;}
        
        public void setBad(Number bad) {this.bad = bad;}
        
        // just an operation!
        public long doIt() {
            System.out.println("doIt!");
            return weird+1;
        }
    }
    
    public static void main(String[] args) throws Exception {
        
        final MyPojo obj = new MyPojo();
        final ModelMBean mbean = makeModelMBean(obj);
        ManagementFactory.getPlatformMBeanServer().
                registerMBean(mbean,new ObjectName("test:type=MyPojo"));
        System.out.println("Connect now with JConsole.");
        System.out.println("Strike <Return> to exit.");
        System.in.read();
    }

Here is how it appears in JConsole:

Using a Dynamic MBean

A lighter solution might be to write a DynamicMBean that will introspect the class of your resource object. Here is the code of aDynamicPOJOMBean that does exactly that. If your resource can emit some notifications, then you could modify this DynamicPOJOMBean to extend NotificationBroadcasterSupport - and arrange to call sendNotification() when a Notification needs to be sent, or code aDynamicPOJOEmitterMBean that extends DynamicPOJOMBean and implements NotificationEmitter, and delegates notification handling to a wrappedNotificationBroadcasterSupport object (see the Advanced JMX Example for JDK 6 for a discussion of these two notifications patterns).

The code required for our DynamicPOJOMBean is quite simple:

    
public class DynamicPOJOMBean implements DynamicMBean {
    
    /**
     * A logger for this class.
     **/
    private static final Logger LOG =
            Logger.getLogger(DynamicPOJOMBean.class.getName());
    
    final Map<String,Method> getters;
    final Map<String,Method> setters;
    final Set<Method> operations;
    final Object resource;
    final MBeanInfo info;
        
    /** Creates a new instance of DynamicPOJOMBean */
    public DynamicPOJOMBean(Object obj) {
        getters = new LinkedHashMap<String,Method>();
        setters = new LinkedHashMap<String,Method>();
        operations = new LinkedHashSet<Method>();
        resource = obj;
        try {
            info = initialize();
        } catch (IntrospectionException ex) {
            throw new IllegalArgumentException(obj.getClass().getName(),ex);
        }
    }
    private MBeanInfo initialize() throws IntrospectionException {
        final List<MBeanAttributeInfo> attributesInfo = 
                new ArrayList<MBeanAttributeInfo>();
        final List<MBeanOperationInfo> operationsInfo = 
                new ArrayList<MBeanOperationInfo>();
        final Set<String> attributesName = new HashSet<String>();
        final ArrayList<Method> ops = new ArrayList<Method>();
        for (Method m:resource.getClass().getMethods()) {
            if (m.getDeclaringClass().equals(Object.class)) continue;
            if (m.getName().startsWith("get") &&
                    !m.getName().equals("get") &&
                    !m.getName().equals("getClass") &&
                    m.getParameterTypes().length == 0 &&
                    m.getReturnType() != void.class) {
                getters.put(m.getName().substring(3),m);
            } else if (m.getName().startsWith("is") &&
                    !m.getName().equals("is") &&
                    m.getParameterTypes().length == 0 &&
                    m.getReturnType() == boolean.class) {
                getters.put(m.getName().substring(2),m);
            } else if (m.getName().startsWith("set") &&
                    !m.getName().equals("set") &&
                    m.getParameterTypes().length == 1 &&
                    m.getReturnType().equals(void.class)) {
                setters.put(m.getName().substring(3),m);
            } else {
                ops.add(m);
            }
        }
        
        attributesName.addAll(getters.keySet());
        attributesName.addAll(setters.keySet());
        
        for (String attrName : attributesName) {
            final Method get = getters.get(attrName);
            Method set = setters.get(attrName);
            if (get != null && set != null && 
                get.getReturnType() != set.getParameterTypes()[0]) {
                set = null;
                ops.add(setters.remove(attrName));
            }
            final MBeanAttributeInfo mbi = 
                    getAttributeInfo(attrName,get,set);
            if (mbi == null && get != null) {
                ops.add(getters.remove(attrName));
            }
            if (mbi == null && set != null) {
                ops.add(setters.remove(attrName));
            }
            if (mbi != null) attributesInfo.add(mbi);
        }
        
        for (Method m:ops) {
            final MBeanOperationInfo opi = getOperationInfo(m);
            if (opi == null) continue;
            operations.add(m);
            operationsInfo.add(opi);
        }
        return getMBeanInfo(resource,attributesInfo.
                toArray(new MBeanAttributeInfo[attributesInfo.size()]),
                operationsInfo.
                toArray(new MBeanOperationInfo[operationsInfo.size()]));
    }

    protected MBeanAttributeInfo getAttributeInfo(String attrName, 
            Method get, Method set) throws IntrospectionException {
        return new MBeanAttributeInfo(attrName,attrName,get,set);
    }

    protected MBeanOperationInfo getOperationInfo(Method m) {
        if (m.getDeclaringClass()==Object.class) return null;
        return new MBeanOperationInfo(m.getName(),m);
    }

    protected MBeanInfo getMBeanInfo(Object resource, 
            MBeanAttributeInfo[] attrs, MBeanOperationInfo[] ops) {
        return new MBeanInfo(resource.getClass().getName(),
                resource.getClass().getName(),attrs,null,ops,null);
    }

    public Object getAttribute(String attribute) 
        throws AttributeNotFoundException, MBeanException, ReflectionException {
        final Method get = getters.get(attribute);
        if (get == null) throw new AttributeNotFoundException(attribute);
        try {
            return get.invoke(resource);
        } catch (IllegalArgumentException ex) {
            throw new ReflectionException(ex);
        } catch (InvocationTargetException ex) {
            final Throwable cause = ex.getCause();
            if (cause instanceof Exception) 
                throw new MBeanException((Exception)cause);
            throw new RuntimeErrorException((Error)cause);
        } catch (IllegalAccessException ex) {
            throw new ReflectionException(ex);
        }
    }

    public void setAttribute(Attribute attribute) 
        throws AttributeNotFoundException, InvalidAttributeValueException, 
            MBeanException, ReflectionException {
        final Method set = setters.get(attribute);
        if (set == null) 
            throw new AttributeNotFoundException(attribute.getName());
        try {
            set.invoke(resource,attribute.getValue());
        } catch (IllegalArgumentException ex) {
            throw new ReflectionException(ex);
        } catch (InvocationTargetException ex) {
            final Throwable cause = ex.getCause();
            if (cause instanceof Exception) 
                throw new MBeanException((Exception)cause);
            throw new RuntimeErrorException((Error)cause);
        } catch (IllegalAccessException ex) {
            throw new ReflectionException(ex);
        }
    }

    public AttributeList getAttributes(String[] attributes) {
        if (attributes == null) return new AttributeList();
        final List<Attribute> result = 
                new ArrayList<Attribute>(attributes.length);
        for (String attr : attributes) {
            final Method get = getters.get(attr);
            try {
                result.add(new Attribute(attr,get.invoke(resource)));
            } catch (Exception x) {
                continue;
            }
        }
        return new AttributeList(result);
    }

    public AttributeList setAttributes(AttributeList attributes) {
        if (attributes == null) return new AttributeList();
        final List<Attribute> result = 
                new ArrayList<Attribute>(attributes.size());
        for (Object item : attributes) {
            final Attribute attr = (Attribute)item;
	    final String name = attr.getName();
            final Method set = setters.get(name);
            try {
		set.invoke(resource,attr.getValue());
		final Method get = getters.get(name);
                final Object newval = 
		    (get==null)?attr.getValue():get.invoke(resource);
                result.add(new Attribute(name,newval));
            } catch (Exception x) {
                continue;
            }
        }
        return new AttributeList(result);
    }

    public Object invoke(String actionName, Object[] params, String[] signature) 
        throws MBeanException, ReflectionException {
        Method toInvoke = null;
        if (params == null) params = new Object[0];
        if (signature == null) signature = new String[0];
        for (Method m : operations) {
            if (!m.getName().equals(actionName)) continue;
            final Class[] sig = m.getParameterTypes();
            if (sig.length == params.length) {
                if (sig.length == 0) toInvoke=m;
                else if (signature.length == sig.length) {
                    toInvoke = m;
                    for (int i=0;i<sig.length;i++) {
                        if (!sig[i].getName().equals(signature[i])) {
                            toInvoke = null; 
                            break;
                        }
                    }
                }
            }
            if (toInvoke != null) break;
        }
        if (toInvoke == null) 
            throw new ReflectionException(new NoSuchMethodException(actionName));
        try {
            return toInvoke.invoke(resource,params);
        } catch (IllegalArgumentException ex) {
            throw new ReflectionException(ex);
        } catch (InvocationTargetException ex) {
            final Throwable cause = ex.getCause();
            if (cause instanceof Exception) 
                throw new MBeanException((Exception)cause);
            throw new RuntimeErrorException((Error)cause);
        } catch (IllegalAccessException ex) {
            throw new ReflectionException(ex);
        }
    }

    public MBeanInfo getMBeanInfo() {
        return info;
    }
  
}

I have tested this DynamicPOJOMBean with the same dummy resource that I used for my ModelMBean above. Here is a JConsole window that shows both MBeans: "test:type=MyPojo,name=Model" is the previously coded ModelMBean, "test:type=MyPojo,name=dynamic" is my lighter DynamicMBean version...

Wrapping your POJO inside an MXBean Proxy

The drawback of the two methods described above is when your POJO use custom classes (other POJOs) as input/output to its getter/setter and/or operations. In that case - if you use a plain MBean to expose your POJO, you will need to:

  1. make sure that those custom classes are serializable
  2. distribute the jars containing those classes to the clients that monitor your POJO.

In that case, using an MXBean would be the best answer. We have already discussed how to turn your POJO into an MXBean, in the first section of this blog. However, this implies that you can modify the source code of your object, and make it implement an MXBean interface.

An alternative to that would be to wrap your POJO inside a java.lang.reflect.Proxy that would implement the appropriate MXBean interface. This is one of the use cases described by Eamonn in one of his recent blog entries. Here are the steps that would be required:

  1. You would need to write an MXBean interface containing all the getters, setters, and methods you want to expose.
  2. You would need to write an InvocationHandler, which when invoked on a method m defined in 1), finds out the method with the same name and signature on your object, and invokes it. The code of the InvocationHandler below is extracted from Eamon's blog: Build Your Own Interface
    The MBeanInvocationHandler used in the constructed Proxy looks like this: 
    public class MBeanInvocationHandler implements InvocationHandler {
        public MBeanInvocationHandler(Object wrapped) {
            this.wrapped = wrapped;
        }
    
        public Object invoke(Object proxy, Method method, Object[] args)
        throws Throwable {
            Class wrappedClass = wrapped.getClass();
            Method methodInWrapped =
                wrappedClass.getMethod(method.getName(), method.getParameterTypes());
            try {
                return methodInWrapped.invoke(wrapped, args);
            } catch (InvocationTargetException e) {
                throw e.getCause();
            }
        }
        
        private final Object wrapped;
    }
    
  3. Using this invocation handler, you would be able to create a Proxy implementing your MXBean interface created in 1) and forward all its methods to the wrapped Plain Old Java Object.

Provided that the name of the interface you provide ends with "MXBean", the proxy obtained will implement that MXBean interface, and is therefore itself an MXBean, that you can directly register in an MBeanServer.

Of course, in this case, you still need to write an MXBean interface for your POJO, but your POJO no longer needs to implement it - it simply needs to have the "same" methods than those defined in the interface. The code required for the InvocationHandler is completely generic, and is driven by introspection of the provided interface. If your reason for not turning your POJO into an MBean was simply that you couldn't change its codebase, then statically writing an MXBean interface for creating an MXBean proxy (or statically generating that interface by introspecting your POJO class) might be enough to suit your needs.

Using dynamic code generation

This latter case is a use case that exactly corresponds to what is described by Eamonn in his blog about generating interfaces. Instead of statically writing (or generating) the interface needed to create the MXBean Proxy for your POJO, you could dynamically generate it at runtime.

Before you start generating code however, make really sure that none of the previously described methods can be used!

Waiting for JMX 2.0?

One of the RFE considered for JMX 2.0 is Annotations to simplify MBean development. This could come in the form of e.g. @Managedannotations as suggested in Eamonn's Blog. Having such annotation would not solve our original problem, which was to expose a resource whose source code cannot be changed, but it could simplify the development of MBean wrappers, such as those discussed in this entry.

What? What time is it? So late? When I started writing this entry, I thought it would take me ten minutes! I had only these two small classes that I wanted to share! Sorry to have been so long, now I need to run!

Cheers,
-- daniel

See also: javax.management.StandardMBean: When and Why. 
Look also for other JMX related articles in this blog...
Tags:     
Posted by dfuchs ( Nov 22 2006, 09:01:43 PM CET ) Permalink Comments [3]

'general java' 카테고리의 다른 글

중요한 DBCP 설정 공부  (0) 2009.04.10
Jad 사용법  (0) 2009.04.10
JMX 개발중 POJO를 이용하기  (0) 2009.03.31
jdk 5,6 클래스를 jdk 3,4에서 실행하기  (0) 2009.03.30
JMX에서의 Internalization  (0) 2009.03.25
toString 구현을 쉽게 구현하기  (0) 2009.03.25
Posted by 김용환 '김용환'
TAG 영화

댓글을 달아 주세요

Retrotranslator(http://retrotranslator.sourceforge.net/)를 사용하면 된다.
Posted by 김용환 '김용환'

댓글을 달아 주세요

JMX 스펙리더의 글을 퍼온다.
Posted by 김용환 '김용환'
TAG 영화

댓글을 달아 주세요

 

아파치 commons의 org.apache.commons.lang.builder.ToStringBuilder 클래스는 이런 일을 위해서 만들어졌다.

 

toString을 구현할려면, 그냥 StringBuffer나 StringBuilder쓰는 삽질이 있었는데. 얘는 쓰면 참 편하다.

이젠 toString 삽질은 그만..

 

1) 그냥 쓰는 경우.

 @Override
 public String toString() {
  return ToStringBuilder.reflectionToString(this);
 }

 

 @Override
 public String toString() {
  return ToStringBuilder.reflectionToString(this, ToStringStyle.DEFAULT_STYLE);
 }

 

2) 여러줄로 나눠서 Properties를 보여주는 것 (내가 주로 사용)

 @Override
 public String toString() {
  return ToStringBuilder.reflectionToString(this, ToStringStyle.MULTI_LINE_STYLE);
 }

 

2) 한줄로 나눠서 Properties를 보여주되 클래스와 클래스의 hash값은 보여주지 않는다.

 @Override
 public String toString() {
  return ToStringBuilder.reflectionToString(this, ToStringStyle.SIMPLE_STYLE);
 }

 

Posted by 김용환 '김용환'

댓글을 달아 주세요

어느날 Transport 클래스의 send 메소드를 사용하는데, 영원히 기다리는 현상이 발견되었다.

이는 메일서버가 약간 이상해지거나. 잘못된 주소로 계속 retry하는 시도가 있으면, 마치 block되는 느낌으로 계속 있는 현상이 발생된다.

 

확인을 해보니.. Tranport 클래스의 send 메소드는 block 형 메소드였다.

 

http://forums.sun.com/thread.jspa?forumID=43&threadID=474662

 

 

자세한 내용을 확인하기 위해서 API를 살펴보았다.

http://java.sun.com/products/javamail/javadocs/com/sun/mail/smtp/package-summary.html

 

mail.smtp.connectiontimeout int Socket connection timeout value in milliseconds. Default is infinite timeout.
mail.smtp.timeout int Socket I/O timeout value in milliseconds. Default is infinite timeout.

 

 

대충 사용법은 다음과 같다..

 

 

 

  Properties props = System.getProperties();
  props.put("mail.smtp.host", conf.get("mail/smtp/host"));
  props.put("mail.debug", conf.get("mail/debug"));
  props.put("mail.transport.protocol", "smtp");
  props.put("mail.smtp.connectiontimeout", "5000");
  props.put("mail.smtp.timeout", "5000");  

 

Session session = Session.getInstance(props);

MimeMessage message = new MimeMessage(session);

message.setFrom(ia);
  
  message.setRecipient(Message.RecipientType.TO, new InternetAddress(to));
  message.setSubject(subject, encoding);
  message.setContent(text, "text/html; charset=" + encoding);

  Transport.send(message);

 

 

'general java' 카테고리의 다른 글

JMX에서의 Internalization  (0) 2009.03.25
toString 구현을 쉽게 구현하기  (0) 2009.03.25
javax.mail.Transport.send 메소드 사용시 block되는 현상  (0) 2009.03.24
sping 2.0 aop @aspect 사용 예제  (0) 2009.03.06
Hadoop 사용 예제  (0) 2009.03.04
Hadoop 설치 하기  (0) 2009.03.04
Posted by 김용환 '김용환'
TAG 영화

댓글을 달아 주세요

http://blog.daum.net/oraclejava/15192553

 

sping 2.0 aop @aspect 사용 예제

'general java' 카테고리의 다른 글

JMX에서의 Internalization  (0) 2009.03.25
toString 구현을 쉽게 구현하기  (0) 2009.03.25
javax.mail.Transport.send 메소드 사용시 block되는 현상  (0) 2009.03.24
sping 2.0 aop @aspect 사용 예제  (0) 2009.03.06
Hadoop 사용 예제  (0) 2009.03.04
Hadoop 설치 하기  (0) 2009.03.04
Posted by 김용환 '김용환'
TAG 영화

댓글을 달아 주세요

하둡을 막상 사용하려면 좀 갑갑하다. 설치 빼고는 제대로 된 문서가 별로 없다. 그래서 간단하게 내가 만든 예제를 보여드리고자 한다.

 

두서 없이 쓰긴 하는데.. 읽다보면 알 수 있을 것이다.

 


// 하둡 내부에서 output 삭제
bin/hadoop  fs -rmr output

// 하둡 내부에 jsp grep한 결과를 output 이라는 hdfs에 저장
bin/hadoop jar hadoop-*-examples.jar grep input output 'jsp'

// hdfs에 저장된 output을 로컬로 복사
bin/hadoop fs -get output1 output

 

한번 만들어진 hdfs의 output 디렉토리는 다시 쓸 수 없다. 지워야 한다..

 

 

[handev001:/share/hadoop-0.18.3] bin/hadoop
[handev001:/share/hadoop-0.18.3] bin/hadoop dfs -mkdir source
[handev001:/share/hadoop-0.18.3] cat > source.txt
input

[handev001:/share/hadoop-0.18.3] bin/hadoop dfs -put source.txt source/

 

[handev001:/share/hadoop-0.18.3] bin/hadoop jar contrib/streaming/hadoop-0.18.3-streaming.jar -input source -output "./log_result" -mapper mapper.pl -file mapper.pl -reducer reducer.pl -file reducer.pl

 

이 때 -input 프로퍼티의 값은 source는 디렉토리를 의미한다. 즉, hdfs의 root 상대경로 인 source 디렉토리릉 의미한다. 아래를 볼 것 ..

그리고, 마찬가지로 -output이다. 하지만, 펄 스크립트는 내  local 파일에 있다.

 

/share/hadoop-0.18.3] bin/hadoop fs -ls
Found 11 items
drwxr-xr-x   - hadoop supergroup          0 2009-03-03 14:45 /user/hadoop/input
drwxr-xr-x   - hadoop supergroup          0 2009-03-03 17:37 /user/hadoop/log_result
drwxr-xr-x   - hadoop supergroup          0 2009-03-03 17:39 /user/hadoop/output
drwxr-xr-x   - hadoop supergroup          0 2009-03-03 17:41 /user/hadoop/output1
drwxr-xr-x   - hadoop supergroup          0 2009-03-03 17:46 /user/hadoop/output2
drwxr-xr-x   - hadoop supergroup          0 2009-03-03 18:53 /user/hadoop/output3
drwxr-xr-x   - hadoop supergroup          0 2009-03-03 18:54 /user/hadoop/output4
drwxr-xr-x   - hadoop supergroup          0 2009-03-03 18:58 /user/hadoop/output5
drwxr-xr-x   - hadoop supergroup          0 2009-03-04 10:11 /user/hadoop/output6
drwxr-xr-x   - hadoop supergroup          0 2009-03-04 10:26 /user/hadoop/output7
drwxr-xr-x   - hadoop supergroup          0 2009-03-03 17:43 /user/hadoop/source


 

mapper.pl

 #!/usr/bin/perl


my $line;


while(<STDIN>)  {
$line = $_;
chomp ($line);
print $line;
break;
}

#$r = `grep -r $line /user/hadoop/input`;
print "grep -r $line /user/hadoop/input ";
print $r."\n";

 

 

reducer.pl

 #!/usr/bin/perl
while (<>) {
 print;
}

 

아래를 참조하면 좋다.

 

참고하기

http://www.cs.brandeis.edu/~cs147a/lab/hadoop-troubleshooting/

http://www.cs.brandeis.edu/~cs147a/lab/hadoop-example/

 

 

계속 쓰려고 헀지만, hadoop을 중간에 안쓴 이유가 있었다.

우선 가지고 있는 파일들이 대용량이 아니라는 점, 그리고 hadoop을 써도 아주 성능상 효과를 가지지 못한 점이었다. 그것은 로컬만 썼으니까 그랬던 것 같다.

대용량 파일, 검색이 필요할 때는, 다양하게 거기에 맞는 솔류션을 쓰는 것이 많긴 한데, 아직은 나에게는 그런 일이 적어서..^^;; 하루만에 hadoop 써본 것을 정리한다.

 

또한 Hadoop의 전문가가 쓴 아래의 글도 많은 도움을 받았다.

 

http://www.jaso.co.kr/159

 

 

'general java' 카테고리의 다른 글

JMX에서의 Internalization  (0) 2009.03.25
toString 구현을 쉽게 구현하기  (0) 2009.03.25
javax.mail.Transport.send 메소드 사용시 block되는 현상  (0) 2009.03.24
sping 2.0 aop @aspect 사용 예제  (0) 2009.03.06
Hadoop 사용 예제  (0) 2009.03.04
Hadoop 설치 하기  (0) 2009.03.04
Posted by 김용환 '김용환'
TAG 영화

댓글을 달아 주세요

새로운 것 함 해볼려고 Hadoop설치했는데. 아직 나한테 잘 다가온다. 대용량 파일, 검색에는 유효하지만, 그리 많지 않다면 쓸 필요가 없다. 내가 하는 일에는 대용량 파일이 없고, 검색할 일이 없어서.. 아직 쓰지 않는다.

 

Hadoop이 무엇인지 아는 사람이 설치하러 왔으므로, 자세한 Hadoop의 이야기는 http://hadoop.apache.org/ 에서 참조하길 바란다.

 

자 이제 설치해보자, (리눅스에서 설치)

#1. 하둡을 다운로드 한다.

http://hadoop.apache.org/core/releases.html

나는 개인적으로 최신 release는 쓰지 않는다. 그 바로 전 것을 쓴다. 현재 0.19버젼이 나왔으니. 0.18버젼으로 다운로드

/share/hadoop-0.18.3 디렉토리에 다운받고 풀어준다.

 

#2. JAVA_HOME 설정

.bashrc에, export JAVA_HOME=/usr/local/jdk6

 

#3. Hadoop 개인정보 설정

groupadd users
useradd -d /hdfs/home -g users hadoop
chown -R hadoop:users /hdfs
chown -R hadoop:users /share
passwd hadoop

 

#4. hadoop 계정으로 들어

su - hadoop

 

#5 Hadoop 설정 파일

나는 로컬에서만 돌려볼 것이라서 single machine 설정이다. 만약 multiple machine에서 돌릴꺼면, 아래 참조 references를 볼 것.

/share/hadoop-0.18.3/conf] vi hadoop-site.xml

 

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>


<property>
        <name>fs.default.name</name>
        <value>hdfs://localhost:9000</value>
</property>

<property>
        <name>mapred.job.tracker</name>
        <value>localhost:9001</value>
</property>

<property>
        <name>dfs.name.dir</name>
        <value>/hdfs/filesystem/name</value>
</property>

<property>
        <name>dfs.data.dir</name>
        <value>/hdfs/filesystem/data</value>
</property>

<property>
        <name>mapred.system.dir</name>
        <value>/hdfs/filesystem/mapreduce/system</value>
</property>

<property>
        <name>mapred.local.dir</name>
        <value>/hdfs/filesystem/mapreduce/local</value>
</property>

 

</configuration>

설정은 자유다. 하지만 중요한 것이 있다. 절대로 dfs.name.dir은 mkdir로 만들지 말 것...

임의로 만들면 namenode daemon이 뜨지 않는다.

 

#6 디렉토리 생성

/share/hadoop-0.18.3]# mkdir -p /hdfs/filesystem/data
/share/hadoop-0.18.3]# mkdir -p /hdfs/filesystem/mapreduce/system
/share/hadoop-0.18.3]# mkdir -p /hdfs/filesystem/mapreduce/local
/share/hadoop-0.18.3]# mkdir -p /hdfs/filesystem/name/image

 

#7 namenode 데몬을 위핸 포맷을 한다.

/share/hadoop-0.18.3]# bin/hadoop namenode -format

주의점, 꼭 로그를 확인해서 문제가 없는지 확인한다.

logs라는 디렉토리에서 꼼꼼하게 제대로 format되는지 확인한다.

여기서 문제면 진도가 나가지 않는다.

 

#8 Hadoop 실행

/share/hadoop-0.18.3/bin/start-all.sh

 

hadoop계정으로 실행했으므로 namenode 데몬(관리자), datanode(실행노드)로 지정된 서버에 ssh 들어가게 된다. 이거 패스워드 입력하라고 하는데.. 귀찮지 않은가??

그냥 password 안넣게 아래 링크보고 하면 된다.

http://kldp.org/node/40578

 

#9 자바 데몬이 뜨는 것을 보고 아래 URL이 정상적인지 확인하다.

http://서버 ip:50060/tasktracker.jsp
http://서버 ip:50030/jobtracker.jsp
http://서버 ip:50070/dfshealth.jsp

#10. 테스트

소스를 보면 많은 도움 된다. 역시 Batch 개념은 어디서나 만난다.

 

/share/hadoop-0.18.3]  bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'
09/03/03 10:26:59 INFO mapred.FileInputFormat: Total input paths to process : 10
09/03/03 10:26:59 INFO mapred.FileInputFormat: Total input paths to process : 10
09/03/03 10:27:00 INFO mapred.JobClient: Running job: job_200903031008_0001
09/03/03 10:27:01 INFO mapred.JobClient:  map 0% reduce 0%

09/03/03 10:27:06 INFO mapred.JobClient:  map 18% reduce 0%
09/03/03 10:27:09 INFO mapred.JobClient:  map 36% reduce 0%
09/03/03 10:27:12 INFO mapred.JobClient:  map 45% reduce 0%
09/03/03 10:27:13 INFO mapred.JobClient:  map 54% reduce 0%
09/03/03 10:27:15 INFO mapred.JobClient:  map 63% reduce 0%
09/03/03 10:27:16 INFO mapred.JobClient:  map 72% reduce 0%
09/03/03 10:27:17 INFO mapred.JobClient:  map 81% reduce 0%
09/03/03 10:27:19 INFO mapred.JobClient:  map 90% reduce 0%
09/03/03 10:27:20 INFO mapred.JobClient:  map 100% reduce 0%

09/03/03 10:27:29 INFO mapred.JobClient:  map 100% reduce 9%
09/03/03 10:27:30 INFO mapred.JobClient:  map 100% reduce 12%
09/03/03 10:27:34 INFO mapred.JobClient:  map 100% reduce 18%
09/03/03 10:27:39 INFO mapred.JobClient:  map 100% reduce 27%
09/03/03 10:27:42 INFO mapred.JobClient: Job complete: job_200903031008_0001
09/03/03 10:27:42 INFO mapred.JobClient: Counters: 16
09/03/03 10:27:42 INFO mapred.JobClient:   File Systems
09/03/03 10:27:42 INFO mapred.JobClient:     HDFS bytes read=48319
09/03/03 10:27:42 INFO mapred.JobClient:     HDFS bytes written=1907
09/03/03 10:27:42 INFO mapred.JobClient:     Local bytes read=1549
09/03/03 10:27:42 INFO mapred.JobClient:     Local bytes written=3680
09/03/03 10:27:42 INFO mapred.JobClient:   Job Counters
09/03/03 10:27:42 INFO mapred.JobClient:     Launched reduce tasks=1
09/03/03 10:27:42 INFO mapred.JobClient:     Launched map tasks=11
09/03/03 10:27:42 INFO mapred.JobClient:     Data-local map tasks=11
09/03/03 10:27:42 INFO mapred.JobClient:   Map-Reduce Framework
09/03/03 10:27:42 INFO mapred.JobClient:     Reduce input groups=49
09/03/03 10:27:42 INFO mapred.JobClient:     Combine output records=100
09/03/03 10:27:42 INFO mapred.JobClient:     Map input records=1554
09/03/03 10:27:42 INFO mapred.JobClient:     Reduce output records=49
09/03/03 10:27:42 INFO mapred.JobClient:     Map output bytes=1552
09/03/03 10:27:42 INFO mapred.JobClient:     Map input bytes=47484
09/03/03 10:27:42 INFO mapred.JobClient:     Combine input records=107
09/03/03 10:27:42 INFO mapred.JobClient:     Map output records=56
09/03/03 10:27:42 INFO mapred.JobClient:     Reduce input records=49
09/03/03 10:27:42 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
09/03/03 10:27:43 INFO mapred.FileInputFormat: Total input paths to process : 1
09/03/03 10:27:43 INFO mapred.FileInputFormat: Total input paths to process : 1
09/03/03 10:27:43 INFO mapred.JobClient: Running job: job_200903031008_0002
09/03/03 10:27:44 INFO mapred.JobClient:  map 0% reduce 0%
09/03/03 10:27:48 INFO mapred.JobClient:  map 100% reduce 0%
09/03/03 10:27:51 INFO mapred.JobClient: Job complete: job_200903031008_0002
09/03/03 10:27:51 INFO mapred.JobClient: Counters: 16
09/03/03 10:27:51 INFO mapred.JobClient:   File Systems
09/03/03 10:27:51 INFO mapred.JobClient:     HDFS bytes read=1907
09/03/03 10:27:51 INFO mapred.JobClient:     HDFS bytes written=1135
09/03/03 10:27:51 INFO mapred.JobClient:     Local bytes read=1549
09/03/03 10:27:51 INFO mapred.JobClient:     Local bytes written=3134
09/03/03 10:27:51 INFO mapred.JobClient:   Job Counters
09/03/03 10:27:51 INFO mapred.JobClient:     Launched reduce tasks=1
09/03/03 10:27:51 INFO mapred.JobClient:     Launched map tasks=1
09/03/03 10:27:51 INFO mapred.JobClient:     Data-local map tasks=1
09/03/03 10:27:51 INFO mapred.JobClient:   Map-Reduce Framework
09/03/03 10:27:51 INFO mapred.JobClient:     Reduce input groups=3
09/03/03 10:27:51 INFO mapred.JobClient:     Combine output records=0
09/03/03 10:27:51 INFO mapred.JobClient:     Map input records=49
09/03/03 10:27:51 INFO mapred.JobClient:     Reduce output records=49
09/03/03 10:27:51 INFO mapred.JobClient:     Map output bytes=1429
09/03/03 10:27:51 INFO mapred.JobClient:     Map input bytes=1821
09/03/03 10:27:51 INFO mapred.JobClient:     Combine input records=0
09/03/03 10:27:51 INFO mapred.JobClient:     Map output records=49
09/03/03 10:27:51 INFO mapred.JobClient:     Reduce input records=49

 

 

#11 테스트 결과 확인

좀 익숙해져야 하는게. bin/hadoop fs -ls  나 bin/hadoop dfs -cat 이런거다.

마치 CDROM에 dev 장치에 mount해서 파일 복사해오는 느낌이랄까??

 

/share/hadoop-0.18.3] bin/hadoop fs -get output output
/share/hadoop-0.18.3] cat output/*

또는

/share/hadoop-0.18.3] bin/hadoop fs -ls output/  해서 결과파일이 있는지 확인한다. 파일크기가 0이면 내가 잘못했음을 의미한다.

/share/hadoop-0.18.3] bin/hadoop fs -cat output/part-00000

 

결과값 확인

 

3       dfs.class
3       dfs.
2       dfs.name.dir
2       dfs.data.dir
2       dfs.period
1       dfs.http.address
1       dfs.balance.bandwidth
1       dfs.block.size
1       dfs.blockreport.initial
1       dfs.blockreport.interval
1       dfs.client.block.write.retries
1       dfs.client.buffer.dir
1       dfs.datanode.address
1       dfs.datanode.dns.interface
1       dfs.datanode.dns.nameserver
1       dfs.datanode.du.pct
1       dfs.datanode.du.reserved
1       dfs.datanode.handler.count
1       dfs.datanode.http.address
1       dfs.datanode.https.address
1       dfs.datanode.ipc.address
1       dfs.default.chunk.view.size
1       dfs.df.interval
1       dfs.file
1       dfs.heartbeat.interval
1       dfs.hosts
1       dfs.hosts.exclude
1       dfs.https.address
1       dfs.impl
1       dfs.max.objects
1       dfs.namenode.decommission.interval
1       dfs.namenode.decommission.interval.
1       dfs.namenode.decommission.nodes.per.interval
1       dfs.namenode.handler.count
1       dfs.namenode.logging.level
1       dfs.permissions
1       dfs.permissions.supergroup
1       dfs.replication
1       dfs.replication.consider
1       dfs.replication.interval
1       dfs.replication.max
1       dfs.replication.min
1       dfs.replication.min.
1       dfs.safemode.extension
1       dfs.safemode.threshold.pct
1       dfs.secondary.http.address
1       dfs.servers
1       dfs.web.ugi
1       dfsmetrics.log

 

잘 모르는 게 있으면, 이곳을 참조하자. 여기서 참조해서 나도 한 거라서^^

 

http://ausgang.egloos.com/1806517
http://tida.tistory.com/
http://hadoop.apache.org/core/docs/current/quickstart.html
http://www.joinc.co.kr/modules/moniwiki/wiki.php/JCvs/Search/Document/nutch/Hadoop
http://hadoop.apache.org/core/docs/current/cluster_setup.html
http://www.hadoop.or.kr/?mid=lecture&document_srl=406
http://public.yahoo.com/gogate/hadoop-tutorial/start-tutorial.html

 

 

출처는 꼭 남기시길..^^

'general java' 카테고리의 다른 글

JMX에서의 Internalization  (0) 2009.03.25
toString 구현을 쉽게 구현하기  (0) 2009.03.25
javax.mail.Transport.send 메소드 사용시 block되는 현상  (0) 2009.03.24
sping 2.0 aop @aspect 사용 예제  (0) 2009.03.06
Hadoop 사용 예제  (0) 2009.03.04
Hadoop 설치 하기  (0) 2009.03.04
Posted by 김용환 '김용환'
TAG 영화

댓글을 달아 주세요