Tomcat 에서 protocol을 org.apache.coyote.http11.Http11NioProtocol 을 사용하는 상황에서 응답을 무시하거나 505 HTTP Version Not Found Exception이 발생되었다.

6.0.20버젼으로 다운그레이드하니 Exception 발생이 안된다..

원인은 Nio connnector 연결 관련해서 처리 코드가 잘못하였다. 그래서, 505 Http Version Not Found Exception이 발생되었을 뿐 아니라. Async Http 연결을 처리하는 부분에서 문제가 있었다.

관련 50505 버그는 Blank responses (이슈 https://issues.apache.org/bugzilla/show_bug.cgi?id=50072) 문제와 함께 처리되었다.

문제가 되는 버젼은  다음과 같다.
- tomcat 6.0.21~29, tomcat 7.0.1~ 7.0.4까지

 톰캣은 6.0.30와 7.0.5에서 패치되었다.

http://tomcat.apache.org/tomcat-6.0-doc/changelog.html

50072: NIO connector can mis-read request line if not sent in a single packet. (markt/kkolinko)

http://tomcat.apache.org/tomcat-7.0-doc/changelog.html

50072: Fix issues when using a non-blocking read for the request line with the NIO connector that could result in the request line being mis-read. (markt)

패치 내용은 소켓상태에 대한 처리와 패킷 파싱하는 부분이다.

  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.

Tomcat 5 Startup Sequence

Sequence 1. Start from Command Line
Class: org.apache.catalina.startup.Bootstrap
What it does:
	a) Set up classloaders 
		commonLoader (common)-> System Loader
		sharedLoader (shared)-> commonLoader -> System Loader
		catalinaLoader(server) -> commonLoader -> System Loader
	b) Load startup class (reflection)
		org.apache.catalina.startup.Catalina
		setParentClassloader -> sharedLoader
		Thread.contextClassloader -> catalinaLoader
	c) Bootstrap.daemon.init() complete
	
Sequence 2. Process command line argument (start, startd, stop, stopd)
Class: org.apache.catalina.startup.Bootstrap (assume command->start)
What it does: 
	a) Catalina.setAwait(true);
	b) Catalina.load()
		b1) initDirs() -> set properties like 
		                  catalina.home
		                  catalina.base == catalina.home (most cases)
		b2) initNaming
			setProperty(javax.naming.Context.INITIAL_CONTEXT_FACTORY,
				    org.apache.naming.java.javaURLContextFactory ->default)
		b3) createStartDigester() 
			Configures a digester for the main server.xml elements like
			org.apache.catalina.core.StandardServer (can change of course :)
			org.apache.catalina.deploy.NamingResources
				Stores naming resources in the J2EE JNDI tree
			org.apache.catalina.LifecycleListener
				implements events for start/stop of major components
			org.apache.catalina.core.StandardService
				The single entry for a set of connectors,
				so that a container can listen to multiple connectors
				ie, single entry
			org.apache.coyote.tomcat5.CoyoteConnector
				Connectors to listen for incoming requests only
			It also adds the following rulesets to the digester
				NamingRuleSet
				EngineRuleSet
				HostRuleSet
				ContextRuleSet
		b4) Load the server.xml and parse it using the digester
		    Parsing the server.xml using the digester is an automatic
		    XML-object mapping tool, that will create the objects defined in server.xml
		    Startup of the actual container has not started yet.
		b5) Assigns System.out and System.err to the SystemLogHandler class
		b6) Calls initialize on all components, this makes each object register itself with the 
		    JMX agent.
		    During the process call the Connectors also initialize the adapters.
		    The adapters are the components that do the request pre-processing.
		    Typical adapters are HTTP1.1 (default if no protocol is specified,
		    org.apache.coyote.http11.Http11Protocol)
		    AJP1.3 for mod_jk etc.

	c) Catalina.start()
		c1) Starts the NamingContext and binds all JNDI references into it
		c2) Starts the services under <Server> which are:
			StandardService -> starts Engine (ContainerBase ->Logger,Loader,Realm,Cluster etc)
		c3) StandardHost (started by the service)
				Configures a ErrorReportValvem to do proper HTML output for different HTTP 
				errors codes
				Starts the Valves in the pipeline (at least the ErrorReportValve)
				Configures the StandardHostValve, 
					this valves ties the Webapp Class loader to the thread context
					it also finds the session for the request
					and invokes the context pipeline
				Starts the HostConfig component
					This component deploys all the webapps
						(webapps & conf/Catalina/localhost/*.xml)
					Webapps are installed using the deployer (StandardHostDeployer)
					The deployer will create a Digester for your context, this digester
					will then invoke ContextConfig.start()
						The ContextConfig.start() will process the default web.xml (conf/web.xml)
						and then process the applications web.xml (WEB-INF/web.xml)
						
		c4) During the lifetime of the container (StandardEngine) there is a background thread that 
		    keeps checking if the context has changed. If a context changes (timestamp of war file, 
		    context xml file, web.xml) then a reload is issued (stop/remove/deploy/start)
		    
	d) Tomcat receives a request on an HTTP port
	    d1) The request is received by a separate thread which is waiting in the PoolTcpEndPoint 
	         class. It is waiting for a request in a regular ServerSocket.accept() method.
	         When a request is received, this thread wakes up.
	    d2) The PoolTcpEndPoint assigns the a TcpConnection to handle the request. 
	        It also supplies a JMX object name to the catalina container (not used I believe)
	    d3) The processor to handle the request in this case is Coyote Http11Processor, 
	        and the process method is invoked.
	        This same processor is also continuing to check the input stream of the socket
	        until the keep alive point is reached or the connection is disconnected.
	    d4) The HTTP request is parsed using an internal buffer class (Coyote Http11 Internal Buffer)
	        The buffer class parses the request line, the headers, etc and store the result in a 
	        Coyote request (not an HTTP request) This request contains all the HTTP info, such
	        as servername, port, scheme, etc.
	    d5) The processor contains a reference to an Adapter, in this case it is the 
	        Coyote Tomcat 5 Adapter. Once the request has been parsed, the Http11 processor
	        invokes service() on the adapter. In the service method, the Request contains a 
	        CoyoteRequest and CoyoteRespons (null for the first time)
	        The CoyoteRequest(Response) implements HttpRequest(Response) and HttpServletRequest(Response)
	        The adapter parses and associates everything with the request, cookies, the context through a 
	        Mapper, etc
	    d6) When the parsing is finished, the CoyoteAdapter invokes its container (StandardEngine)
	        and invokes the invoke(request,response) method.
	        This initiates the HTTP request into the Catalina container starting at the engine level
	    d7) The StandardEngine.invoke() simply invokes the container pipeline.invoke()
	    d8) By default the engine only has one valve the StandardEngineValve, this valve simply
	        invokes the invoke() method on the Host pipeline (StandardHost.getPipeLine())
	    d9) the StandardHost has two valves by default, the StandardHostValve and the ErrorReportValve
	    d10) The standard host valve associates the correct class loader with the current thread
	         It also retrieves the Manager and the session associated with the request (if there is one)
	         If there is a session access() is called to keep the session alive
	    d11) After that the StandardHostValve invokes the pipeline on the context associated
	         with the request.
	    d12) The first valve that gets invoked by the Context pipeline is the FormAuthenticator
	         valve. Then the StandardContextValve gets invoke.
	         The StandardContextValve invokes any context listeners associated with the context.
	         Next it invokes the pipeline on the Wrapper component (StandardWrapperValve)
	    d13) During the invocation of the StandardWrapperValve, the JSP wrapper (Jasper) gets invoked
	         This results in the actual compilation of the JSP.
	         And then invokes the actual servlet.
	e) Invocation of the servlet class

Download Type: Source Code (38.33 MB | .zip file)

Download Now

1)  \p{Graph}  
     A visible character: [\p{Alnum}\p{Punct}]
     alphanumeric character && Punctuation: One of !"#$%&'()*+,-./:;<=>?@[\]^_`{|}~

 2) \s
    A whitespace character: [ \t\n\x0B\f\r]

3) &&
    Any letter except an uppercase letter (subtraction)

4) [^,#:=]

이렇게 나눠볼 수 있을 것이다.

읽어보면 space와 숫자문자와 ^,#:= 를 제외한 특수문자의 셋(zero or more time) parameter로 지원한다.


[수정된 소스] ; 언뜻 보면 [ ] 때문에 헛갈린다.

1) a-zA-Z0-9

2)\\.

3) \\]

4)\\[

5)_'\\s]+

이렇게 분석해놔야 조금 이해가 되었다.

General하게 받지 않고, 특정 패턴만 들어오게 했다.

Webwork는 버려진지 오래되어서, 따로 패치는 안될 것 같다. (이래서 Strtus2의 미래가 불안하다. 담당 개발자 사라지면 버려지지 않을까? ) 대신 Struts2로 가서 패치는 했다.


하지만, 여기서 잘 봐야하는 문제는.. 기존의 Webwork 서비스에서는 잘 돌았는데, 패치 버젼 사용 및 위의 내용을 그대로 패치 이후에 문제가 생길 수 있다. 빠질 수 있는 패턴을 고민해야 봐야한다.

특히 특수 문자 쪽으로 봐야하는데..

기존에 파라미터에 - (bar)를 사용하고 있는데, 패치 이후로는 문제를 해결하지 못할 수 있다. 헐... 
Struts2 그냥 믿지 마시라.. 꼼꼼히 살펴볼 것.

-(bar)를 추가해서 적용할 필요가 있다.
"[a-zA-Z0-9\\.\\]\\[_'-\\s]+"

3.6.2. The setRequestHeader() method

client . setRequestHeader(header, value)

Appends an header to the list of author request headers or if the header is already in the author request headers its value appended to.

Throws an INVALID_STATE_ERR exception if the state is not OPENED or if the send() flag is true.

Throws a SYNTAX_ERR exception if header is not a valid HTTP header field name or if value is not a valid HTTP header field value.

As indicated in the algorithm below certain headers cannot be set and are left up to the user agent. In addition there are certain other headers the user agent will take control of if they are not set by the author as indicated at the end of the send() method section.

When the setRequestHeader(header, value) method is invoked, the user agent must run these steps:

1. If the state is not OPENED raise an INVALID_STATE_ERR exception and terminate these steps.

2. If the send() flag is true raise an INVALID_STATE_ERR exception and terminate these steps.

3. If any code point in header is higher than U+00FF LATIN SMALL LETTER Y WITH DIAERESIS or after deflating header it does not match the field-name production raise a SYNTAX_ERR exception and terminate these steps. Otherwise let header be the result of deflating header.

4. If any code point in value is higher than U+00FF LATIN SMALL LETTER Y WITH DIAERESIS or after deflating value it does not match the field-value production raise a SYNTAX_ERR exception and terminate these steps. Otherwise let value be the result of deflating value.

   The empty string is legal and represents the empty header value.

5. Terminate these steps if header is a case-insensitive match for one of the following headers:

   • Accept-Charset
   • Accept-Encoding
   • Connection
   • Content-Length
   • Cookie
   • Cookie2
   • Content-Transfer-Encoding
   • Date
   • Expect
   • Host
   • Keep-Alive
   • Referer
   • TE
   • Trailer
   • Transfer-Encoding
   • Upgrade
   • User-Agent
   • Via

   … or if the start of header is a case-insensitive match for Proxy- or Sec- (including when header is just Proxy- or Sec-).

   The above headers are controlled by the user agent to let it control those aspects of transport. This guarantees data integrity to some extent. Header names starting with Sec- are not allowed to be set to allow new headers to be minted that are guaranteed not to come from XMLHttpRequest.

6. If header is not in the author request headers list append header with its associated value to the list and terminate these steps.

7. If header is in the author request headers list either use multiple headers, combine the values or use a combination of those (section 4.2, RFC 2616). [RFC2616]

See also the send() method regarding user agent header handling for caching, authentication, proxies, and cookies.

// The following script:
var client = new XMLHttpRequest();
client.open('GET', 'demo.cgi');
client.setRequestHeader('X-Test', 'one');
client.setRequestHeader('X-Test', 'two');
client.send();

// ...would result in the following header being sent:
...
X-Test: one, two
...

http://code.google.com/intl/ko-KR/closure/

input 파라미터 검사를 아파치 모듈인 mod_security를 이용한다.
http://www.securenet.or.kr/main.jsp   KISA의 securenet 에서 공개 된 정보를 가지고 한다.

물론, 웹 서버에서 코딩으로 막으면 된다. 하지만 자바 String 비교는 상대적으로 부하를 일으킬 수 있다..
아파치단에서 막아도 될 듯..

 내용을 살펴보면, 다음과 같다.

# 리눅스 명령어 사용못하게 막기..

SecRule ARGS ";[[:space:]]*(ls|pwd|wget|cd)" "msg:'Command execution attack'"
SecRule REQUEST_URI "(perl|lynx|mkdir|cmd|lwp-(download|request|mirror|rget))" "msg:'Command execution attack'"
SecRule REQUEST_URI "(uname|net(stat|cat)|curl|telnet|gcc|rm\-[a-z|A-Z])" "msg:'Command execution attack'"

# 일부 XSS 필터 금지..
SecRule ARGS "alert[[:space:]]*\(" "msg:'XSS attack'"
SecRule ARGS "&#[[0-9a-fA-F]]{2}" "msg:'XSS attack'"
SecRule ARGS "eval[[:space:]]*\(" "msg:'XSS attack'"
SecRule ARGS "onKeyUp" "msg:'XSS attack'"
SecRule ARGS "\x5cx[0-9a-fA-F]{2}" "msg:'XSS attack'"

 CSRF 공격을  통해서 옥션 해킹, 네이버 카페 지기 강퇴 등이 있었다고 한다.


사례)
공격자가 특정 사이트의 게시물에 아래와 같이 img 태그를 이용한다.

동일 사이트의 아래와 같이 예금을 인출 하는 악성 코드를 심어 넣는다. img의 src가 dynamic 호출되서 불려지는 것을 노림. 실제 웹 싸이트에서는 png, gif, jpg처럼 안쓰는 경우가 있을 수 있다.

 <img src="http://bank.example/withdraw?account=bob&amount=1000000&for=mallory">

 피해자는 동일 사이트에 로그인 된 상태로 악성 코드가 심겨져 있는 게시물을 보는 순간 예금 인출 URL을 GET으로 동일 사이트에 보내게 된다.

 서버에서는 공격에 의한 요청인지 정상적인 요청인지를 구분할 수 있는 방법이 없기 때문에 공격을 당하게 된다.

해결사례)
1. GET 방식 요청은 허용하지 않는다.
2. referer white list 를 관리한다.
3. 특정 패턴으로 오는 것만 필터링한다.
4. Tomcat 7에서는 아래 필터를 사용한다.

http://tomcat.apache.org/tomcat-7.0-doc/api/org/apache/catalina/filters/CsrfPreventionFilter.html