wget spider 옵션

wget --spider http://www.naver.com
--14:23:14--  http://www.naver.com/
           => `index.html'
Resolving www.naver.com... 222.122.195.6, 202.131.29.70
Connecting to www.naver.com|222.122.195.6|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
200 OK

header정보만 받아서 보여주는 기능이 spider 임.

http://linux.die.net/man/1/wget

--spider

When invoked with this option, Wget will behave as a Web spider, which means that it will not download the pages, just check that they are there. For example, you can use Wget to check your bookmarks:

wget --spider --force-html -i bookmarks.html

This feature needs much more work for Wget to get close to the functionality of real web spiders.