This site is now 100% read-only, and retired.

XML logo

httping, wget, curl: none does what I want!
Posted by endecotp on Tue 14 Apr 2009 at 20:10
Tags: none.
I was expecting that this would be easy, but I haven't yet found the right tool for the job. I have a long list of URLs, and for each one I want to do an HTTP HEAD request to retrieve the headers. If the initial request gets a redirect response I want to follow the redirect so that I end up with probably either a 200 OK or a 404 not found, and I want to log the content-type header in that final response. It's important that I do a HEAD request not a GET because these are large files, and I have a lot of them to look at.

I was expecting that Wget would have an option to send a HEAD, but it doesn't seem to. Neither does Curl. httping sends a HEAD, but it doesn't follow the redirect.

Can anyone point me in the direction of a simple utility that does this, or shall I write my own?

 

Comments on this Entry

Re: httping, wget, curl: none does what I want!
Posted by Steve (2001:0xx:0xx:0xxx:0xxx:0xxx:xx) on Tue 14 Apr 2009 at 20:50
[ View Weblogs ]
skx@gold:~$ HEAD www.debian-administration.org
200 OK
Connection: close
Date: Tue, 14 Apr 2009 19:49:21 GMT
Server: Apache/2.2.9
Vary: Accept-Encoding
Content-Type: text/html
Client-Date: Tue, 14 Apr 2009 19:49:21 GMT
Client-Peer: 89.16.161.98:80
Client-Response-Num: 1
Set-Cookie: CGISESSID=0ddfeed40pies381656665f1928ea75e49d; path=/; expires=Tue, 21-Apr-2009 19:49:21 GMT; HttpOnly
X-Pad: avoid browser bug

The HEAD command comes from the libwww-perl package.

Steve

[ Parent ]

Re: httping, wget, curl: none does what I want!
Posted by endecotp (86.6.xx.xx) on Tue 14 Apr 2009 at 20:56
[ View Weblogs ]
Thanks Steve! I've checked and yes it does seem to follow redirects. Great!

[ Parent ]